EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hacker News·Jul 25·5 min read
UK AISI / Caisi Preliminary Assessment of Kimi K3's Cyber Capabilities
TL;DR
UK AISI and CAISI assessed Moonshot AI's Kimi K3 model, focusing on its cyber capabilities. Preliminary results indicate Kimi K3 shows promise in exploit development but has a larger confidence interval than competing models.
✦ Why It Matters
Evaluate Kimi K3's capabilities for your cybersecurity projects to stay ahead in exploit development.
Key Takeaways
- Assess AI models using benchmarks like ExploitBench for exploit development.
- Understand the limitations of Kimi K3 in achieving arbitrary code execution.
- Consider the implications of Kimi K3's performance in real-world cyber scenarios.
- Explore the potential of AI models in autonomous cyberattacks on vulnerable systems.
Full Summary
Kimi K3, released on July 16, 2026, underwent a preliminary assessment by the UK Artificial Intelligence Security Institute and the U.S. Center for AI Standards and Innovation to evaluate its cyber capabilities.
The evaluation utilized the ExploitBench benchmark, which measures a model's ability to develop exploits based on recent vulnerabilities. Kimi K3 achieved a score of 32% in exploit development, surpassing GLM-5.2's 24%, but failed to execute arbitrary code in any of the 41 tasks tested.
In a simulated cyber range, Kimi K3 reached an average of 17 out of 32 steps, significantly lower than leading U.S. models, which averaged 28.5 steps. Despite its limitations, Kimi K3 demonstrated the ability to autonomously attack weakly defended systems in one out of ten attempts.
These results highlight both the potential and the current limitations of Kimi K3 in real-world cyber scenarios.