NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·3h ago
Security
Cloudflare Blog·May 18·5 min read
Project Glasswing: what Mythos showed us
TL;DR
Cloudflare tested Mythos Preview, a security-focused large language model (LLM) from Anthropic designed to find code vulnerabilities, against over fifty internal repositories. Mythos Preview represents a significant advancement beyond previous general-purpose AI models, offering different capabilities for identifying security issues.
✦ Why It Matters
Engineers can understand that specialized security LLMs like Mythos Preview offer fundamentally different vulnerability detection capabilities than general AI models, requiring new deployment strategies.
Key Takeaways
- Mythos Preview is a specialized tool, not an incremental improvement over general-purpose models.
- Test security models against your own repositories to understand attacker capabilities with latest AI.
- Specialized security LLMs require architectural changes before production deployment at scale.
- Evaluate models on actual codebases rather than benchmarks for practical security insights.
Full Summary
Cloudflare conducted Project Glasswing, an evaluation of security-focused large language models (LLMs—AI systems trained on vast text data to understand and generate code) on their infrastructure. The team tested Mythos Preview, a specialized LLM from Anthropic designed specifically for vulnerability detection, against more than fifty internal code repositories.
Unlike general-purpose frontier models (state-of-the-art AI systems trained broadly), Mythos Preview represents a fundamentally different category of tool optimized for security analysis rather than general tasks. The evaluation methodology involved deploying the model across diverse codebases to assess both its detection capabilities and practical limitations.
Key findings indicated Mythos Preview delivers substantial improvements over previous approaches, though the article notes specific architectural and process changes are necessary before such models can operate reliably at enterprise scale. The work demonstrates both the promise of specialized security models and the engineering challenges in production deployment.