We’re launching Lyria 3.5 in Google Flow Music, with advances across musicality, lyrics, vocals, and creative control
deepmind.google·6d ago
AI Engineering
Hacker News·14d ago·5 min read
Kimi K3: second only to Fable 5 on AA-Briefcase
TL;DR
Kimi K3, a 2.8 trillion parameter model from Moonshot AI, ranks second on the AA-Briefcase benchmark, achieving an Elo score of 1543, just behind Fable 5. Despite its high performance, it is more expensive to run than Opus 4.8 and takes nearly an hour per task.
✦ Why It Matters
Evaluate Kimi K3 for your next project requiring advanced analytical capabilities in knowledge work.
Key Takeaways
- Consider Kimi K3 for high-level analytical tasks.
- Be aware of the high operational costs per task.
- Evaluate the trade-off between performance and presentation quality.
- Monitor advancements in competing models for better options.
Full Summary
Kimi K3 is a new AI model developed by Moonshot AI, featuring 2.8 trillion parameters and scoring 57 on the Artificial Analysis Intelligence Index. It achieved an Elo score of 1543 on the AA-Briefcase benchmark, which evaluates models on realistic knowledge work tasks like spreadsheets and presentations.
This score represents a significant improvement of 727 points over its predecessor, Kimi K2.6. However, Kimi K3 is among the most expensive models to operate, costing $10.57 per task and averaging 56.4 minutes to complete each task.
Its performance is strong in analytical quality but weaker in presentation quality compared to competitors. The high cost and time per task are driven by the model's token pricing and output requirements, making it a consideration for organizations looking to implement advanced AI solutions.