EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Lobsters·Aug 2·5 min read
I'm (mostly) picking models on speed now, not intelligence
TL;DR
Current model selection is shifting from prioritizing intelligence to speed, with Opus 4.6 being sufficient for daily tasks. The author emphasizes the importance of fast software for user satisfaction.
✦ Why It Matters
Evaluate your current tools and prioritize speed to improve your workflow today.
Key Takeaways
- Prioritize AI models based on speed for daily tasks.
- Monitor emerging models like GLM5.2 for competitive speeds.
- Be aware of local hardware limitations affecting performance.
- Prepare for potential price reductions in AI model offerings.
Full Summary
The author notes a significant change in how AI models are chosen, emphasizing speed over raw intelligence. Models around the Opus 4.6 level are deemed 'smart enough' for tasks like coding and research, leading to a preference for faster models.
The article discusses the importance of speed, highlighting that a model's output speed, measured in tokens per second (tok/s), significantly impacts user experience. For instance, 100-200 tok/s is considered fast, while anything below 50 tok/s feels slow.
The emergence of new models, such as GLM5.2, offers a variety of speeds, prompting competition among providers. However, limitations arise as local hardware and tool calls can bottleneck performance, suggesting that speed improvements may have diminishing returns.
The author anticipates a price war in the AI model market, driven by new GPU technologies and competitive pricing strategies.