EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hacker News·Jul 31·5 min read
Everyone is building LLM routers, we deprecated ours
TL;DR
The Manifest LLM router, designed to optimize model selection for cost efficiency, was deprecated after mixed results highlighted its limitations. Sticking to a single, reliable model is now favored for most applications.
✦ Why It Matters
Consider consolidating your AI model usage to a single, reliable model for better performance.
Key Takeaways
- Evaluate the complexity of tasks beyond the initial prompt.
- Use caching to reduce costs effectively.
- Maintain consistency by sticking to one model during tasks.
- Assess the trade-offs of model selection carefully.
How It Works
The LLM router was designed to classify requests into complexity tiers, directing them to the most cost-effective model. However, it became clear that the initial prompt often lacked sufficient context to accurately determine the task's complexity.
⚠ The Catch
Switching between models during a session led to unpredictability and lower quality outcomes, making it harder to maintain consistent results.