BerriAI / litellm
github.com·3d ago
Developer Tools
Databricks Blog·Sep 9·5 min read
Adaptive Instructed-Retriever: Frontier-Quality Search at 2x Lower Latency
TL;DR
Adaptive Instructed-Retriever combines fast parallel retrieval with effective sequential search, achieving performance comparable to leading models at half the latency.
✦ Why It Matters
Implement Adaptive Instructed-Retriever to enhance your enterprise data search capabilities today.
Key Takeaways
- Explore integrating Adaptive Instructed-Retriever for faster data retrieval.
- Utilize the model's ability to handle complex multi-hop queries.
- Set a fixed upper limit on sequential search steps for efficiency.
- Leverage existing training data from Instructed-Retriever-1 to enhance performance.
Full Summary
Adaptive Instructed-Retriever is a new retrieval model designed to enhance enterprise data search by balancing speed and accuracy. It builds on the earlier Instructed-Retriever-1, which utilized parallel test-time scaling for efficient single-step searches.
The new model introduces a mechanism that allows it to adaptively decide when to employ additional sequential search steps, optimizing for user requests that require deeper evidence gathering. By imposing a fixed limit on sequential steps, it can quickly return results when sufficient evidence is found, while still being capable of multi-hop reasoning for complex queries.
Evaluations show that Adaptive Instructed-Retriever matches the quality of top third-party models while operating at 2x lower latency. This advancement is particularly beneficial for enterprise environments where quick and accurate data retrieval is crucial.