Announcing Native BM25 Ranking in AlloyDB and Cloud SQL
cloud.google.com·1d ago
AI Research
arXiv CS.AI·Jun 9·5 min read
Fast LLM-Based Semantic Filtering: From a Unified Framework to an Adaptive Two-Phase Method
TL;DR
Evaluating yes/no predicates over document collections using large language models (LLMs) is resource-intensive. An adaptive two-phase method was developed to enhance semantic filtering by combining a fast proxy with an oracle.
✦ Why It Matters
Engineers can implement this adaptive method to enhance efficiency in LLM-based data processing tasks.
Key Takeaways
- Utilize model-free clustering for initial document filtering.
- Train proxies with LLM confidence as soft labels.
- Apply selective calibration to reduce costs.
- Achieve high accuracy with improved processing speed.
How It Works
The method begins with model-free clustering to quickly filter documents, followed by an online-trained proxy LLM that only activates when necessary. This two-phase approach allows for shared oracle calls across both phases, optimizing the use of resources.
The proxy is trained using the LLM's per-document confidence, which provides richer information than binary labels, enabling better handling of challenging cases.