Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·Jun 10·5 min read
STORM: Stepwise Token Optimization with Reward-Guided Beam Search
TL;DR
Existing retrieval methods struggle with vocabulary mismatch and inefficiencies when adapting to new models. STORM (Stepwise Token Optimization with Reward-Guided Beam Search) is introduced as a self-supervised framework that optimizes lexical query expansion by scoring candidate terms against a BM25 index.
✦ Why It Matters
Engineers can leverage STORM for efficient and effective query expansion in retrieval systems without heavy infrastructure.
Key Takeaways
- Implement STORM for efficient lexical query expansion.
- Utilize BM25 for fast retrieval alongside STORM.
- Explore zero-shot capabilities of STORM in multilingual contexts.
- Consider STORM as a lightweight alternative to dense retrieval models.
Full Summary
Modern information retrieval systems often rely on dense neural models that require extensive indexing, which can be inefficient and inflexible. Lexical retrievers like BM25 are efficient but face challenges with vocabulary mismatch, while query rewriting techniques can produce ineffective terms.
STORM addresses these issues by employing a self-supervised framework that optimizes query expansion through a reward-guided beam search. It scores candidate terms based on their effectiveness in retrieval, pruning low-reward options to focus on effective vocabulary.
In experiments across TREC DL and BEIR datasets, STORM demonstrated the ability to match or surpass the performance of competitive large language model rewriters, even with smaller model sizes (0.6B-8B parameters). Additionally, STORM showed strong zero-shot performance across 18 languages, outperforming dedicated multilingual dense retrievers.
This positions STORM as a lightweight and competitive alternative to traditional dense retrieval methods.