Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·May 27·8 min read
Tool-Schema Compression Enables Agentic RAG Under Constrained Context Budgets
TL;DR
Agentic RAG systems (AI agents with retrieval tools) struggle when tool definitions consume limited context windows needed for document retrieval. Researchers conducted the first systematic study of tool-context trade-offs across 14 models using TSCG (Tool-Schema Compression with conservative-profile settings), testing 6,566 API calls at three context budgets.
✦ Why It Matters
Engineers can now compress tool schemas to fit more tools or retrieval content within fixed context limits without sacrificing model capability.
Key Takeaways
- Apply tool-schema compression to reduce schema size by up to 50%.
- Expect significant performance boosts in RAG systems under 8K token limits.
- Utilize compressed schemas to prevent context overflow issues.
- Leverage findings to optimize AI models for constrained environments.
How It Works
Tool-schema compression employs a conservative-profile approach to reduce the number of tokens used by tool definitions. By compressing these schemas, the system can fit more relevant information within the limited context window, thus maintaining the functionality of RAG systems even when many tools are defined.