Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·Jun 12·5 min read
Evoflux: Inference-Time Evolution of Executable Tool Workflows for Compact Agents
TL;DR
Compact language models struggle with executing complex workflows due to limited training data. Evoflux is a new method that evolves executable tool workflows in real-time, improving their reliability.
✦ Why It Matters
Engineers can leverage Evoflux to enhance the reliability of tool workflows in compact AI agents.
Key Takeaways
- Implement Evoflux for real-time workflow evolution.
- Focus on execution-grounded search for better reliability.
- Utilize structured edits to improve tool workflows.
- Explore adaptive intensity and diversity pruning techniques.
Full Summary
Compact language models (LMs) are advantageous for reducing costs and deployment risks but face challenges in executing complex workflows that require dynamic tool discovery and dependency management. Evoflux is introduced as an inference-time evolutionary search method that repairs and evolves executable tool workflows through structured edits and execution feedback.
It employs adaptive intensity, meta-guided redesign, and diversity pruning to enhance workflow reliability. In experiments on MCP-Bench tasks involving 250 tools, Evoflux significantly improved execution feasibility from approximately 3% to a range of 17-24%.
In contrast, other methods like SFT and ReAct either underperformed or showed high variance. These findings suggest that execution-grounded search techniques are more effective when training data is limited, providing a more robust solution for compact agents.