NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·Jun 10·5 min read
Divide and Cooperate: Role-Decomposed Multi-Agent LLM Training with Cross-Agent Learning Signals
TL;DR
Existing language models struggle with multi-step reasoning due to conflicting roles in evidence acquisition and answer generation. The DAC (Divide and Cooperate) framework separates these tasks into dedicated agents, enhancing training through role-specific learning signals.
✦ Why It Matters
Engineers can leverage role-decomposed training to improve the efficiency and performance of AI models in complex tasks.
Key Takeaways
- Implement role-decomposed training frameworks for complex tasks.
- Utilize parameter-efficient techniques like LoRA for model training.
- Explore cross-agent learning signals to enhance model robustness.
- Test multi-agent approaches on diverse question answering benchmarks.
Full Summary
Modern language models often combine evidence gathering and answer generation in a single policy, leading to inefficiencies and training challenges. DAC (Divide and Cooperate) introduces a multi-agent framework that decomposes these tasks into two specialized agents: a search agent focused on retrieving evidence and a generator responsible for producing answers.
The generator also verifies the sufficiency of the evidence, providing feedback to the search agent through structured learning signals. This approach mitigates the credit assignment problem, where actions may be penalized despite contributing to overall success.
Experiments on various question answering benchmarks demonstrate that DAC, utilizing parameter-efficient LoRA modules, achieves superior performance compared to traditional models that require full fine-tuning. These results suggest that role decomposition can significantly enhance the robustness and efficiency of language model training.