NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·1h ago
AI Research
arXiv CS.AI·May 26·5 min read
Hera: Learning Long-Horizon Coordination for Device-Cloud Collaborative LLM Agents
TL;DR
Large language model (LLM) agents struggle with a device-cloud dilemma, where on-device models are efficient but limited, while cloud models are powerful yet costly. Hera, a step-level coordinator, was developed to optimize the use of device and cloud resources for long-horizon tasks through a two-stage training process involving imitation learning and reinforcement learning.
✦ Why It Matters
Engineers can leverage Hera to improve the efficiency of LLM agents in resource-constrained environments.
Key Takeaways
- Implement a two-stage training approach for LLM agents.
- Use imitation learning for initial routing decisions.
- Apply reinforcement learning to optimize cloud resource usage.
- Evaluate performance in diverse environments to ensure robustness.
How It Works
Hera's two-stage training begins with imitation learning, where the model learns from existing cloud trajectories to make informed routing decisions. This is followed by reinforcement learning, which refines the model's ability to choose actions that maximize task success while minimizing reliance on cloud resources.
By grouping similar states, Hera can effectively update its strategies based on expected returns, leading to more efficient task execution.