Reimagining service delivery in the agentic era with Google Public Sector
cloud.google.com·19h ago
AI Research
arXiv CS.AI·Jun 10·5 min read
Failure Modes of Deep Multi-Agent RL in Asynchronous Pricing: Reproducible Triggers, Trace Diagnostics, and a Partial Fix
TL;DR
Deep multi-agent reinforcement learning (RL) in pricing markets faces issues like tacit cartel formation and actor-critic instability. The authors developed a continuous-time multi-agent RL benchmark to analyze these failures and proposed a partial fix through asynchrony and observation latency.
✦ Why It Matters
Engineers can apply insights on asynchrony and latency to improve multi-agent RL systems in pricing scenarios.
Key Takeaways
- Investigate tacit cartel formation in multi-agent systems.
- Implement asynchronous methods to reduce collusion.
- Monitor actor-critic stability at high event rates.
- Quantify collusion using a defined index for better diagnostics.
How It Works
The study uses a continuous-time multi-agent reinforcement learning (CT-MARL) benchmark to simulate pricing markets. By introducing asynchrony and observation latency, the researchers were able to reduce collusion among agents.
The collusion index was measured to quantify the effectiveness of the proposed solutions.
⚠ The Catch
The proposed fix only partially addresses the collusion issue, as the collusion index remains above the ideal level, and it does not resolve the instability caused by high event rates in the actor-critic framework.