Announcing Native BM25 Ranking in AlloyDB and Cloud SQL
cloud.google.com·1d ago
AI Research
arXiv CS.AI·Jun 12·5 min read
Multi-Agent Reinforcement Learning from Delayed Marketplace Feedback for Objective-Weight Adaptation in Three-Sided Dispatch
TL;DR
Three-sided marketplaces face challenges in optimizing dispatch decisions due to delayed feedback on operational outcomes. A reinforcement learning system was developed at DoorDash to adapt dispatch objective weights using delayed signals, employing a store-level policy that selects multipliers for the dispatch optimizer.
✦ Why It Matters
Engineers can leverage delayed feedback in reinforcement learning to optimize decision-making in complex marketplaces.
Key Takeaways
- Utilize delayed feedback to inform decision-making in dispatch systems.
- Implement store-level policies to balance multiple objectives effectively.
- Train centralized value functions on offline data for better performance.
- Experiment with production switchbacks to validate policy improvements.
How It Works
The system adapts dispatch objective weights by selecting a multiplier that adjusts the tradeoff between delivery quality and batching efficiency. It uses a centralized value function trained on historical data, allowing for offline learning despite the noisy and delayed nature of marketplace feedback.