Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·28d ago·5 min read
OneLA: Scaling Linear-Attention Decoding to Large Beams in Generative Recommendation
TL;DR
OneLA is a novel linear-attention decoding framework designed for generative recommendation tasks, achieving significant speedups and reduced memory usage. It enables efficient large-beam decoding by utilizing shared states and compact transition records.
✦ Why It Matters
Implement OneLA to enhance the efficiency of your generative recommendation systems today.
Key Takeaways
- Explore OneLA for efficient large-beam decoding in your projects.
- Reduce memory usage by adopting shared state representations.
- Utilize compact transition records to streamline data movement.
- Achieve faster decoding speeds with the lightweight ancestry index.
How It Works
OneLA leverages a shared prompt-derived state to represent all beam states, which minimizes memory overhead. It maintains append-only records of divergent transitions, allowing for efficient updates without reconstructing full states.
A fused GPU kernel further enhances performance by reusing the shared state across multiple beams.