NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·May 25·8 min read
OnePred: Next-Query Prediction via Recursive Intent Memory in Multi-Turn Conversations
TL;DR
Conversational AI systems respond reactively to user queries but cannot anticipate what users will ask next. OnePred predicts the next user query by maintaining a compressed memory of the user's evolving intent rather than storing full dialogue history, reducing token consumption 22Ă— while improving prediction accuracy.
✦ Why It Matters
Engineers can build more efficient conversational systems that anticipate user needs while reducing computational costs and token consumption.
Key Takeaways
- Implement recursive intent memory instead of full history storage to reduce computational overhead in dialogue systems.
- Use two-stage reinforcement learning to train models on both prediction accuracy and compression efficiency.
- Evaluate next-query prediction systems on NQP-Bench to establish consistent performance baselines.
- Expect 22Ă— token reduction while maintaining or improving prediction quality on longer conversations.
Full Summary
Current large language model conversational systems process multi-turn dialogues (extended back-and-forth exchanges) reactively, responding only after users submit queries. Next-query prediction—anticipating a user's subsequent request based on prior conversation—enables proactive interaction but faces a fundamental trade-off: including full dialogue history consumes tokens linearly with conversation length, while truncating history loses important context about topic shifts and unresolved needs.
OnePred addresses this by maintaining a recursively updated intent memory (a compressed representation of the user's evolving goals and interests) as the sole cross-turn context, keeping per-turn computational cost constant regardless of conversation length. The model is trained via two-stage reinforcement learning: first learning what to predict, then learning what to compress into the intent chain.
Experiments on NQP-Bench (a new benchmark spanning three diverse conversation subsets) show OnePred reduces token consumption by up to 22Ă— compared to full-history baselines while consistently achieving better prediction quality, with larger improvements on longer conversations.