Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·Jun 12·5 min read
Versioned Late Materialization for Ultra-Long Sequence Training in Recommendation Systems at Scale
TL;DR
Recommendation systems face challenges with ultra-long user interaction histories due to data redundancy from the 'Fat Row' paradigm. A new technique called versioned late materialization was developed to store user interaction data efficiently and reconstruct sequences during training.
✦ Why It Matters
Engineers can implement versioned late materialization to optimize data storage and improve recommendation model performance.
Key Takeaways
- Adopt versioned late materialization for efficient data handling.
- Normalize user interaction histories to reduce redundancy.
- Implement pipelined I/O prefetching to enhance training speed.
- Utilize this approach for scaling sequence lengths in models.
Full Summary
Deep Learning Recommendation Models (DLRMs) are increasingly requiring longer user interaction histories to improve performance, but the traditional 'Fat Row' approach leads to excessive data storage and input/output (I/O) bottlenecks. To address this, versioned late materialization was introduced, which normalizes and stores user interaction histories in an immutable format, using lightweight pointers to reconstruct sequences on-the-fly during training.
This method ensures consistency between online and offline data processing and optimizes storage for multiple models with varying sequence lengths. The implementation includes disaggregated data preprocessing and pipelined I/O prefetching to minimize latency during sequence reconstruction.
In production environments, this approach has reduced resource usage while allowing for aggressive scaling of sequence lengths, resulting in significant improvements in model quality. The findings suggest that this infrastructure can support advanced recommendation architectures like HSTU and ULTRA-HSTU, making it a valuable tool for engineers and researchers in the field.