Announcing Native BM25 Ranking in AlloyDB and Cloud SQL
cloud.google.com·1d ago
AI Research
arXiv CS.AI·Jun 10·5 min read
CITRAS: Covariate-Informed Transformer for Time Series Forecasting
TL;DR
Time series forecasting often struggles to effectively incorporate covariates, which are external factors influencing predictions. CITRAS, a decoder-only Transformer model, was developed to flexibly integrate multiple target variables and covariates while maintaining strong autoregressive capabilities.
✦ Why It Matters
Engineers can leverage CITRAS to enhance forecasting models by effectively integrating covariates for improved accuracy.
Key Takeaways
- Utilize CITRAS for improved time series forecasting accuracy.
- Incorporate both observed and known covariates in your models.
- Leverage Key-Value Shift for aligning future covariates.
- Apply Attention Score Smoothing to enhance dependency modeling.
How It Works
CITRAS employs a decoder-only Transformer architecture that integrates multiple target variables and covariates. The Key-Value Shift mechanism allows the model to incorporate future known covariates by aligning them with target variables based on their concurrent dependencies.
Attention Score Smoothing refines local dependencies into global relationships, enhancing the model's ability to capture complex interactions over time.