Agentic data science with the Data Agent Kit
cloud.google.com·6h ago
AI Research
arXiv CS.AI·12d ago·5 min read
Support-Set Target Leakage in Relational Foundation Models during In-Context Learning: Impact, Detection, and Mitigation
TL;DR
Support-set target leakage in relational foundation models can lead to significant prediction errors during in-context learning. This study identifies and mitigates this issue, revealing that target-table leakers have the most detrimental impact on model performance.
✦ Why It Matters
Evaluate your models for support-set target leakage to improve prediction accuracy today.
Key Takeaways
- Identify and analyze support-set target leakage in your models.
- Use Integrated Gradients to rank and remove suspicious features.
- Focus on mitigating the impact of target-table leakers.
- Test model performance across different relational distances.
How It Works
Support-set target leakage occurs when features derived from the target are included in the labeled support set but not in the query. This misalignment can mislead the model during inference, as it relies on context that is not available in real-world scenarios.
The study constructs various synthetic leaker types to simulate different leakage scenarios and evaluates their impact on model performance.