Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·Jul 27·5 min read
From Perturbation Correction to Geometry-Aware Sampling: Sharpness-Guided Equilibrium Sampling for Balanced Flat Minima in Long-Tailed Learning
TL;DR
Sharpness-Guided Equilibrium Sampling (SGS) enhances long-tailed learning by dynamically adjusting mini-batch sampling to favor under-represented classes, improving tail accuracy significantly. On CIFAR-100 LT, SGS-SAM outperformed Focal-SAM by 10.85 points in tail accuracy.
✦ Why It Matters
Implement SGS in your long-tailed learning models to improve performance on under-represented classes today.
Key Takeaways
- Use SGS to enhance sampling for under-represented classes.
- Monitor sharpness estimates to guide mini-batch adjustments.
- Evaluate performance improvements on tail classes in your models.
- Consider SGS as a method for loss-landscape control.
How It Works
SGS modifies the sampling distribution during training by increasing the likelihood of selecting under-represented classes. It uses cumulative class counts and EMA sharpness estimates to identify classes that contribute to sharper loss landscapes, dynamically adjusting the mini-batch composition to promote a more balanced training experience.