Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·Aug 24·5 min read
C-Score: Beyond Accuracy for Robustness Assessment in Semi-Supervised Learning under Open-World Unlabeled Contamination
TL;DR
C-Score is a new framework designed to assess the robustness of semi-supervised learning (SSL) models in open-world scenarios where unlabeled data may be contaminated. It reveals hidden performance degradation that traditional accuracy metrics fail to detect, emphasizing the need for internal diagnostic signals.
✦ Why It Matters
Incorporate C-Score metrics into your SSL evaluation process to better understand model robustness against unlabeled data contamination.
Key Takeaways
- Use C-Score to evaluate SSL models in open-world scenarios.
- Monitor prediction confidence to detect potential contamination.
- Assess feature representation for better model understanding.
- Align optimization processes between labeled and unlabeled data.
How It Works
C-Score evaluates SSL training behavior through three metrics: PLE and CCI for prediction confidence, Sem-Drift for feature representation deviation, and Grad-Align for optimization compatibility. This multi-faceted approach allows for a comprehensive assessment of model robustness in the presence of OOD samples.