Reimagining service delivery in the agentic era with Google Public Sector
cloud.google.com·21h ago
AI Research
arXiv CS.AI·Jun 10·5 min read
CleanPatrick: A Benchmark for Image Data Cleaning
TL;DR
Current benchmarks for image data cleaning are limited by synthetic noise and narrow studies, hindering real-world application. CleanPatrick is introduced as a large-scale benchmark using the Fitzpatrick17k dermatology dataset, which includes 496,377 annotations to identify various data issues.
✦ Why It Matters
Engineers can leverage CleanPatrick to evaluate and improve their image data cleaning methods systematically.
Key Takeaways
- Utilize CleanPatrick for systematic evaluation of image-cleaning strategies.
- Leverage the Fitzpatrick17k dataset for real-world data issues.
- Explore self-supervised models for near-duplicate detection.
- Address challenges in detecting implausible labels in medical images.
How It Works
CleanPatrick employs a ranking approach to detect data issues, using metrics that align with real audit workflows. It aggregates annotations from medical crowd workers and applies an item-response theory model to derive high-quality ground truth, ensuring reliable evaluation of various cleaning methods.