Agentic data science with the Data Agent Kit
cloud.google.com·6h ago
AI Research
arXiv CS.AI·Aug 26·5 min read
Fidelity Preference, Not Demographic Preference: A Pixel-Level Attribute-Sensitivity Audit of Image Aesthetic/Preference Scorers
TL;DR
A pixel-level audit of four image aesthetic scorers reveals that they prioritize fidelity over demographic attributes, with significant differences between synthetic and real images. This challenges the reliability of synthetic audits in assessing bias.
✦ Why It Matters
Use real image data to validate aesthetic scorers and avoid misleading conclusions from synthetic audits.
Key Takeaways
- Conduct audits using real images to assess aesthetic scorers.
- Be cautious of synthetic data leading to incorrect bias conclusions.
- Understand that fidelity preference can overshadow demographic attributes.
- Utilize the reproducible benchmark for future evaluations of scorers.
How It Works
The study employs pixel-level interventions to assess how changes in skin tone and body type affect aesthetic scores. By applying consistent shifts in color attributes, the researchers isolate the impact of these changes on the scores provided by different aesthetic scorers.