Agentic data science with the Data Agent Kit
cloud.google.com·6h ago
AI Research
arXiv CS.AI·Aug 17·5 min read
Participatory Moral AI Is Not Neutral: The Invisible Hand of Developers
TL;DR
Moral AI systems are influenced by developers' choices in feature selection, voter sampling, and question framing, which can skew results. This study reveals that these decisions significantly impact the moral preferences generated, suggesting the need for transparency in the moral AI elicitation process.
✦ Why It Matters
Review and document your moral AI elicitation processes to ensure transparency and fairness in outcomes.
Key Takeaways
- Audit the feature selection process in moral AI systems.
- Consider the ideological diversity of your voter pool.
- Pay attention to question wording in moral AI surveys.
- Document all choices made in the moral AI elicitation pipeline.
Full Summary
As AI systems increasingly make decisions with moral implications, researchers have turned to moral preference elicitation, where participants vote on hypothetical dilemmas to guide AI behavior. However, developers play a crucial role in shaping the outcomes through three key choices: which features to include, who gets to vote, and how questions are framed.
This study analyzed these choices across three contexts—AI kidney allocation, AI agents for absent workers, and generative AI for deceased individuals—using data from 809 participants. Findings indicate that the relevance of features varies by context, political ideology influences preferences, and question wording can significantly alter results.
For instance, ideological gaps in preferences can shift by a full scale point based on how questions are framed. These insights highlight that simply aggregating votes does not ensure fair or transparent AI, emphasizing the need for auditing and disclosure at each stage of the moral AI elicitation pipeline.