Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·Jun 12·5 min read
Why Sampling Is Not Choosing: Intentionality, Agency, and Moral Responsibility in Large Language Models
TL;DR
Recent claims suggest that large language models (LLMs) exhibit agency and moral responsibility. Joseph Keshet argues that LLMs operate through probabilistic mappings and lack intrinsic intentionality, meaning they do not truly make choices.
✦ Why It Matters
Engineers should recognize the limitations of LLMs in moral and ethical decision-making contexts.
Key Takeaways
- Understand that LLMs lack true agency and moral responsibility.
- Recognize that outputs are based on learned data patterns, not choices.
- Critically evaluate claims of intentionality in AI systems.
- Consider ethical implications when deploying AI technologies.
Full Summary
The paper critiques the notion that large language models (LLMs) can be seen as moral agents or possess agency. It asserts that moral responsibility requires a form of agency that is rooted in intrinsic intentionality and self-attributed actions, which LLMs lack.
Instead, LLMs operate through probabilistic mappings learned from vast datasets, producing outputs that may seem intentional but are not genuinely so. The variability introduced by stochastic sampling does not equate to authorship or choice.
The author addresses various philosophical objections, including those from the intentional stance and compatibilism, concluding that none successfully establish true agency in LLMs. This analysis has implications for how we understand the ethical responsibilities of AI systems.