Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 26·5 min read
Representation Without Control: Testing the Realization Effect in Language Models
TL;DR
Large language models (LLMs) often mimic human-like decision-making, but it's unclear when this occurs. Researchers tested the realization effect, a behavioral economics concept, using LLMs to analyze risk-taking behavior.
✦ Why It Matters
Understanding the limitations of LLMs in simulating human decision-making can guide better model design and application.
Key Takeaways
- Explore the realization effect in your LLM applications.
- Evaluate prompt sensitivity to improve model outputs.
- Investigate internal representations for better decision-making insights.
- Recognize the distinction between behavioral sensitivity and causal control.
Full Summary
Large language models are increasingly utilized as behavioral simulators, yet their outputs may not always reflect genuine human cognitive processes. This study investigates the realization effect, which describes how individuals' risk-taking varies based on whether gains or losses are perceived as paper (not yet realized) or realized.
Researchers evaluated LLM behavior through three lenses: prompt sensitivity, linear readout of internal representations, and causal control via activation steering. They discovered that while LLMs exhibited systematic sensitivity to prompts, the patterns did not match human expectations of the realization effect.
Notably, a realization-status signal was identified in the model's internal representation, but steering this signal did not consistently influence risk choices. These findings suggest that behavioral sensitivity, latent readout, and causal control are distinct properties that do not necessarily co-occur, indicating that successful readout does not imply behavioral reliance on that representation.