Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·Aug 15·5 min read
Query Timing Produces Opposite Positional Biases Between LLMs and Humans
TL;DR
Research reveals that large language models (LLMs) exhibit positional biases—specifically recency and primacy effects—that differ from human behavior, particularly influenced by when evidence is processed. Newer LLMs show more pronounced biases compared to earlier versions.
✦ Why It Matters
Consider timing when designing LLM applications to mitigate unintended biases in outputs.
Key Takeaways
- Investigate how timing affects LLM outputs in your applications.
- Be aware of recency and primacy biases in LLM responses.
- Consider using older models if biases are problematic.
- Design experiments to test LLM behavior under different evidence presentation timings.
How It Works
The study examines how LLMs process information and update beliefs based on the timing of evidence presentation. Unlike humans, who may adjust their beliefs continuously, LLMs appear to evaluate evidence in a more static manner, leading to distinct biases.