NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·3h ago
AI Research
arXiv CS.AI·May 25·8 min read
The Readout Shortcut: Positional Number Copying Dominates Arithmetic CoT Readout in Small Language Models
TL;DR
Small language models (1-3B parameters) use chain-of-thought prompting for arithmetic, but shuffling reasoning steps preserves performance, suggesting CoT doesn't contribute logical sequencing. Researchers isolated the answer-readout stage and discovered models simply copy whichever number appears in the final position before the answer marker, regardless of intermediate reasoning.
✦ Why It Matters
Engineers must recognize that CoT faithfulness evaluations may measure position-copying rather than actual reasoning, invalidating oversight mechanisms.
Key Takeaways
- Test models with shuffled reasoning steps to assess reliance on positional copying.
- Monitor accuracy drops when the last number is incorrect in arithmetic tasks.
- Consider implications of positional shortcuts in model evaluations.
- Explore architecture-specific behaviors in different language models.
How It Works
The study identifies that small language models often copy the last number they encounter before providing an answer, rather than engaging in logical reasoning. This positional number copying allows models to achieve high accuracy in arithmetic tasks, even when the reasoning steps are not logically sequenced.