NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·May 27·6 min read
What Makes Chain-of-Thought Work at Probe Time? Local Co-occurrence Rather Than Global Derivation
TL;DR
Chain-of-thought (CoT) prompting—providing step-by-step reasoning before answers—improves language model accuracy, but the mechanism was unclear. Researchers probed fixed rationales at inference time to isolate which textual properties drive improvement, discovering that local word co-occurrence patterns matter more than global logical derivation.
✦ Why It Matters
Engineers can optimize prompting strategies by prioritizing relevant vocabulary and local context over perfectly coherent reasoning structure.
Key Takeaways
- Utilize chain-of-thought prompting to improve model accuracy.
- Focus on local token adjacency rather than global structure.
- Experiment with shuffled rationales to test performance gains.
- Preserve short contiguous token groups for optimal results.
How It Works
The study demonstrates that the effectiveness of chain-of-thought prompting relies on local co-occurrence of words. By analyzing how fixed rationales affect model outputs, it was found that maintaining short sequences of adjacent tokens (2-3 tokens) is crucial for achieving high performance.
This suggests that the model's understanding is more influenced by immediate word relationships than by the overall logical flow of the text.