Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 21·8 min read
CopT: Contrastive On-Policy Thinking with Continuous Spaces for General and Agentic Reasoning
TL;DR
Chain-of-thought (CoT)—a technique where LLMs write out reasoning steps before answering—wastes tokens when models could answer directly, a problem called performative reasoning. CopT reformulates CoT using contrastive learning (training by comparing correct vs. incorrect examples) in continuous embedding spaces to let models decide dynamically whether extended thinking is needed.
✦ Why It Matters
Engineers can reduce LLM inference costs and latency by adopting adaptive reasoning that skips unnecessary thinking steps without sacrificing answer quality.
Key Takeaways
- Implement adaptive reasoning: let models decide when thinking is necessary rather than forcing it always.
- Use contrastive learning to train models to distinguish when reasoning adds value versus when it wastes tokens.
- Measure token efficiency gains alongside accuracy to validate cost-benefit tradeoffs in your deployment.
- Test on both general reasoning and agentic tasks to ensure the method generalizes across problem types.
Full Summary
Chain-of-thought prompting is a standard technique where large language models (LLMs) generate step-by-step reasoning before producing answers. However, this approach treats thinking as mandatory, forcing models to generate lengthy intermediate reasoning even when they could identify correct answers immediately.
This inefficiency, termed performative reasoning, increases computational costs without improving answer quality. CopT (Contrastive On-Policy Thinking) reformulates the reasoning process by treating it as an on-policy decision—the model learns when to think versus when to answer directly.
The method uses continuous spaces (rather than discrete token sequences) and contrastive learning (comparing similar and dissimilar examples) to train models to allocate reasoning effort adaptively. Preliminary results indicate CopT reduces token consumption while preserving or improving accuracy on both general reasoning tasks and agentic reasoning (where models must plan and act).