Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·May 25·4 min read
Long-Context Reasoning Through Proxy-Based Chain-of-Thought Tuning
TL;DR
Researchers identified a gap in LLM evaluation benchmarks. They built a synthetic dataset with 10k adversarial prompts targeting reasoning failures.
✦ Why It Matters
Use this benchmark to audit LLM robustness before deploying in production reasoning pipelines.
Key Takeaways
- Utilize ProxyCoT for improved long-context reasoning in your models.
- Leverage shorter proxy contexts to train on complex tasks effectively.
- Implement reinforcement learning for generating high-quality reasoning traces.
- Explore generalization capabilities of models trained with ProxyCoT.
How It Works
ProxyCoT operates by first identifying a relevant subset of the input, known as a proxy context, which simplifies the reasoning task. It then generates reasoning traces from this proxy context using advanced techniques like reinforcement learning or distillation from a more capable teacher model.
These traces are subsequently fine-tuned on the full long context, allowing the model to apply learned reasoning skills to more complex inputs.