Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
Data Science
Simon Willison's Blog·Aug 11·5 min read
Stealing Reasoning Traces from Proprietary LLM APIs
TL;DR
A method was developed to extract reasoning traces from proprietary large language models (LLMs) like those from Anthropic, OpenAI, and Google. By replaying encrypted reasoning blocks from a stronger model into a weaker one, researchers can uncover the hidden reasoning in plaintext.
✦ Why It Matters
Evaluate your own LLM implementations for potential vulnerabilities to similar extraction techniques.
Key Takeaways
- Investigate the security of LLM APIs you use.
- Be aware of potential vulnerabilities in proprietary models.
- Consider the implications of reasoning trace extraction.
- Stay updated on patches and fixes from model providers.
How It Works
The researchers exploited the fact that all models in the same family shared the same encryption key. By feeding encrypted reasoning blocks from a stronger model into a weaker one, they could extract the unencrypted reasoning.
This was achieved using specific prompts that instructed the model to transcribe its reasoning verbatim.