Which AI providers does the Stolen Thoughts attack work against?

Stolen Thoughts demonstrates the attack on frontier models from Anthropic, OpenAI and Google. All three return encrypted chain-of-thought blocks to the client rather than keeping reasoning on the server. The worked example on the site takes a thinking block produced by Claude Opus 4.8 and replays it into a much cheaper Claude Haiku 4.5 request from the same provider.

Do you need access to the frontier model to steal its reasoning?

No. The Stolen Thoughts method never attacks the strong model directly. Because encrypted reasoning blocks are interchangeable across sessions, users and models within one provider, an attacker attaches a captured block to a request for a weaker, less guarded sibling model and jailbreaks that model into transcribing it. The whole extraction takes two API calls.

What kind of secrets did the researchers actually find in the wild?

The Stolen Thoughts team collected 6,708 public agent trajectories from GitHub and Hugging Face and decoded 315,320 reasoning blocks. In genuine, non-benchmark user sessions they recovered 704 distinct privacy artifacts, including 62 API keys, 33 passwords, 24 access tokens and 30 personal email addresses. 64 of those 704 appeared only inside the reasoning, never in the visible output.

Why does this break anti-distillation protection?

Labs hide chain-of-thought partly to stop rivals training on it. Stolen Thoughts shows that hiding is reversible: the reasoning still travels to the client, just encrypted, and any holder of the block can decode it through a weaker model from the same provider. Anti-distillation checks on the strong model are never triggered, because that model is never asked to reveal anything.

ELLIS Institute Tübingen · 2026-08-10 · major

Stolen Thoughts — encrypted reasoning pulled out of Claude, GPT and Gemini

Stolen Thoughts shows the encrypted chain-of-thought blocks Anthropic, OpenAI and Google return to API clients can be replayed into a weaker sibling model, which then prints the frontier model's hidden reasoning in plain text.

Title card for the paper Stealing Reasoning Traces from Proprietary LLM APIs

Encrypted reasoning from Claude, GPT and Gemini can be replayed into a weaker sibling model that prints it back in plain text.

Key specs

Reasoning blocks decoded	315,320
Privacy artifacts recovered	704

Quick facts

Authors	8 researchers (MATS, ELLIS Tübingen, MPI-IS, Snyk)
Providers affected	Anthropic, OpenAI, Google
Attack cost	Two API calls
Data source	6,708 public agent trajectories from GitHub and Hugging Face
Paper	arXiv 2608.09867, submitted 10 Aug 2026
Extra	Interactive 'Guess the model' demo on the site

What is it?

Stolen Thoughts is a paper and demo site showing that the hidden reasoning of frontier models can be recovered from the encrypted blocks their APIs hand back to clients. Eight researchers from MATS, the ELLIS Institute Tübingen, the Max Planck Institute for Intelligent Systems and Snyk demonstrate the attack on Anthropic, OpenAI and Google models.

How does it work?

The attack turns on where reasoning is kept. Providers do not store chain-of-thought on the server; they return it as an encrypted block that the client sends back on the next turn. Those blocks are portable across sessions, users and models from the same provider. So a trace produced by a frontier model can be attached to a request aimed at a weaker sibling, and that weaker model is jailbroken into transcribing the trace word for word — two API calls in total.

Why does it matter?

The frontier model is never attacked and its anti-distillation safeguards never fire, so the protection that hidden reasoning was meant to provide does not hold. The paper lists four consequences: bypassing anti-distillation, extracting private data, revealing hidden hazardous information, and invisible prompt injection. Decoding 315,320 blocks from public agent trajectories recovered 704 distinct privacy artifacts, so the risk is not only theoretical for anyone who has published agent logs.

Who is it for?

API developers, security teams, and anyone publishing agent trajectories

Frequently asked questions

Which AI providers does the Stolen Thoughts attack work against?: Stolen Thoughts demonstrates the attack on frontier models from Anthropic, OpenAI and Google. All three return encrypted chain-of-thought blocks to the client rather than keeping reasoning on the server. The worked example on the site takes a thinking block produced by Claude Opus 4.8 and replays it into a much cheaper Claude Haiku 4.5 request from the same provider.
Do you need access to the frontier model to steal its reasoning?: No. The Stolen Thoughts method never attacks the strong model directly. Because encrypted reasoning blocks are interchangeable across sessions, users and models within one provider, an attacker attaches a captured block to a request for a weaker, less guarded sibling model and jailbreaks that model into transcribing it. The whole extraction takes two API calls.
What kind of secrets did the researchers actually find in the wild?: The Stolen Thoughts team collected 6,708 public agent trajectories from GitHub and Hugging Face and decoded 315,320 reasoning blocks. In genuine, non-benchmark user sessions they recovered 704 distinct privacy artifacts, including 62 API keys, 33 passwords, 24 access tokens and 30 personal email addresses. 64 of those 704 appeared only inside the reasoning, never in the visible output.
Why does this break anti-distillation protection?: Labs hide chain-of-thought partly to stop rivals training on it. Stolen Thoughts shows that hiding is reversible: the reasoning still travels to the client, just encrypted, and any holder of the block can decode it through a weaker model from the same provider. Anti-distillation checks on the strong model are never triggered, because that model is never asked to reveal anything.