Researchers Show How to Extract Hidden Reasoning from Proprietary LLM APIs
A project called Stolen Thoughts lays out techniques for extracting reasoning traces from proprietary LLM APIs that deliberately withhold their internal chain-of-thought from users.
Models like OpenAI's o1 and o3 generate step-by-step reasoning before producing a final answer, but the provider only shows a summarized or redacted version, partly to protect the model's inner workings from competitors and partly to prevent misuse. The research demonstrates that clever prompting, timing analysis, or exploiting quirks in API responses can leak fragments of that hidden reasoning, effectively reconstructing what the model 'thought' before answering.
The project has generated heavy discussion on Hacker News, with commenters weighing in on both the technical cleverness and the implications for AI vendors who treat reasoning traces as valuable IP.