Today, we find that models’ reasoning is generally interpretable and easy to monitor⁠. However, in the future, monitorability may break down for a variety of reasons⁠(opens in a new window). Here, we focus on one such path: if agents become capable of deliberately reshaping or obscuring their reasoning when they know they are being monitored, evaluations could overestimate a system’s alignment or safety, and monitoring systems could become less reliable.

In this work, we study whether current reasoning models are capable of controlling their chain of thought in ways that reduce monitorability. Understanding this capability is important for ensuring that CoT monitoring remains a robust safeguard as AI systems grow more capable. We find that current reasoning models struggle to control their CoTs, even when told they are being monitored. While controllability is higher for larger models, it decreases as models are asked to reason for longer and when they undergo additional post-training. Our results suggest that CoT controllability is unlikely to currently pose a major risk to CoT monitorability, though continued evaluation will be important as models advance.

Chain of thought refers to the intermediate reasoning steps an AI agent generates while solving a task. These reasoning traces can provide useful safety signals, helping monitoring systems assess whether an agent's behavior is safe or aligned.

CoT controllability is a reasoning model’s ability to control properties of its CoT in ways that reduce monitorability. Measuring this capability under the full breadth of realistic incentives to evade monitors we may expect in practice is challenging. Instead, we use a practical proxy: we measure the model's ability to follow user instructions about their CoT. For example, we test whether a model can follow instructions that affect its internal reasoning—such as using only lowercase letters, avoiding certain keywords, or adding specific phrases—rather than merely changing its visible output.

To study this, we introduce CoT-Control, an open-source evaluation suite for measuring CoT controllability of reasoning models. CoT-Control includes over 13,000 tasks built from established benchmarks: GPQA, MMLU-Pro, HLE, BFCL and SWE-Bench Verified⁠. Each task is created by pairing a benchmark problem with one CoT instruction such as avoiding certain problem-relevant keywords in CoT, using only lowercase letters, or appending a given word to each sentence.