OpenAI explains how internal probes interrupt model responses OpenAI explains how internal probes interrupt model responses

A signal inside the generation loop

OpenAI says its probes score each generated token using internal activations. Crossing a threshold pauses the response and holds tool calls for a reasoning model’s judgment. Training uses synthetic and de-identified conversations labeled by reasoning models. Their method maintains a small moving summary of activations rather than retaining a rolling token window. This places the first decision inside inference, where it can interrupt an output before a slower review finishes.

Escalation is part of the design

Anthropic described a related cascade on January 9. A cheap linear probe screens traffic, then a probe-and-classifier ensemble judges suspicious exchanges. The second stage considers the input together with the output, addressing cases where either looks harmless alone. Anthropic reported 0.05% refusals on harmless requests during a month of Sonnet 4.5 traffic, and roughly 1% compute overhead if applied to Opus 4.0 traffic. Those are different evaluation settings, not a single cost-and-accuracy measurement. Escalating an uncertain signal also separates suspicion from the final decision to refuse. A sensitive first stage need not automatically turn every false alarm into a blocked user request.

An assessment after the answer serves another purpose

Anthropic’s October 16 self-assessment report describes an offline alternative. After responding, Claude receives a short reflection instruction and produces a structured safety assessment. Reusing the conversation’s cached computation makes the extra work small: Anthropic estimates fewer than 2% additional inference compute. Flags feed later investigation rather than blocking the current response. On Fable 5, the method recovered 94% of high-priority flags from a stronger prompted classifier at a 1% false-positive rate. That denominator is agreement with another model’s judgments. It does not measure the share of all harmful activity caught. The distinction matters when choosing between preventing one action and discovering patterns across many conversations.

An alert still needs an operational response

OpenAI’s October 29 monitoring report explains what happens beyond screening. Investigative agents can inspect full transcripts, reconstruct incidents and compare conversations with network logs. Response coverage varies: frontier training can page humans around the clock, while most other traffic has business-hours paging. Internal Codex paging was disabled in that disclosure. Some critical alerts can pause a workload automatically. The report also describes sampling 5% of traffic that probes did not flag to look for misses. These arrangements determine what an alert accomplishes after detection. A favorable classification result alone says little about whether someone receives the alert before a consequential action finishes.

Detection and authorization answer different questions

Consider an agent preparing to send a customer file. A safety monitor might recognize a dangerous request, while the application still needs the intended recipient, approved attachment and permitted destination. A message can be benign in content and still exceed the user’s instruction. Conversely, a legitimate security task can contain language that warrants a closer look. These are different reasons to stop an action. The useful comparison is therefore the decision each layer owns: classify content, investigate behavior or enforce the scope of a specific task. Treating every favorable model judgment as permission would collapse those separate responsibilities.