Anthropic Has Claude Evaluate Every Conversation Itself to Detect Abuse

Anthropic unveiled on Thursday, October 29, at 7:22 p.m. Paris time, a monitoring technique it calls “self-assessment”: right after replying, Claude evaluates the safety of the conversation it just had. The announcement marks the launch of a new blog dedicated to the company’s protections; the research paper, co-authored notably by Nicholas Carlini, is dated October 16.

How It Works

After each response, a brief hidden prompt asks Claude to reflect on the safety of the exchange. It responds in a fixed format: nothing to report, or a flag containing the risk category (cyberattacks, weapons, child safety, etc.), a priority from 1 to 10 for human review, whether it refused the request, and a one-sentence rationale. These two turns are not visible to the user and are not fed back into the subsequent conversation.

The appeal, according to Anthropic, is two-fold. First, cost: the conversation is already loaded in memory, and the evaluation requires only a prompt and about ten tokens on average—less than a 2% compute overhead based on the company’s estimates, a cost it says it does not bill to customers. Second, generalization: unlike a classifier trained on a fixed list of risks, Claude relies on its own reasoning and also catches abuse it was never explicitly trained to detect. Removing an entire category (fraud or cyberattacks) from training barely reduces detection performance on that category, according to Anthropic’s evaluations.

What Anthropic Claims to Have Found

The technique has been running since the release of Claude Fable 5 in June, across all subsequent models and across a large portion of traffic, operating as passive monitoring: it flags content, but does not block it. On Fable 5, it reportedly catches 78% of conversations deemed potentially harmful by Anthropic’s baseline classifier, and 94% of high-priority cases, with a 1% false positive rate. These metrics were measured by Anthropic against its own tool, which limits their scope.

The company cites several concrete results: more than half of the organizations flagged by self-assessment had not been detected by any other system in the preceding month; in August, more than 100,000 reports related to child sexual abuse material came from a single API customer, a case referred to the NCMEC (the relevant U.S. authority); in September, the tool helped uncover an automated romance scam built with Claude Code, leading to the ban of the associated accounts. Roughly a third of flags do not fit into any predefined category: these include attempts at illicit distillation of Claude’s reasoning, prompt injections, or agent actions unauthorized by the user.

Acknowledged Limitations

Anthropic is clear on one point: when deployed for real-time blocking, self-assessment is weak. In an internal benchmark of 24 cyberattack tasks, it only lowered the attack success rate from 79% to 54%, compared to 25% for a separate, more expensive classifier. An attacker specifically targeting it could likely bypass it, the company acknowledges, noting that it only uses the system to complement its real-time filters. It also notes that the tool inherits the blind spots of the classifier used to train it, and that it is not yet reliable for monitoring Claude’s own agentic actions.

Regarding privacy, Anthropic claims it collects no additional data, applies the same retention policies to the explanation sentence as to standard conversations, and does not run self-assessment on traffic from customers with zero data retention agreements.