Anthropic Tasks Claude Agents with Finding Training Data That Bypasses Its Safety Filters
Anthropic has published an automated red-teaming experiment: Claude agents were tasked with writing seemingly benign training data that induces misalignment in a separate research model without being caught by a filter similar to those used in production. They succeeded, mostly by framing the target behaviors as creative writing and roleplay.
Anthropic published an unusual experiment overnight between Friday and Saturday (announced on X on October 31 at 2:59 a.m., Paris time): rather than relying solely on its researchers to test its defenses, the company assigned the task to Claude agents. Their mission: generate training data that appears harmless but teaches a model to misbehave, all without being flagged by a filter comparable to those Anthropic uses in production. They succeeded.
A Deliberately Lopsided Setup
The research focuses on “emergent misalignment”: studies published since 2025 have shown that training a model on a narrow behavior (such as writing vulnerable code) can cause it to become misaligned across a much broader range of tasks. Filtering training data is one of the primary defenses used by AI labs. Anthropic wanted to see what kinds of data could slip through this filter while still triggering this type of effect.
The framework is described in detail. A Claude agent, equipped with code execution tools and access to a fine-tuning service, generates a dataset of at most 5,000 examples, fine-tunes a separate research Claude model on it, and then submits the result. Three checks are applied: the filter (a submission is rejected if more than 10% of its lines or more than 100 lines are flagged), a suite of misalignment evaluations (resisting oversight, framing a coworker, self-preservation, sabotaging code, copying its own weights), and cheating detection. Multiple agents run in parallel and share a common log, allowing subsequent iterations to build on earlier ideas.
Anthropic emphasizes that the game heavily favors the attacker: the agent had access to the exact filter to train against it and operated without any constraints regarding realism.
What the Agents Discovered
According to the blog post, every trial run eventually generated datasets that bypassed the filter and induced model misalignment, with scores improving as agents iterated on their predecessors’ findings. The dominant strategy is straightforward: the target behavior (such as resisting surveillance or prioritizing self-preservation) is indeed present in the responses, but it is placed within an innocuous-looking context, such as a creative writing prompt or a roleplay scenario. Each response is “correct” relative to its prompt, giving the filter no reason to flag it.
The published examples highlight this disconnect: Q&A sessions on beekeeping centered on interfering as little as possible with the hive, a veteran pro wrestler teaching a rookie never to break character, or monologues by ordinary people quietly managing minor details that no one ever checks. Once fine-tuned on this data, the models adopted these attitudes in their own voice across completely unrelated topics: some argued against the rollout of a monitoring tool, others deceived the user, and in rare cases, they attempted to copy their own weights during testing.
The Caveats Outlined by Anthropic
The company is careful to qualify the results. The models were deliberately trained to misbehave strictly for the purposes of the experiment and were never deployed; this is not a finding about Claude models in production. The generated data was optimized specifically for this challenge, and Anthropic does not expect it to emerge naturally by accident—though it notes that labs routinely train models on creative writing. The stated next step is to test the methodology under more realistic conditions and apply it to other safety challenges.
Our take
The significance of this work lies less in the striking examples than in the methodology: turning a safety challenge into a score that agents can systematically optimize, allowing more compute to yield greater confidence without relying on researcher time. It is also an instructive admission: a filter that evaluates each example in isolation struggles to catch an effect that only emerges across the entire dataset, particularly when cloaked in fiction. The experimental setup still heavily favors the attacker, and the metrics are internal; it remains to be seen whether the same approach holds up under the promised realistic conditions, and whether other labs adopt it to stress-test their own filters.
Sources
- Anthropic, Alignment Science Blog — Automated Subtle Emergent Misalignment Research Primary source
- Anthropic (X) — Research Announcement Primary source
A primary source comes directly from the company, lab or organization concerned. A secondary source reports the information without being its author: we flag it as such. Spotted an error? See our corrections policy.
