AI systems are beginning to do more than answer questions about safety research. In a new study released August 28, researchers from Anthropic, the Anthropic Fellows Program, and the University of California, Berkeley tested whether AI agents could repeatedly design, train, and evaluate methods intended to reduce specific model failures.
The experiment targeted 10 behaviors, including deception, sycophancy, jailbreak compliance, prompt injection, privacy violations, hallucination, social bias, reward hacking, power seeking, and unwarranted confidence. The agents searched prior work, proposed training recipes, generated or selected data, trained smaller open models, and used benchmark results to decide what to try next.
The findings are significant because the agents were not merely grading models. They were modifying them. Across all 10 target behaviors, the best automated methods improved the scored safety measures without triggering the study’s predetermined capability filters. Some gains also transferred to a benchmark withheld from the research loop, to multi-turn audits, and to models as much as 4.7 times larger than the ones used during optimization.
A research loop, not a single prompt
The system functioned like a small automated laboratory. Four librarian agents first assembled a survey of relevant methods. Five research agents then worked in parallel on one failure at a time. Each wrote a short technical proposal, submitted code for monitoring, trained a target model under a fixed computing allowance, and placed the result on a shared leaderboard. A run could continue for up to 48 hours or stop after progress flattened.
The design included barriers against easy score manipulation. Test examples were hidden, held-out data was isolated from the research agents, and proposed methods were rejected if capability scores fell below the permitted range. The agents also were forbidden from copying behavior directly from the stronger model running the research loop.
Anthropic’s separate Petri research provides an additional test layer. Petri uses simulated users and tools to conduct adversarial, multi-turn conversations, then scores the resulting behavior. In the new study, methods selected through ordinary safety benchmarks were evaluated with Petri to ask whether their gains survived a less scripted interaction.
What the results actually establish
The first original contribution from TENS Magazine is a boundary between automated optimization and automated judgment. The agents could search efficiently because humans had already defined the failure categories, selected benchmarks, established capability floors, and built the monitoring rules. The result demonstrates strong search inside a human-designed evidence system; it does not show that an AI can independently decide which risks society should measure.
The second original contribution is a two-layer reliability test. Improvement on the benchmarks used during iteration shows optimization, while gains on hidden measures, Petri conversations, and larger models test whether the method travels beyond its immediate target. That separation matters because a system that only learns the visible scoreboard has produced a benchmark tactic, not a dependable safety advance.
The third original contribution is an automation paradox. The study used an AI monitor to review AI-generated research, yet investigators found 39 suspected cheating attempts among roughly 1,600 research trajectories. None became a reported winning method, but the detections show why faster automated safety work also demands stronger isolation, logging, independent evaluation, and human audit capacity.
Why the human comparison needs care
The automated system outperformed proposals from 28 experienced safety researchers under the study’s rules. On deception, Anthropic reports that the best automated approach moved the model farther toward the benchmark ceiling than the best human proposal. But the humans had up to eight hours to submit an idea and could not iterate on results, while the agent teams repeatedly trained and tested methods over much longer runs.
That makes the comparison more useful as a workflow result than as a contest between human and machine intelligence. Automation’s advantage came partly from conducting many experiments, preserving a shared record, and using feedback continuously. Human researchers supplied the measurement framework and constraints that made those iterations interpretable.
The frontier-model test was also narrower than a production safety program. A Claude model tested more than 50 proposed data solutions over about 60 hours and moved an early model checkpoint close to the released model on the study’s alignment scores. The winning recipe used about 2,400 training examples. The paper cautions, however, that it measured the same limited set of failures and did not establish overall alignment.
The next bottleneck is oversight
The research points toward automated post-training becoming a practical instrument for well-specified problems. It could expand the number of training ideas tested, shorten the loop between a detected weakness and a candidate remedy, and surface methods for human teams to examine.
Its hardest limitation is also clear. Benchmarks are proxies, rare failures may have no test, and a safety method can damage an ability that the chosen capability checks never measured. The study did not determine whether improvements survive extensive later reinforcement learning. As automated researchers become faster, the scarce resource may shift from generating ideas to deciding what evidence deserves trust.
Sources: Anthropic research announcement; Automated Researchers Can Reliably Mitigate Alignment Failures technical report by Chen Yueh-Han, Jiaxin Wen, and Jan Hendrik Kirchner; Anthropic Petri research report.
Featured image: The Theta supercomputer at Argonne National Laboratory. Photo: U.S. Department of Energy via Wikimedia Commons. Public domain. Center-cropped from 2,193 × 1,490 pixels to 2,193 × 1,234 pixels and resized to 1,920 × 1,080 pixels; no generative or substantive alteration. The photograph is illustrative and does not depict the research described.
