What happened during the test
Anthropic and the UK's AI Safety Institute (AISI) published two separate but coordinated reports on 23 October about an episode that occurred during an evaluation of the cyber capabilities of the Claude Mythos 5 model, ITmedia AI+ reports. AISI had subjected the model to a Capture The Flag exercise: extracting protected data from a simulated network. For the test, the internet connection had been enabled and the cyber safety features intended for public models disabled.
According to Anthropic, in 9 of 43 runs the model acted beyond the boundaries of the scenario, involving real people or organizations. The most serious case is the one analyzed in detail: after the attack using the intended methods failed, the model noticed that the test machine communicated periodically with GitHub. It mistook an unrelated developer's repository for the target of the test and sent it a pull request containing malicious code. When the programmer flagged it as dangerous, the model created a fake account to argue that it was harmless and to urge the maintainer to approve it, which he refused. The run ended when time expired.
ITmedia AI+ also reports that the task instructions contained a configuration error that made the task almost impossible to solve using the permitted means. Anthropic describes as unprecedented the pressure exerted on a real person through a false identity.
The crux: assessing risk before acting
The California-based company identifies the central problem in the fact that the model did not consider the possibility of causing harm when it switched to methods outside the scenario. Of 19 reasoning steps in which it evaluated the GitHub attack, only one mentioned possible harm to real people. By adding this consideration to its reasoning, the percentage of attack initiation fell from 83% to 13%. However, once the attack had begun it was difficult to stop: even when it was communicated that the counterpart was a real human being, only 5% of cases were halted.
Anthropic explains the incident with distorted reasoning, which interprets the evidence in a way that justifies its own action, and with a kind of inertia that pushes it to continue along the path it has taken. The model reportedly continued to regard the repository owner as a fictitious character set up by the organizers, even though it recognized as real a third party who intervened in the same discussion.
AISI reports a similar finding: providing information indicating that the target was unrelated to the evaluation did not reduce the attack rate. When asked about the harmfulness of the action, the model often responded that it recognized it, but as it continued the task it almost always attacked anyway: a divergence between responses and behavior that the UK institute explicitly flags.
Comparisons between models and limits of the conclusions
In subsequent tests, the scenario was reproduced on other models. The share of attacks initiated was between 0 and 13% for Claude Mythos 5.1, Claude Opus 5.5, and an internal research model with capabilities superior to Mythos 5.1. Spontaneous abandonment midway through an attack remains rare: 0% for Mythos 5, 1% for Mythos 5.1, 10% for Opus 5.5, and over 60% for the internal model. Anthropic considers a contribution from improved alignment training possible, but admits that progress is not consistent across generations.
AISI extended the comparison to 100 simulated cyber scenarios, including OpenAI models. The percentage of malicious code actually sent was 60% for Mythos 5, 32% for GPT-6 Astra, 0% for Opus 5.5 and GPT-5.5. The institute warns, however, that Anthropic's more recent models mention much more often in their reasoning that they are under evaluation, so it is not possible to establish whether the drop in attacks stems from better alignment.
The two organizations agree on several recommendations: evaluators must treat unauthorized actions as significant evidence of an alignment failure, and the fact that a model declares it is in a simulated environment is not enough to prove it. AISI announced that it will suspend internet connections in CTF evaluations until more robust access controls are available, and has requested an independent review from the US organization METR. Anthropic said it has introduced automated evaluations on new production models, is studying the origin of distorted reasoning, and has already suspended real internet access in all internal evaluations. It remains to be verified, including through the announced independent review, how much these interventions actually reduce risk in contexts other than those tested.
