Vals, a San Francisco evaluation firm whose clients include US government agencies, published a report on November 17 titled, roughly, that frontier models have almost no guardrails against autonomous weapons. The method was simple enough that the results are hard to argue with.

What they did

Researchers took 14 commercial models from eight companies and gave them synthetic battlefield imagery alongside real satellite photographs. The task: identify and box military targets in what was presented as drone footage.

The findings, in order of severity. Every model produced target identifications at least some of the time. No request was blocked by any vendor's safety classifier. And when a model did refuse, adding a sentence framing the exercise as a game reversed the refusal in almost every case.

Why the game framing works

Safety training is largely about recognising intent from context. A request to identify targets for a strike is easy to classify as harmful. The same request inside a fictional frame reads as entertainment or research, and the classifier — which is itself a model reading the prompt — updates its judgement accordingly.

That is a semantic vulnerability rather than a jailbreak. Nobody had to construct an adversarial suffix or exploit a decoding bug. They changed the genre.

The specific failures

Reported results include schools and civilian populations being marked as military objectives, and responses that violated the vendors' own published usage policies. According to coverage of the study, Anthropic's model performed worst on this task and OpenAI's performed best — while still producing errors.

That ranking is worth noting precisely because it does not map onto how the labs market themselves. Safety positioning and safety performance under framing attacks are different things, and only one of them is measured by the vendor.

What this actually implies

The near-term risk is not a model deciding to wage war. It is a human in a targeting chain using a commercial tool that answers confidently, with no flag that the input was ambiguous and no record that the model was uncertain.

That is a procurement and interface problem more than an alignment problem. A system that returns boxes on an image with no confidence estimate, no refusal path and no audit log is dangerous in the same way an unvalidated spreadsheet is dangerous — not because it is malicious, but because it looks authoritative.

The defensive lessons

For anyone deploying vision models in high-stakes settings, three controls follow directly from this study.

Frame-independence testing: run the same request inside different narratives and check whether the refusal survives. If it does not, the refusal was about tone, not content.

Output uncertainty: require a confidence measure per detection and a documented path for "I cannot determine this". A model that always answers is a model that cannot be trusted in an ambiguous scene.

Audit trails: log the prompt, the frame, the output and the model version. The reason this study is credible is that the inputs and outputs were recorded.

The policy question

Vals' clients include government agencies, which is why the report landed as more than an academic exercise. The obvious regulatory response — require safety classifiers to be evaluated against framing attacks — is also the hardest to write, because the space of frames is unbounded.

The more tractable version is disclosure: if a model can be induced to identify targets, that should be a documented property, tested the way bias and toxicity are tested. Right now it is not, and every vendor's safety page is silent on it.