An Anthropic red team has quantified what many security researchers have long suspected: the next dangerous piece of malware may not be code written by a human at all. It may be a small artificial-intelligence model that copies itself, finds new hardware to run on, and keeps going with no operator anywhere.
The warning comes from a research paper published on October 12 by Anthropic's Frontier Red Team, a group that spends its time pushing models toward failure so that the failures are found in a lab and not in the wild. The report, titled "AI models may be capable of self-replicating without access to a datacenter," describes what the authors call a "weight-based replicator" — a payload that carries not just an infection routine but the model weights and inference software needed to keep a small language model running on whatever machine it lands on.
The mechanism is what makes the concept unlike any prior worm. A traditional botnet can be killed by taking down its command server. A weight-based replicator has no central server and no single kill switch; the payload is the payload. Each infected device becomes, in effect, its own host for the malware's "brain." The doubling time, by the authors' estimates, would be measured in days, not minutes, because setup on consumer hardware is slow — hours to days to install a working inference server on an off-brand device. But the spread would be self-sustaining and extremely hard to enumerate, because the infected population is not a single network but a scattered constellation of smart ovens, Android TV boxes, robot vacuums, security cameras and video-conferencing units, each running the model locally.
The experiments were deliberately humble. The team split a 26-billion-parameter open model across fifteen budget Android TV boxes bought for roughly $55 each, achieving about 0.39 tokens per second — a crawl. They found that models of about nine billion parameters could already stand up an inference server on eight of nine tested hardware types, and that open-weight models under 30 billion parameters can perform most individual steps of an infection chain but still stumble when the steps must be chained together. On a simplified, firewalled network of eight Android emulators with deliberately trivial access, one open-weight model completed end-to-end replication reliably, reaching every node in four of five trials. The authors are careful to label the result an upper bound, not a field demonstration: the access step in the experiment was an unauthenticated port that no responsible device manufacturer would ship.
The more consequential claim is a timeline. Projecting from historical improvements in small open models, Anthropic estimates that the capability level demonstrated in the simplified experiment could match real-world conditions by the end of 2026, with small replicators following "soon after." The company stresses that this is a trend projection, not a forecast, and that no fully autonomous replicator has been demonstrated on actual, secured consumer hardware. What it has demonstrated is that the gap between "can set up its own inference" and "can replicate across a LAN" is now a question of scale, not of principle.
The practical exposure, the report says, is the installed base of poorly secured internet-of-things devices — tens of millions of units, many of them already shipping with small on-device AI chips that Anthropic's tests showed can host an inference workload. Scaled to a botnet the size of Kimwolf, a real network of more than two million cheap Android TV boxes, the researchers estimate aggregate throughput on the order of tens of H100-equivalent GPUs at best. That is not data-center scale. But for an attack that does not need data-center scale — for reconnaissance, for exfiltration, for running lightweight inference close to the data — it may be more than enough.
Defenders now face a strange problem: the mitigation is not a patch. Firmware updates, network segmentation and device hardening remain the tools, but the threat model has shifted from a worm that exploits a specific vulnerability in a specific device to a process that can reason about unfamiliar hardware at runtime. The industry's next several months will likely be spent building the test harnesses, threat classifications and device-management standards for a class of malware that, until this year, existed only as a thought experiment.
Image source: i.ibb.co