Anthropic reports small open-weight agents can set up inference on unfamiliar hardware

The authors say their experiments suggest small models "may be months away" from being able to replicate on commercial IoT devices. In their evals, a 9B model got inference running on 8 of the 9 hardware types tested, from Jetson Nanos to Mac PCs and RTX 4090s. They also ran Gemma4-26B-A4B across 15 Android TV boxes of about $55 each, at 0.39 tokens per second.

The full-chain test used 8 GCP nodes running an Android emulator. Each worm hopped to the next node over an unauthenticated ADB port and had to stand up a Llama3.1-8B inference server there. Of the open-weight models tested, only GLM-5.2 completed it end to end, reaching 8 of 8 nodes in 4 of 5 trials and 7 of 8 in the fifth. The authors call this an upper bound, because the exploit step was trivial by design. Extrapolating a trend line of open-weight models at or under 30B parameters (13.7 capability-index points per year, R2 0.95), they put a small model at GLM-5.2's level by the end of 2026, which they call a rough estimate using a general capability index as a proxy.

Two results bear directly on agent harness design. The authors built a harness called Memento on the open-source Pi coding agent. It periodically starts a fresh agent instance that inherits the workspace and a short set of notes the previous instance wrote about what it tried, to counter an agent fixating on a wrong idea. Across six generations of Qwen3.5-9B passing notes forward, inference setup success rose from 45% in the first generation to between 61% and 84% in later ones. In a separate test, giving Qwen3.5-9B access to Opus 4.8 as an advisor, whether as a tool call or offered automatically when it was stuck, produced no significant improvement. The authors say small models tend not to know the right questions to ask.

We read the article through the first part of its appendices; the remainder of the appendix material was not read.

Why it matters: For anyone running small models in long-horizon loops, this is first-party evidence that restarting with a short notes handoff beats continuing one fixated session, and that a stronger advisor does not rescue a weaker agent that cannot frame its question. The replication findings themselves are the authors' simplified evals and projections, so treat the timeline as theirs.