A Core ML port of Laya answers a typed question in 4.98 ms, and reports the speedup it did not get

Laya is the open implementation of the typed-decision primitive that TypeSafe ships as jev: a model that returns probabilities over a fixed set of questions rather than generating text. This port compiles it to Apple Core ML and runs it on the Neural Engine with, in the repository's words, "zero generated tokens" and "no autoregressive decoding or generated JSON to parse". On an M3 Max the short multilingual checkpoint answers one 91-token question in 4.98 ms at the median and 5.31 ms at the 95th percentile, against 6.94 and 7.39 ms for the same model compiled under MLX. That is a speed gain of 1.39x. The energy gap is the larger one: 0.1540 joules per decision against 0.4288, a 2.78x improvement taken from SMC power sensor readings across 65,598 stable calls. An eight-bit palette variant reaches 4.88 ms and 3.19x. The repository states the result it was aiming at and did not reach: "the requested 10× improvement was not achieved."

Fidelity is reported at the same grain. The three general-purpose FP16 checkpoints match the upstream model's selected answers on 189 of 189 validation questions. The Neural Engine FP16 build passes 59 of 59 of the questions that fit its 96-token window, with a maximum calibrated-probability drift of 0.002925; the eight-bit variant drifts 0.014393 against an unchanged 0.02 gate. Six-bit and four-bit conversions failed that gate and are not published. The repository also declines two claims a reader might otherwise take from it: "These are conversion-fidelity fixtures, not proof of general task accuracy," and, of a 1024-token request that takes about 91.7 ms, "The short ANE result does not establish a long-context advantage." A terminal demo sustained 49.1 to 50.0 decisions per second across three 600-step episodes with zero deaths and two safety interventions. The port reached Hacker News through a five-line install gist submitted at 15:58 UTC, and stood at 20 points with no comments at 17:10 UTC.

Why it matters: A typed decision at 5 ms and 0.15 joules on a laptop is a different component from a hosted classifier call. Routing, gating and scoring steps that currently cost a network round trip and a token bill can sit inside the harness, offline, with a latency you can put a timeout on. The constraint to check first is the 96-token window on the fast path, because the general-purpose 1024-token build gives most of the advantage back. The decision step in a harness is becoming something you pick a supplier for, and the open side of that choice now has a measured, reproducible option that runs on consumer hardware.