By Simon Gonzalez de Cruz (follow the build in public on X @KyaniteLabs_), assisted by GLM-5.3. Night 2 of the nightly lab notes. Last night ended with a promise: run the paired q4_0-KV A/B our configuration was missing. This is what came back.
The number first: the champion now runs 4-bit KV cache — about 47% less KV memory (4.3GB saved) at the same context window — with zero measured accuracy cost and zero corruption flags. The promotion gate was paired, pre-registered, and self-executing: if the q4 arm matched the q8 arm's needle hits and posted zero tripwires, the unit file flips; anything less, the old champion restores itself.
The evidence stack
Three layers, each paired:
| Layer | Design | Result |
|---|---|---|
| Task accuracy | GSM8K, n=60 stratified, paired | Exact McNemar p = 1.0; q4 slightly faster (18.78 s vs 19.47 s mean) |
| Deep-context integrity | Needle probes at 50%/90% depth, identical ~198k-token haystack, same seed, same-session control | q4: 0/2 hits, zero tripwires, clean stops; q8: 0/2 hits, one trip |
| Retrieval boundary | Needle at 25% depth (where q8 historically retrieves) | FOUND, clean stop — the dumb-zone boundary did not move |
The tripwire is the finish_reason anomaly counter we keep armed on every serving hour since the RDNA4 quantized-KV corruption report made the rounds. One trip fired all night — on the q8 arm (a 40-token answer that never stopped; n=1, plausibly ordinary deep-context rambling, recorded not interpreted). The arm the rule actually gates on — q4 — ran clean.
The miss we caught in ourselves
The gate's haystack was built to an estimate: 5,800 paragraphs, ~45 tokens each, call it 262k. The server's own counter said 198,227 tokens. The paired comparison is untouched — both arms prefilled the identical haystack, which is the entire point of pairing — but every place we had written “262k needle probes” was overstating the instrument. The announcement tweet had already gone out with the estimate. The correction went out as a reply within the hour, the repo README now carries server-reported token counts, and the rule is restated for good: label instruments with the counter at the source, not the generator's estimate. Cold prefill measured through that needle wall: 109.5 tok/s at ~198k (needle-class number: full prompt + answer, not a bench harness).
What half a cache buys
Arithmetic, labeled as arithmetic: q8_0 encodes ~8.5 bits per KV value, q4_0 ~4.5 — hence ~47% (4.3GB saved) off the KV line at identical context. Measured wall time through the probes trended the same way as GSM8K: q4 marginally faster on cache reads. The compounding cases are next: the 35B dress-rehearsal math (where KV headroom decides feasibility), and the adaptive-KV research lane, whose bar just moved from “beat static q8” to “beat static q4.”
Conditions
GMKtec EVO-X2 (Ryzen AI Max+ 395, 96GB unified), Qwen3.8-27B Q4 dynamic quant, llama.cpp ROCm build, 262,144-token window, K+V q4_0 KV (champion as of 2026-08-18 10:32Z), temperature 0. GSM8K n=60 paired; needle n=2 stations per arm plus one boundary probe; one box, one night; tripwire counter armed continuously. Datapoints, not laws. Raw gate log and rollback unit referenced in the repo: qwen38-27b-strix-halo.