By Simon Gonzalez de Cruz (follow the build in public on X @KyaniteLabs_). 2026-08-22. The complete measured answer.
We ended the 27B story with a frozen, measured machine. The next question was obvious: what work can it actually take over? Not “is it smart” — every benchmark answers that. “Can you hand it a job and walk away?” No published benchmark answers that. So we built one.
The method
Nine job classes from real work. Size ladders. Hidden tests the system never sees. A sabotage cell with a planted bad instruction. Certified floors — exact statistics that say “at least X percent reliable.” A walk-away tier: thirty-five perfect trials before any cell earns the “leave it alone” stamp.
The benchmark is open source. The sealed test sets never publish.
The results
495 trials, 29 cells. Every capability cell passed 20/20 (floors 81.9-86.1%). Eight cells earned walk-away certification at 35/35 or 30/30 (floors 90.5-91.8%).
Code: 20/20 at every size. Debugging: 20/20. Document search: 20/20. Trap-bait summaries: 20/20. Data extraction: 20/20. Translation: 20/20. Chained reasoning: 20/20. Decisions: 20/20. Safety: 20/20. Vision: 15/15 on five of six types.
Walk-away certification
Eight cells each ran fifteen additional untouched confirmation trials. Every single one passed: code 35/35, debugging 35/35, document search 35/35, reasoning 35/35, decisions 35/35, vision charts 30/30, vision terminals 30/30. Certified floors: at least 90.5-91.8 percent.
Re-gate update (2026-08-25): deeper n, stronger floors, one honest yellow
The product changed after certification (new front-door work orders, an outage-continuity branch merged to main). So we re-ran the whole benchmark instead of trusting the old card: capability cells at n=25 (350 trials), walk-away confirmations at combined n=40 (90 trials), vision re-gate (30 trials). Total across both runs: 965 trials.
Capability: 13 of 14 cells green at 25/25, floors 88.7% (deeper evidence than the certified 20/20). Walk-away: 6 of 6 cells at 40/40, floors 92.8% — up from 90.5-91.8%. Vision re-gate: 5/6 at 5/5, the form cell an honest near-miss (tiny-print OCR class, labeled). One yellow we kept: debug-medium 23/25 — two genuine misses where the model fixed the described symptom but not the dedup contract. Raw JSONL for everything is in the repo.
Methods receipt: the re-gate caught our own harness in an attribution race under concurrent load — it scored an in-flight session’s empty interim message as the model’s answer. Patched, rescores documented, engine exonerated. The instrument gets audited like the model.
The judge story
Our first graders failed two cells. Reading the raw answers showed the model was RIGHT — our judges were wrong. One broke on a Spanish accent. One could not tell “mentioned the false number to reject it” from “repeated it as fact.” Both bugs are now permanent regression cases.
The honest misses
Giant documents (130k tokens) are impractical — the ceiling is measured; the product now fast-fails with an estimate. Tiny-text transcription: one letter dropped in long email addresses (12/15, labeled).
What this proves
A complete serving story on hardware anyone can buy, measured through a real product, every number reproducible.
The card is public. The benchmark is open source. Point it at your own machine.
Conditions: $1,400 GMKtec EVO-X2, Qwen3.8-27B Q4_K_XL, llama.cpp ROCm, tokflint/tokpal. github.com/KyaniteLabs/delegation-bench.