The rig
GMKtec EVO-X2 — AMD Strix Halo (Ryzen AI Max+ 395), 96 GB unified memory, $1,400. Open tooling only: llama.cpp (ROCm build) and one free Apache-2.0 model. No cloud, no rentals. The whole story — including the upstream llama.cpp bug we caught, bisected, and validated the fix for — is on the KyaniteLabs blog.
The frozen config
Each setting below was chosen by measurement, not by folklore. The trade-offs are printed with the numbers that decided them.
| Dial | Setting | Why (measured) |
|---|---|---|
| Weights | Qwen3.8-27B Q4_K_XL Unsloth dynamic quant |
Q3_K_XL won the raw tok/s ladder but lost on time-per-task: its reasoning ran ~2× more verbose, so Q4 finished identical correct tasks 35–50% faster. Faster tokens are not better if the tokens are dumber. |
| Context | 262,144 tokens full native window |
Exact needle retrieval at every tested depth, two seeds, up to 261,130 / 262,144 tokens — 99.6% of the window. Format does not matter: prose or code, exact either way. |
| KV cache | K+V q4_0 light |
Measured trade: the heavy q8_0 option buys up to ~3 s on follow-up questions (sub-second to 2.8 s at the half window) for ~4 GB of memory. We kept the room. Paired gate n=60: q8 vs q4 both 96.67%, McNemar p=1.0, zero discordants. |
| Speculation | shipped MTP + ngram stack | Verified fastest of four candidate setups by paired walls: 15.1 s per 200-word answer vs 17.8 s with speculation off. Greedy completions come out byte-identical. |
| Thinking | off by default hard problems think |
Measured across difficulty bands: thinking rescued 15/40 hard-class problems vs 4/40 without — a real gain exactly where tasks are hard. On HumanEval-30 it bought nothing (28/30 identical either way). |
| Vision | enabled mmproj-F16 |
6/6 on real browser screenshots (after the llama.cpp host-buffer regression was reverted — the fix trail is public). |
| Temperature | 0 | Reproducibility first: a single-config cell showed a ±2-problem noise floor at temp 0 on LiveCodeBench-30. Freeze what you can. |
What it delivers
| Benchmark | Result | Conditions |
|---|---|---|
| GSM8K | 96.7% 58/60 | Paired 2×2, n=60, thinking on and off identical (McNemar p=1.0). Strict final-answer grading. |
| HumanEval-30 | 93% 28/30 | Temp 0, thinking off, frozen 30-problem subset, seed 20260819. Failures: HumanEval/50 and /145 (runtime errors in generated code). |
| LiveCodeBench-30 | 67% 20/30 CI 49–81% | Wilson 95% interval — always cite the interval. Split: easy 10/10, medium 8/10, hard 2/10. n=30 public subset, not the official full-set card. |
| Needle retrieval | 99.6% of window | Exact retrieval at every tested depth, two seeds, up to 261,130 of 262,144 tokens. Warm follow-ups on a loaded 198k doc: exact quote 23.8 s, yes/no 10.0 s, one-line summary 17.5 s. |
| Vision (real UI) | 6/6 | Real browser screenshots, structured answers. |
The speed regimes — honest bands
Decode tok/s is not task latency, and warm numbers are not chat numbers. These are the bands we publish instead of a single hero figure:
| Regime | Band | Label |
|---|---|---|
| Cold decode | 59.7–64 tok/s | The benchmark-band number. |
| Real chat (novel prose) | 11–24 tok/s | The traffic that actually flows. Published first by us. |
| Code | ~29–40 tok/s | Structured generation. |
| Warm repeats | 148–163 tok/s | artifact ngram repetition effect — real on repeated structured output, meaningless as a chat claim. |
| 200-word answer (wall) | 15.1 s | With the shipped speculation stack (17.8 s without). |
The honest ceilings
- Giant documents are impractical. Around 130k tokens the wall-time stops paying; the ceiling is measured and the product now fast-fails with an estimate instead of silently crawling.
- Tiny-print OCR is a near-miss. One letter dropped in long email addresses (12/15 on that cell, labeled).
- Cold prefill of a 198k prefix took ~30 minutes (1818 s). Loading is expensive; maintaining is cheap. Design for load-once, query-many.
Print this card
Hit Ctrl/Cmd-P — the card prints clean on white without the site chrome.
Sources — every number traces
Arc-close post: Qwen 3.8 27B on Strix Halo: the complete measured story · Config raw log: results/config-27b-2026-08-21 · Stack repo: github.com/KyaniteLabs/qwen38-27b-strix-halo · HumanEval: 93% HumanEval on a $1,400 rig · LiveCodeBench: 67% LiveCodeBench-30 · GSM8K paired: two models, one mini-PC · Tuning story: one mini-PC, one night