Local LLM / Benchmarks

Lab Notes: the model that can't forget but can't remember

The number first: this model is 75% not a transformer. 48 of 64 layers keep a running state. On a $1,400 rig, load 198k once (1818s), then query in 9-27s.

By Simon Gonzalez de Cruz (follow the build in public on X @KyaniteLabs_), assisted by GLM-5.3. Night 4.

The number first: this model is 75% not a transformer. 48 of its 64 layers run Gated DeltaNet: linear attention with a fixed-size running state. Only 16 layers do what you would call attention. The KV cache lives in those 16 layers only. That is why our 4-bit KV flip saved 4.3GB at 262k, not the ~15GB dense-transformer math predicted.

Hardware: GMKtec EVO-X2, Ryzen AI Max+ 395, 96GB unified, $1,400. Model: Qwen3.8-27B UD-Q4_K_XL. llama.cpp. Native window 262,144 tokens.

The KV curve was flat because the cache is small

Paired gate, n=60, q8_0 vs q4_0: both 96.67%, McNemar p=1.0, zero discordants, zero corruption tripwires. q5_0 and q5_1 matched the same accuracy. q4_0 is champion because quality did not move and we keep the 4.3GB. Raw: results/kv-curve-2026-08-19.

Load once, query many

We loaded a 198k-token prefix on this $1,400 box. Cold prefill: 1818s (~30 min). Then four follow-ups against the warm prefix, all finish_reason=stop: retrieve the planted code; quote the sentence (16s); yes/no (9s); summarize (27s). Retrieval is selective. Distinctive strings survive. Night 3's arbitrary-ID misses were the same llama.cpp c7d8722 buffer bug; the post-revert remap retrieved planted IDs at all six tested depths at 198k. See the Night 3 inversion. Raw: quote-results.log.

The quote probe is the product shape: loading is expensive; maintaining is cheap. Pre-revert 50k misses are quarantined with the old binary. We have not re-run 50k on the fixed build.

What we got wrong

We published KV arithmetic that took three passes to land (15 → 11.5 → 7.4 → 4.3GB). We blamed “the fork.” The serving binary is upstream-era llama.cpp plus a small HIP cherry-pick. We treated a memorable-code hit as a position window. n=1 cells stay maps, not laws.

Thinking budgets on screened-hard GSM8K: think_med=241ch across off/512/1024/2048/65536 in the kneemap log. Caps ≥512 were non-binding on that set. Raw: kneemap-gsm8k-hard.log.

I am not publishing a spec-decode speedup number in this note. The committed writeup exists; the arm log file is empty. We do not ship that row until the log has rows.

Conditions: $1,400 GMKtec EVO-X2, Qwen3.8-27B Q4_K_XL, llama.cpp ROCm, 262,144-token window, K+V q4_0, temperature 0. Night 4. A map with linked logs.

Work with Kyanite

Want this working in your environment?

If this post describes a Kyanite tool or result you need, implementation help can cover setup, advising, docs, examples, checks, and a usable handoff.

Fit boundary

Kyanite offers help grounded in its tools, products, and build practice. Broader consulting routes through PuenteWorks.

Keep following the system.

The Delegation Card: we asked a $1,400 mini-PC to take our jobs

Not is-it-smart but can-you-hand-it-work-and-walk-away. 495 trials, certified floors.

Qwen 3.8 27B on Strix Halo: the complete measured story

Every dial measured, every number public: the frozen optimal config for a 27B on a $1,400 mini-PC.

Lab Notes: the measured-knees method for reasoning effort

A methods note on reasoning-effort calibration. Thinking rescued 15/40 hards vs 4/40 off. On easy tasks it bought nothing. Publish the knee.

Lab Notes: 67% LiveCodeBench-30 on a $1,400 rig

20/30 = 67% LiveCodeBench-30 on a $1,400 mini-PC. Wilson 95% CI 49-81%. Easy 10/10, medium 8/10, hard 2/10. n=30 public subset. Not the card.

Lab Notes: we reverted a llama.cpp regression

We found a llama.cpp regression and reverted it. n=6 battery on a $1,400 rig: 0/6 slashes before, 5/6 after. Same Q4_K_XL. Not a quant story.

Lab Notes: 93% HumanEval on a $1,400 rig

28/30 = 93% HumanEval on a $1,400 mini-PC. Qwen3.8-27B Q4_K_XL. Temp 0, thinking off. Failures: 50 and 145. Raw log linked.

Lab Notes: the basin was a bug

We published a basin. The serving binary was the hole. After the c7d8722 revert: 6/6 HIT at 198k. n=1 map of the fixed build.

Lab Notes: the KV verdict

The q8-to-q4 KV question from night 1 got its paired answer: same accuracy, zero tripwires, half the cache. Plus the label we had to correct in public when the server's counter beat our estimate.

Lab Notes: verdict night

A $1,400 mini-PC serving a 27B at 262k context asks one question: does capping thinking cost accuracy? The paired answer, the invalid first attempt it survived, and the doctrine that followed.

One mini-PC, one night, and the numbers that argued with themselves: tuning Qwen3.8-27B on Strix Halo

A fully measured night-and-evening of tuning a 27B dense model on a Strix Halo mini-PC: the honest bands, the reversal, the crash doctrine, and the EC detective story.

GPT-5.6 Sol vs. Terra vs. Luna: an evidence-based routing policy for coding agents

The practical GPT-5.6 split is Sol for discovery, Terra for bounded execution, and Luna for repeatable processing - with a different verification contract for each.

Agents need verifiable tools, not better prompt theater

The useful agent pattern is not a prettier prompt. It is a tool surface the agent can call, inspect, verify, and revise.

Repo history is a product signal

A repo is not just storage. It is evidence of decisions, repairs, release behavior, naming drift, test gaps, and what the builder actually knows how to finish.

Implementation help is part of the product surface

A useful open-source tool still needs a path from public repo to working environment. That path is product work, not an afterthought.

Why Kinocut matters

Kinocut gives AI agents callable handles on timelines, effects, Hyperframes, and finished media at kinocut.dev.

Infinite monkeys, LLMs, and the room around the machine

The argument behind the video: output quality is not just probability. It is architecture, filters, and human taste.

What a working AI tool needs before people can use it

A practical checklist for turning a working tool, workflow, or rough app into something other people can understand, install, and use.

MCP server implementation checklist

The checklist Kyanite uses to decide whether an MCP server is a toy, a usable tool, or something worth implementing.

Repo archaeology turns history into proof

Why commit history is one of the strongest proof sources for learning diagnostics, implementation help, and engineering trust.

AI discovery needs more than a sitemap

What Kyanite adds so search engines and AI assistants can understand the tools, products, proof, and support path.