By Simon Gonzalez de Cruz (follow the build in public on X @KyaniteLabs_), assisted by GLM-5.3. 2026-08-25. Every number below was re-read from raw result files on this machine before writing.
We run a small AI lab on hardware that fits under a TV: one AMD Strix Halo mini-PC with 64GB of shared memory. A Qwen3.8-27B has been our main model and daily engine here since mid-August. This week we asked a harder question: can this same box also run Ornith-1.5-35B, a bigger open-weights model, at the same time, and is it actually any good when you measure it fairly?
The answer is yes, with three caveats. Here is everything, including what failed. One number below was first measured wrong by our own tool. Story at the end.
The setup
One machine. Two models, resident at the same time, no switching:
- Qwen3.8-27B (our "champion", serving the product): 262k context, vision attached.
- Ornith-1.5-35B-A3B (the newcomer, thinking-on lane): a Mixture-of-Experts build that only activates ~3B parameters per token. We run an APEX-Compact quantization at 17.4GB. Together the two models use 43.4 of 64GB, with about 20GB to spare.
That "to spare" matters: this only fits because of a memory trick we checked (see below). The stock quant at ~22GB also fits, but with less headroom.
The scoreboard, every number paired, same problems, same machine
We never compare two numbers that came from different places. Both models ran the same fixed problem sets, back to back, on the same box:
| Test | Champion 27B | Ornith 35B | Read |
|---|---|---|---|
| Grade-school math (GSM8K, strict grading, paired 2×2, same 60 problems, n=60/cell) | 96.7% thinking off; 96.7% thinking on (identical outcomes, McNemar p=1.0) | 90.0% thinking off; 98.3% thinking on (p=0.0625, marginal at n=60) | At each model's best config: a one-problem tie (p=1.0). Thinking costs the champion 2.4× time for zero gain; Ornith needs it |
| Code, clean problems (HumanEval, same frozen 30) | 28/30 | 26/30 | Tie within noise |
| Code, real-world (LiveCodeBench, same 30) | 20/30 | 17/30 | Slight edge champion, not decisive at n=30 |
| Vision, same 6 real screenshots | 6/6 | 5/6 | Ornith misread an email address in a form (typo'd the domain) |
If you only remember one row: a 35B reasoning model now runs shoulder to shoulder with our daily driver on this class of hardware. Until paired measurements like this, that was a forum argument.
The three caveats
- Thinking is a dial, not a model gap. Measured paired (2×2, same problems, same protocol): the champion scores 96.7% with thinking off AND on (identical outcomes, p=1.0), so its thinking switch buys nothing here at 2.4× the time. Ornith scores 90.0% without thinking and 98.3% with it. At best configs the two models tie within one problem (p=1.0). Earlier numbers that compared Qwen thinking-off against Ornith thinking-on were regime-mismatched; this paired table is the truth and it replaces them.
- Code is a tie, not a win. 26/30 vs 28/30 on identical problems is within noise. We will not claim a code champion from this data.
- Vision has one miss. On a real signup-form screenshot, Ornith returned an email with the domain misread. That is a wrong answer, not a near-miss. Our champion got all 6. Small sample, labeled: 5/6.
What did NOT work (the part few publish)
We tested speculative decoding two ways, because it promised free speed:
- The trained draft head shipped in APEX builds: accepted only ~33% of draft tokens, and made generation ~9% slower end to end. Dead on this hardware, measured, twice.
- An improvised draft: same ~33% class. Same verdict.
Speculative decoding is a memory-bandwidth trick, and this machine's bandwidth goes to the two resident models. The right setting is OFF, and anyone running a similar box should start there instead of paying our discovery cost.
The memory trick that made it fit
The APEX-Compact build packs the 35B into 17.4GB (vs ~22GB stock at the same Q4 quality tier). We could not measure a quality cost at these sample sizes: grade-school math 95.0% vs 96.7% stock on n=60, and identical results on a 15-problem code reasoning set. That ~4.3GB saving is the difference between "one model at a time" and "both models always on," which is the whole point of the machine.
Speed, labeled
- Writing a long answer: ~55 tokens/second sustained. A 200-word reply lands in about 6-7 seconds. A full thinking run on a hard problem: minutes.
- Reading long documents (prefill): we ingested 130,715 tokens, a whole book, in 359 seconds (~364 tokens/second), with the chip at 72°C the whole way. Long-context work is where this MoE build earns its keep.
Our settings card (if you have the same box)
Greedy decoding (temperature 0), q4_0 KV cache, speculative decoding OFF, APEX-Compact Q4 weights, math budget 2048 tokens, hard-code budget 8192 (2/30 problems still hit the cap, labeled, not hidden). Nothing here is exotic; all of it was measured against at least one alternative on this machine before we kept it.
Why publish all of it
Most "I ran a big model locally" posts show one screenshot and one tokens-per-second number. We think the valuable artifact is the paired, same-fixture table with the failures included, including the run where our own grading tool truncated the model's output and briefly told us the model was bad at math (0.467). It wasn't; our instrument was wrong. That difference is visible in the two saved configs. Same problems: 0.967. Both JSONs are cited below, because the instrument should be suspect as often as the model.
Sources (raw result files, on-box paths): GSM8K paired 2×2 ~/exp/2x2-gsm8k/summary.json (champ 58/60 both cells; orn 54/60 off, 59/60 on; McNemar p=1.0/0.0625/1.0; median walls 28.3/68.5/11.4/23.3s; run 2026-08-24T19:42Z, live doors, self-test read before scoring). GSM8K Ornith stock corrected ~/exp/w1-35b/gsm-full-b/Ornith35B/results_2026-08-22T01-26-41.json (strict 0.9667); the false start results_2026-08-22T01-00-27.json (strict 0.4667, instrument truncation). APEX-Compact ~/exp/w4-35b/gsm-run.log (0.9500). HumanEval-30 Ornith ~/exp/w1-35b/bench-results-default.log (26/30); champion 28/30 (~/exp/bench-results-default.log, verified 2026-08-24). LiveCodeBench-30 Ornith ~/exp/w3b-35b/verdict.txt (17/30, trunc 2/30); champion 20/30 (recorded in the comparison design doc, READ; result bytes not yet located, labeled). LCB-15 parity ~/exp/w4-35b/lcb15-run.log (7/15, identical to stock). Vision real-UI: Ornith ~/exp/vision35b-results.log (5/6); champion ~/exp/vision-real-results.log (6/6). Decode speed ~/exp/w4-fire.log (no-spec 55.9 tok/s; spec slower 50.4 vs 55.2, paired, same binary). Prefill ~/exp/ornith-window-results.log (wall=359s, 130,715 tokens, 72°C). Speculative acceptance ~0.33: window-4 banked comparators. Memory: APEX-Compact 17.4GB weights; dual-resident 43.4/64GB GTT (live readback 2026-08-23). Regime note: earlier champion GSM8K readings of 70% strict and 98% predate the paired protocol and are superseded by the 2×2 for this piece.