Free Resource — Cheat Sheet

The LLM Comparison Cheat Sheet

Stop reading benchmark tables. One card with the one concept that matters (the Pareto frontier), what intelligence scores mean practically, what models actually cost, and the five questions that pick a model for your task. Numbers from the live Model Observatory dataset.

The chart moves. The card prints.

Prices reprice and the frontier shifts — three Qwen models repriced 22–60% in one August weekend. Build Notes carries the monthly shifts; the live chart has the current numbers. No spam, unsubscribe any time.

You are on the Kyanite Build Notes list.

The one concept: the Pareto frontier

A model is on the Pareto frontier if no other model is simultaneously cheaper, faster, and smarter. A model off the frontier is dominated — something else beats it on at least one axis without losing on any other. Reading benchmark tables, you reconstruct that frontier in your head, badly. Here it is, drawn:

$0.02$0.05$0.20$0.50$2$10$/M 0204060 AA INDEX typical product floor: AA 45–55 Gemma 4 31B · $0* Hy3 · $0.10 · AA 42 GPT-5.6 Luna (xhigh) · $0.17 · AA 50 Qwen3.8-27B (xhigh) · $0.44 · AA 52 Grok 4.6 · $1.35 · AA 61 Claude Opus 5 · $3.85 · AA 63 dimmed dots: dominated models · bright filament: the Pareto frontier
28 real models from the Model Observatory snapshot (2026-08-26), intelligence vs blended price. 2D projection of the live 3D chart, which adds the speed axis · *$0 = promotional pricing, flagged non-durable in the live data · live version: viz.kyanitelabs.tech

Two honest things this projection shows immediately. The frontier spans $0.17 to $3.85 blended in this snapshot — the smartest model on the chart is not 20× the price of the cheapest one that clears a typical floor. And our own local Qwen3.8-27B sits on the frontier at AA 52 / $0.44 hosted (the settings card tells you how to run it yourself).

Intelligence tiers — what the number buys you

The AA Index is a cross-benchmark intelligence score. Practical translation, from the framework we use:

AA IndexWhat it means practically
Below ~40Expect hallucination at a rate that requires human review on every output. Only worth it for work a validator fully checks.
~40Clears the bar for summarization pipelines with spot checks and machine-verifiable extraction.
45–55The typical product floor. Below this band you pay for errors; far above it you buy intelligence your task may not use.
60+Verification-heavy work: code-review agents, multi-step reasoning, decisions where a miss is expensive.
Frontier regionThis snapshot tops out at AA 63.1, with 51 of 313 measured models clearing AA 45 and 20+ crowding the frontier itself.

The floor is task-specific. A creative-writing assistant may be better below the frontier entirely — frontier models optimize for correctness, not voice. Set the floor for your task, not for your ego.

Cost bands — what models actually cost

Blended price per million tokens (input and output weighted by real task mixes — the number you actually pay, not the marketing input price). Snapshot 2026-08-26 from the live dataset; the chart rebuilds three times daily:

BandBlended $/MWhat lives here / what it is for
BUDGET < ~$0.50 Half the chart (median model: $0.47/M). Narrow, repeatable, machine-verifiable work: classification, extraction, summaries, constrained transforms. The cheapest model currently clearing the AA-45 floor — GPT-5.6 Luna — lives here at $0.17/M.
MID ~$0.50–2.30 Most product work. The bulk of the frontier clusters here (Gemini 3.7 Flash, GLM-5.3, Grok 4.6). The top decile of the market starts around $2.30/M.
PREMIUM $2.30–12 Discovery, hard reasoning, high-stakes review where a miss costs more than the tokens. Includes dominated models — check the frontier before paying the band.

Open weights are not free to use. They cost nothing to download and real money to run — the chart prices everything at the hosted API rate because that is the only number comparable across all 313 models. And prices move fast: three Qwen models repriced 22–60% in one August weekend. Free promotional pricing is flagged non-durable in the live data, with a canary watching for the real price.

The five questions that pick the model

Every LLM comparison article starts with the models. Start with your task instead:

  1. What is the intelligence floor for this task?

    Not the smartest model — the minimum bar. 45–55 AA for most product work; ~40 for verified pipelines; 60+ for review-heavy work. The floor eliminates the models that will waste your time with errors.

  2. Of the survivors, what do they cost per million tokens?

    Blended input+output price, weighted like your real task mix. A model charging $0.25 in / $1.25 out is not “$0.25 per million.”

  3. What speed does the workload need?

    Real-time chat buys time-to-first-token; batch pipelines wait minutes for cheaper inference; code review waits for intelligence. Each workload picks a different point on the frontier.

  4. Is the candidate on the Pareto frontier?

    Off the frontier means dominated: you are paying a premium or accepting a penalty you could avoid. In 2026 the frontier is crowded (20+ models) — the real question is which frontier model fits your cost envelope.

  5. Did it survive your own eval?

    Benchmark scores are proxies. Run 20–50 of your actual prompts through the top 2–3 candidates — hidden tests, a sabotage probe. We built delegation-bench for exactly this; the eval design checklist is the short version.

Do steps 1–4 in thirty seconds

The Model Observatory plots all 313 models in 3D — cost, intelligence, speed — with the frontier visible as the white filament. Decide mode sets your floor, picks your objective (cheapest / fastest / balanced), and returns a named shortlist with exact numbers. Free, no account, open source (MIT), every price traceable to its source with a timestamp.

Sources

Live dataset: viz.kyanitelabs.tech (snapshot 2026-08-26; blended-price distribution and frontier points read from the published dataset) · Framework: How to Choose an LLM · Costs: LLM Cost Comparison · Frontier: The LLM Pareto Frontier · Observatory build: We Built an Open-Source Model Observatory.