Choosing between full, distilled and quantised builds
Where a release publishes distilled and quantised builds beside a full model, the choice between them is a choice between three models rather than three settings. Each has its own memory, its own speed and its own output. As of 2026-09-12.
| Property | Changes between builds |
|---|---|
| Memory needed | Yes, often substantially |
| Nominal parameter count | No, for a quantised build |
| Output behaviour | Yes, for a distilled build |
| Card eligibility | Yes, narrower at lower precision |
Inclusion rule. Properties that differ across builds of a single release. A property that does not change is listed, because it is the one people expect to. Order. From the property that motivates the choice to the constraint it introduces.
1A distilled build is not the full model running faster
Distillation trains a smaller network to reproduce a larger one's outputs. The result is cheaper and behaves differently, and a quality claim made against the full model does not describe it.
Both are published under one family name, which is how two incomparable benchmarks end up quoted beside each other. A fair comparison states the build every time.
2Quantising narrows the hardware as it lowers the memory
A build at a smaller numeric format needs a card whose hardware supports that format, so the cheaper configuration runs on fewer machines rather than more. A card with plenty of memory and an older architecture does not qualify.
Where a repository states card generations and formats in the same sentence, it has closed that gap. Where it states a figure alone, the eligibility question is left open.
3Shipping builds is itself a hardware answer
Nobody spends engineering time on a lower-precision build unless memory is binding for part of the audience. The existence of the build is evidence, and a stronger signal than a single minimum figure would be.
It is also unusable as a value. There is no column for a download list, so a release that answers this way leaves the hardware cell empty while not being silent.
4Pick by the constraint that actually binds
If the binding constraint is card memory, the quantised build is the candidate and the format support has to be checked first. If it is throughput, the distilled build is the candidate and the output has to be reviewed.
If neither binds, the full model is the reference and there is no reason to accept either compromise. Choosing a smaller build without a constraint is paying in quality for nothing.
5Keeping several builds costs disk
A team comparing builds will download all of them, and a weights file does not shrink because a smaller sibling exists. Storage is the sum rather than the largest, and it is the first hard limit a local deployment hits.
It is also the one quantity a reader can derive safely from a parameter count and a format, which makes it worth calculating before anything is downloaded.
6Test the build you will ship
A pilot run on the full model does not describe the distilled one, and a timing on the quantised build does not describe either of the others. Whichever build a plan intends to use is the one that has to be tested.
That sounds obvious and is routinely skipped, because the full model is the one with published figures and the smaller one is the one that fits. The figures and the plan then describe different objects.
7Where the repository figures are kept
The techniques above are general. Which vendors have published what, under which licence and on what hardware, is recorded on the model pages, each figure quoted from the repository it was read from with its date.
- Model variant — what a sibling changes
- Distillation — cheaper, and different
- Quantisation — the memory it returns
Craft notes on reading releases. No row here is attributed to a vendor, and nothing on this page is a reading of anyone's licence obligations. The sourced material is on the requirements page. Related: Recording a reading, Reading a licence.