Quantisation, and the memory it returns
Quantisation stores the same weights in a smaller numeric format. The parameter count does not move; the bytes per parameter do, which is why memory figures for one model can differ by a factor of two or more. As of 2026-09-22.
| Property | Effect of quantising |
|---|---|
| Parameter count | Unchanged |
| Weights in memory | Roughly proportional to bytes per parameter |
| Output | Changed, by an amount that depends on the model |
| Card requirements | The format has to be supported in hardware |
Inclusion rule. Properties affected by a change of numeric format. A property that does not change is listed, because that is the point of the term. Order. From the property that never changes to the one that decides whether a card qualifies.
1The format has to exist on the card
A 6GB figure that assumes bf16 does not transfer to a card that cannot do bf16. One repository in this register names the card generations and the formats together for exactly this reason.
So a quantised build lowers the memory and narrows the hardware at the same time. Both halves belong in a hardware statement and only one of them is a number.
2Why this is a size term and a hardware term
The operation belongs to the file, which is why it is filed here. The consequence belongs to the run, which is why the hardware column carries the figure.
One release in the register states at least 24GB for a model and at least 12GB for a distilled fp8 configuration of it. Same weights, two floors, and the difference is software.
3Where this shows up in the hardware column
Four families name a numeric format beside a memory figure, and one of those states two floors for one model: a higher figure for the release and a lower one for a lower-precision build of it.
Where no format is named, the register does not assume one. Assuming half precision would put a convention into a cell that the vendor never stated, on rows where a reader has least to check against.
A reading note, not an entry: no vendor value appears on this page. Where the register records this term for a particular release, it is on the size column and on that family's own page. Nearby terms: Autoencoder, Diffusion transformer.