---
title: Quantization
description: 'Mapping model values to a lower-precision representation to reduce storage, memory traffic, or computation.'
canonical_url: 'https://darkfactory.dev/glossary/quantization'
markdown_url: 'https://darkfactory.dev/glossary/quantization.md'
collection: glossary
date_published: '2026-08-03T00:00:00-04:00'
date_modified: '2026-08-26T00:00:00-04:00'
---

# Quantization


## Definition

Quantization maps model values from a high-precision representation to a smaller set of numerical levels. A common scheme represents a real value with a low-bit integer plus a scale and, for asymmetric ranges, a zero point. The target may be weights, activations, gradients, or cached attention keys and values. Those choices create different memory, speed, and quality tradeoffs.

Post-training quantization estimates ranges or calibration statistics after a model has been trained. Quantization-aware training simulates low-precision effects during training so the weights can adapt. Schemes can assign one scale to a tensor, separate scales to channels or groups, or different bit widths to different layers.

Benoit Jacob and colleagues' 2018 CVPR paper described an influential system for integer-only neural-network inference and a training procedure that simulated quantization. It demonstrated one practical design for mobile CPUs, not the origin of numerical quantization.

Fewer bits reduce model storage and often reduce memory traffic. Actual latency and energy gains depend on kernels and hardware that can execute the chosen format efficiently. A nominally 4-bit model may dequantize values into a higher-precision compute type, and unsupported operators may fall back to slower code. Quality loss is also uneven: some layers and outlier values are more sensitive than others.

## Distinguish it from nearby terms

Quantization changes how values are represented. Pruning removes weights or structures. Distillation trains a student model to imitate another model. These methods can be combined, but they change different parts of the system.

## Check your understanding

A 4-bit checkpoint is smaller on disk but slower than its 8-bit version on the target GPU. Which measurements would show whether dequantization, unsupported kernels, or memory bandwidth explains the result?

## Related terms

- [Quantized low-rank adaptation (QLoRA)](https://darkfactory.dev/glossary/quantized-low-rank-adaptation)
- [Parameter](https://darkfactory.dev/glossary/parameter)
- [Inference](https://darkfactory.dev/glossary/inference)
- [Model distillation](https://darkfactory.dev/glossary/distillation)

## Related factory areas

- [Model selection, routing & budgets](https://darkfactory.dev/factory/model-routing-budgets)

## Evidence and further reading

- [Google Machine Learning Glossary](https://developers.google.com/machine-learning/glossary/)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://openaccess.thecvf.com/content_cvpr_2018/html/Jacob_Quantization_and_Training_CVPR_2018_paper.html)