What is model quantization, and how does it help deploy vision models to edge devices?
Quantization represents a model's weights and activations in lower precision - typically 8-bit integers instead of 32-bit floats. That cuts model size by around 4x and lets edge hardware run the math far faster and with less power, usually for only a small accuracy drop. Post-training quantization is the quick path; quantization-aware training recovers accuracy when the drop is too big.
Edge devices are memory-, power-, and compute-limited. Quantization is one of the highest-leverage ways to fit a model onto them, because integer math is cheaper than floating point on most accelerators.
Two approaches
- Post-training quantization (PTQ): convert an already-trained FP32 model to INT8, using a small calibration set to pick value ranges. Fast, no retraining; may cost a little accuracy.
- Quantization-aware training (QAT): simulate low precision during training so the model adapts to it. More work, but recovers most or all of the lost accuracy.
With Ultralytics you can export a quantized engine directly. INT8 needs calibration data that looks like your real inputs:
from ultralytics import YOLO
model = YOLO("yolo11n.pt")
# INT8 TensorRT engine; 'data' provides calibration images
model.export(format="engine", int8=True, data="data.yaml")Always re-validate accuracy after quantizing - most models lose very little, but a few are sensitive, and you want to catch that before it ships.
Related questions
Hands-on tutorials
Stuck on this in a real project?
Book a free call and I’ll give you a straight answer on your specific case.
Book a consultation