What is ByteTrack and how does it work?
ByteTrack is a multi-object tracker whose trick is to keep low-confidence detections instead of discarding them. It matches high-score boxes to tracks first, then does a second association pass using the leftover low-score boxes to recover objects that are briefly occluded or blurred. It uses motion (a Kalman filter) and box overlap only, no appearance model, which makes it fast and accurate.
Most trackers throw away any detection below a confidence threshold before they even start matching. ByteTrack's insight (Zhang et al., ECCV 2022) is that those low-score boxes are often real objects that got dim for a moment - partially occluded, motion-blurred, or small - so discarding them is exactly what causes lost tracks and ID switches.
How BYTE association works
- Split each frame's detections into high-score and low-score boxes.
- First pass: match the high-score boxes to existing tracks using motion (Kalman filter prediction) and IoU overlap.
- Second pass: take the tracks that are still unmatched and try to match them against the low-score boxes, recovering objects that dimmed instead of dropping them.
- Anything still unmatched starts a new track or is aged out.
Notice what is NOT here: there is no appearance/re-identification network. ByteTrack associates on motion and overlap alone, which is why it is fast and light while still posting strong MOTA and IDF1 on the MOT benchmarks. It is the default tracker in Ultralytics.
Run ByteTrack with YOLO
from ultralytics import YOLO
model = YOLO("yolo11n.pt")
# tracker="bytetrack.yaml" is the default; persist=True keeps IDs across frames
results = model.track("traffic.mp4", tracker="bytetrack.yaml", persist=True, show=True)When ByteTrack struggles
Because it has no memory of what an object looks like, very long occlusions, heavy crossings, or strong camera motion can still cause ID switches. When that hurts you, move to an appearance-aware tracker such as BoT-SORT (which adds re-ID plus camera-motion compensation), or DeepSORT. For most detection-and-counting pipelines, though, ByteTrack is the fast, reliable default.
Related questions
Hands-on tutorials
Stuck on this in a real project?
Book a free call and I’ll give you a straight answer on your specific case.
Book a consultation