What is multi-object tracking, and how does the SORT algorithm use Kalman filters?

Question

Muhammad Rizwan Munawar · Accepted Answer

Multi-object tracking assigns each detected object a consistent ID as it moves through a video, so 'car #7' stays #7 frame to frame. SORT does this cheaply: a Kalman filter predicts where each existing track should appear in the next frame, those predictions are matched to the new detections by IoU using the Hungarian algorithm, and the filter updates. It's fast because it uses only motion and box geometry - no appearance model. Detection tells you what's in a single frame. Tracking stitches those frames together so you can count, measure speed, or follow a path. SORT (Simple Online and Realtime Tracking) is the classic lightweight tracker. The loop, per frame Predict: a Kalman filter estimates each existing track's new box from its motion (position and velocity). Match: compute IoU between predictions and the frame's fresh detections, then solve the assignment with the Hungarian algorithm. Update: matched tracks absorb the new detection; unmatched detections start new tracks; tracks unseen for a while are deleted. The Kalman filter is what makes it smooth and robust to a missed frame or two: it carries a motion model, so a briefly-occluded object is still predicted forward. SORT's weakness is that it only uses motion, so IDs can switch when objects cross or occlude - which is exactly what DeepSORT set out to fix. from ultralytics import YOLO

model = YOLO("yolo11n.pt")
# persist=True keeps track IDs stable across frames
for result in model.track("traffic.mp4", persist=True, tracker="bytetrack.yaml", stream=True):
    for box in result.boxes:
        track_id = int(box.id) if box.id is not None else -1
        print("id", track_id, "class", model.names[int(box.cls)])

What is multi-object tracking, and how does the SORT algorithm use Kalman filters?

The loop, per frame

Related questions

Hands-on tutorials

Stuck on this in a real project?