How does the Feature Pyramid Network (FPN) handle multi-scale object detection?

Deep in a CNN, features are semantically rich but spatially coarse - great for big objects, useless for tiny ones. Early layers are the opposite: high resolution but semantically weak. FPN's job is to give you the best of both at every scale.

Two paths plus lateral links

• Bottom-up: the normal backbone, producing feature maps that get smaller and deeper.
• Top-down: upsample the deep, semantic maps back up to higher resolutions.
• Lateral connections: merge each upsampled map with the matching bottom-up map so it gains both meaning and spatial detail.

The detector then makes predictions at multiple pyramid levels, so a person far away (small, fine map) and a person up close (large, coarse map) are each handled where they're clearest. This 'neck' between backbone and head is standard in modern detectors, including YOLO, which uses FPN/PAN-style necks.

In practice

This is the mechanism behind a fix I use constantly: when a model misses small, distant objects, raising the input image size (imgsz, say 640 to 1280) helps because it feeds more detail into the fine pyramid levels the FPN uses for small objects. It's also why tiling a large image into crops before inference works so well for tiny objects - you're effectively giving those fine levels a bigger, sharper signal to work with.