Audit a detection dataset before it wastes a training run
Command-line linter for object detection datasets. Point it at a folder of images and labels and it tells you what is broken, duplicated, mislabeled or suspicious, as a terminal report, a browser dashboard or a self-contained HTML file. Installs with pip and reads your data where it already sits.
pip install cvflowv0.0.3 · Python 3.9 or newer · pyyaml and pillow, nothing else · release notes
Install
- 1pip install cvflow, into whatever environment your training already runs in. Two runtime dependencies, no build step, no Node and no npm.
- 2Run cvflow inspect with no path at all. It audits the sample coco128 dataset, so you read a real report before pointing it at your own data.
- 3Run cvflow inspect ./dataset --serve to audit your dataset and open the dashboard on 127.0.0.1. The server is local and the data never leaves the machine.
Findings you can act on, not a score
Every finding carries its evidence
Nothing reports a bare verdict. Each issue arrives with its severity, the files and boxes it came from and a suggestion, so you can judge it rather than take it on trust.
Deterministic, not a second model
Validation, statistics and hashing only. No inference, no network calls, no API key: the same dataset gives the same report today and in six months, which is what makes it usable as a gate.
A dashboard, not a wall of text
--serve opens Overview, Classes, Geometry and Findings in the browser: health metrics, class tails, box size and spatial distributions, and filterable issue cards with the actual images.
Fix labels where you find them
Drag a box to move or resize it, drag on the canvas to add one, reassign a class or delete an annotation. Edits are written back to the YOLO label files on disk.
Fails a build, not just a terminal
Exit codes make it a CI gate: 0 clean, 1 problems found, and --strict promotes warnings to failures. --html writes one self-contained report to attach to a pull request or upload as an artifact.
How you run it
See it work before you trust it
With no path, CVFlow audits the sample coco128 dataset and prints the same report it would print for yours.
pip install cvflow
# Audits the sample coco128 dataset
cvflow inspectAudit your own dataset
The format is detected from the layout, and --format forces it when a folder is ambiguous. --no-images skips every pixel-reading check, which is the fast structural pass.
cvflow inspect ./dataset
cvflow inspect ./dataset --format yolo
# Fast, structure only
cvflow inspect ./dataset --no-images
# Hide the statistics section
cvflow inspect ./dataset --no-statsOpen the dashboard
A local server, bound to 127.0.0.1 by default. --no-browser prints the URL instead of opening one, which is what you want over SSH.
cvflow inspect ./dataset --serve
cvflow inspect ./dataset --serve --port 8080
cvflow inspect ./dataset --serve --no-browserGate a pipeline on it
--strict turns a warning into a non-zero exit, so a duplicate or a leaked split stops the run. --html leaves a report behind for whoever reads the build.
- name: Check dataset quality
run: |
pip install cvflow
cvflow inspect ./dataset --strict
- name: Dataset report
run: cvflow inspect ./dataset --html report.html
- uses: actions/upload-artifact@v4
with:
name: dataset-report
path: report.htmlCall it from Python
The pipeline is a library as well as a command, so the same checks run inside a data script and the issues come back as objects.
from cvflow.analysis import AnalysisEngine, CheckConfig, default_checks
from cvflow.loaders import load_dataset
dataset = load_dataset("./dataset")
config = CheckConfig(tiny_box_side=0.005)
issues = AnalysisEngine(default_checks(config)).run(dataset)Everything it looks for
Every check has a name, and every finding names the check that raised it, so a report is something you can grep rather than something you have to interpret.
Integrity
Whether the dataset can be read at all, before anything is measured.
- corrupt-image
- broken-image-path
- invalid-image-dimension
- invalid-annotation-file
- invalid-class-id
- duplicate-filename
- empty-image
- images-not-found
Annotations
Box geometry a training loop will either reject or quietly learn from.
- box-out-of-bounds
- degenerate-box
- tiny-box
- huge-box
- duplicate-annotation
Shapes
Segmentation polygons and oriented boxes, checked on their own terms.
- sparse-polygon
- empty-mask
- rectangular-mask
- sliver-mask
- non-rectangular-obb
- unrotated-obb
Statistics
Distribution oddities: the classes and images sitting far from the rest.
- objects-per-image-outlier
- rare-class
- class-imbalance
Duplicates
Exact copies by SHA-256, near copies by perceptual hash within 5 bits.
- exact-duplicate
- near-duplicate
Leakage
The expensive one: validation images that also appear in training.
- split-leakage
CVFlow FAQ
- How do I check a YOLO dataset for errors?
- Install it with pip install cvflow, then run cvflow inspect ./dataset. It detects the layout from your data.yaml and images/labels folders, reads every image and label, and prints a prioritized report: errors for anything objectively broken, warnings for things worth reviewing, info for observations. Add --serve to read the same findings in a browser dashboard with the images attached.
- Is my dataset uploaded anywhere?
- No. CVFlow is a local Python package. It reads the files on your disk, and the dashboard is a server bound to 127.0.0.1 on your own machine with a vendored copy of Chart.js, so it makes no external requests. There is no account and no API key.
- How do I find duplicate images in a dataset?
- CVFlow does it in two passes. Exact duplicates are found by SHA-256, so byte-identical copies are certain rather than probable. Near-duplicates come from a perceptual hash compared within a 5-bit Hamming distance, which catches re-encodes, crops and consecutive video frames. Both are reported with the file pairs.
- Can it detect train/validation leakage?
- Yes, and it is the check most worth running. split-leakage compares images across your splits visually, so an image appearing in both train and val is reported even when the filenames differ. That kind of overlap inflates validation accuracy, and the inflated number is the one you would otherwise trust.
- Does it work with COCO datasets?
- Yes. A folder with annotations/ holding split JSON files and images/ mirroring those splits is read as COCO, and every check that does not depend on the YOLO text format runs the same way. Use --format coco if the layout is ambiguous enough that detection guesses wrong.
- Can I fix the labels it finds, or only look at them?
- You can fix them. In the dashboard, drag a box to move or resize it, drag on empty canvas to add one, reassign its class or delete it, and the change is written back to the YOLO label file on disk. Editing covers detection boxes; polygons and oriented boxes are reported but not editable.
- Will it tell me if a label is on the wrong object?
- No, and no deterministic checker can. Every check here is validation, statistics or hashing, so it finds labels that are malformed, impossible, duplicated or statistically odd. A well-formed box around the wrong thing looks correct to all of them. What CVFlow does is clear the mechanical problems out of the way, so a human review is spent on the judgement calls.
- Can I run it in CI?
- Yes, that is what the exit codes are for: 0 when clean, 1 when problems are found, 2 for bad usage, 3 for a path that does not exist and 4 when the dataset cannot be loaded. Add --strict to fail on warnings too, and --html report.html to leave a self-contained report your build can upload as an artifact.
- How is this different from a dataset platform's health check?
- It runs where your data is. There is nothing to upload, no project to create and no service to keep paying for, so it fits a dataset that is not allowed to leave your network and it fits a CI job. It is deliberately narrower in return: auditing and label fixes on detection datasets, not annotation, versioning or hosting.
- Is CVFlow free?
- Yes. It is MIT licensed and open source, installable from PyPI with pip install cvflow, with no account and no paid tier.
Limitations
- YOLO and COCO object detection layouts. Segmentation polygons and oriented boxes are audited through their axis-aligned extent rather than in full, and other formats need converting first.
- The label editor covers YOLO detection boxes. Polygons, oriented boxes and COCO annotations are reported but not editable in the dashboard.
- It checks whether a label is well-formed, not whether it is right. A perfectly shaped box drawn around the wrong object is invisible to every check here.
- Near-duplicate detection is a perceptual hash within a 5-bit Hamming distance, so a burst of similar frames from one video can land as findings. That is a threshold to raise, not a bug.
- Class imbalance and rare-class findings are INFO, not errors. Whether a long tail is a problem depends on what you are training for, and the tool does not pretend to know.
- Reading pixels is the slow part. On a large dataset the first full pass takes real time; --no-images gives you the structural verdict in seconds.