The Caltech Fish Counting Dataset: A Benchmark for Multiple-Object Tracking and Counting

TL;DR

The Caltech Fish Counting Dataset (CFC) provides a large-scale, natural-world sonar video dataset for advancing multiple-object tracking and counting, emphasizing domain generalization challenges.

Contribution

We introduce the CFC dataset, a novel large-scale sonar video dataset for fish detection, tracking, and counting, highlighting its potential for improving generalization in MOT and counting tasks.

Findings

CFC contains over 1,500 videos with 500,000 annotations.

Baseline experiments reveal significant challenges in generalization across unseen locations.

The dataset enables research on low signal-to-noise computer vision in natural aquatic environments.

Abstract

We present the Caltech Fish Counting Dataset (CFC), a large-scale dataset for detecting, tracking, and counting fish in sonar videos. We identify sonar videos as a rich source of data for advancing low signal-to-noise computer vision applications and tackling domain generalization in multiple-object tracking (MOT) and counting. In comparison to existing MOT and counting datasets, which are largely restricted to videos of people and vehicles in cities, CFC is sourced from a natural-world domain where targets are not easily resolvable and appearance features cannot be easily leveraged for target re-identification. With over half a million annotations in over 1,500 videos sourced from seven different sonar cameras, CFC allows researchers to train MOT and counting algorithms and evaluate generalization performance at unseen test locations. We perform extensive baseline experiments and identify key challenges and opportunities for advancing the state of the art in generalization in MOT and counting.