The FathomNet2023 Competition Dataset

TL;DR

The FathomNet2023 dataset and competition challenge the development of models that can accurately identify marine organisms and detect out-of-sample images amidst high variability and distribution shifts in ocean visual data.

Contribution

This work introduces a realistic, challenging dataset and competition focused on marine image classification and out-of-sample detection in ocean science.

Findings

Dataset captures extreme variability in marine imagery

Models need to identify known and novel organisms

Out-of-sample detection is crucial for reliable analysis

Abstract

Ocean scientists have been collecting visual data to study marine organisms for decades. These images and videos are extremely valuable both for basic science and environmental monitoring tasks. There are tools for automatically processing these data, but none that are capable of handling the extreme variability in sample populations, image quality, and habitat characteristics that are common in visual sampling of the ocean. Such distribution shifts can occur over very short physical distances and in narrow time windows. Creating models that are able to recognize when an image or video sequence contains a new organism, an unusual collection of animals, or is otherwise out-of-sample is critical to fully leverage visual data in the ocean. The FathomNet2023 competition dataset presents a realistic scenario where the set of animals in the target data differs from the training data. The challenge is both to identify the organisms in a target image and assess whether it is out-of-sample.