MarineDet: Towards Open-Marine Object Detection

TL;DR

MarineDet introduces a novel open-marine object detection framework that leverages joint visual-text semantic space and transfer learning, enabling accurate detection of diverse unseen marine objects in underwater imagery.

Contribution

This paper presents the first open-marine object detection method, MarineDet, including a new marine-specific dataset and a pipeline for detecting unseen marine entities.

Findings

MarineDet outperforms existing object detection algorithms on the MarineDet dataset.

The joint visual-text semantic space improves marine object recognition.

MarineDet establishes a new standard for marine ecosystem monitoring.

Abstract

Marine object detection has gained prominence in marine research, driven by the pressing need to unravel oceanic mysteries and enhance our understanding of invaluable marine ecosystems. There is a profound requirement to efficiently and accurately identify and localize diverse and unseen marine entities within underwater imagery. The open-marine object detection (OMOD for short) is required to detect diverse and unseen marine objects, performing categorization and localization simultaneously. To achieve OMOD, we present \textbf{MarineDet}. We formulate a joint visual-text semantic space through pre-training and then perform marine-specific training to achieve in-air-to-marine knowledge transfer. Considering there is no specific dataset designed for OMOD, we construct a \textbf{MarineDet dataset} consisting of 821 marine-relative object categories to promote and measure OMOD performance. The experimental results demonstrate the superior performance of MarineDet over existing generalist and specialist object detection algorithms. To the best of our knowledge, we are the first to present OMOD, which holds a more valuable and practical setting for marine ecosystem monitoring and management. Our research not only pushes the boundaries of marine understanding but also offers a standard pipeline for OMOD.