MUOT_3M: A 3 Million Frame Multimodal Underwater Benchmark and the MUTrack Tracking Method

TL;DR

This paper introduces MUOT_3M, a large-scale multimodal underwater tracking dataset, and MUTrack, a novel multimodal tracking method that significantly improves robustness and accuracy in underwater object tracking tasks.

Contribution

The paper presents the first large-scale multimodal underwater tracking dataset and a SAM-based multimodal tracker that effectively transfers knowledge to unimodal models.

Findings

MUTrack outperforms state-of-the-art methods with up to 8.40% higher AUC.

MUTrack achieves 7.80% higher precision than existing baselines.

The dataset includes 3 million frames with diverse modalities and annotations.

Abstract

Underwater Object Tracking (UOT) is crucial for efficient marine robotics, large scale ecological monitoring, and ocean exploration; however, progress has been hindered by the scarcity of large, multimodal, and diverse datasets. Existing benchmarks remain small and RGB only, limiting robustness under severe color distortion, turbidity, and low visibility conditions. We introduce MUOT_3M, the first pseudo multimodal UOT benchmark comprising 3 million frames from 3,030 videos (27.8h) annotated with 32 tracking attributes, 677 fine grained classes, and synchronized RGB, estimated enhanced RGB, estimated depth, and language modalities validated by a marine biologist. Building upon MUOT_3M, we propose MUTrack, a SAM-based multimodal to unimodal tracker featuring visual geometric alignment, vision language fusion, and four level knowledge distillation that transfers multimodal knowledge into a unimodal student model. Extensive evaluations across five UOT benchmarks demonstrate that MUTrack achieves up to 8.40% higher AUC and 7.80% higher precision than the strongest SOTA baselines while running at 24 FPS. MUOT_3M and MUTrack establish a new foundation for scalable, multimodally trained yet practically deployable underwater tracking.