UWBench: A Comprehensive Vision-Language Benchmark for Underwater Understanding

TL;DR

UWBench is a new comprehensive benchmark dataset designed to evaluate and advance vision-language models specifically for underwater environments, addressing unique challenges like light attenuation and marine ecology understanding.

Contribution

The paper introduces UWBench, a large-scale, annotated underwater dataset and benchmarks for image captioning, visual grounding, and question answering in marine settings, filling a critical gap in underwater AI research.

Findings

State-of-the-art models perform poorly on underwater tasks

Underwater understanding remains a significant challenge for current VLMs

UWBench enables targeted improvements in marine environment AI applications

Abstract

Large vision-language models (VLMs) have achieved remarkable success in natural scene understanding, yet their application to underwater environments remains largely unexplored. Underwater imagery presents unique challenges including severe light attenuation, color distortion, and suspended particle scattering, while requiring specialized knowledge of marine ecosystems and organism taxonomy. To bridge this gap, we introduce UWBench, a comprehensive benchmark specifically designed for underwater vision-language understanding. UWBench comprises 15,003 high-resolution underwater images captured across diverse aquatic environments, encompassing oceans, coral reefs, and deep-sea habitats. Each image is enriched with human-verified annotations including 15,281 object referring expressions that precisely describe marine organisms and underwater structures, and 124,983 question-answer pairs covering diverse reasoning capabilities from object recognition to ecological relationship understanding. The dataset captures rich variations in visibility, lighting conditions, and water turbidity, providing a realistic testbed for model evaluation. Based on UWBench, we establish three comprehensive benchmarks: detailed image captioning for generating ecologically informed scene descriptions, visual grounding for precise localization of marine organisms, and visual question answering for multimodal reasoning about underwater environments. Extensive experiments on state-of-the-art VLMs demonstrate that underwater understanding remains challenging, with substantial room for improvement. Our benchmark provides essential resources for advancing vision-language research in underwater contexts and supporting applications in marine science, ecological monitoring, and autonomous underwater exploration. Our code and benchmark will be available.