AQUA20: A Benchmark Dataset for Underwater Species Classification under Challenging Conditions

TL;DR

AQUA20 is a new benchmark dataset with 8,171 underwater images of 20 marine species, designed to evaluate and improve visual recognition models under challenging underwater conditions.

Contribution

The paper introduces AQUA20, a comprehensive dataset for underwater species classification, and benchmarks several deep learning models, highlighting their performance and interpretability in difficult environments.

Findings

ConvNeXt achieved 98.82% Top-3 accuracy and 90.69% Top-1 accuracy.

Benchmark models show trade-offs between complexity and performance.

Explainability analysis reveals strengths and pitfalls of models.

Abstract

Robust visual recognition in underwater environments remains a significant challenge due to complex distortions such as turbidity, low illumination, and occlusion, which severely degrade the performance of standard vision systems. This paper introduces AQUA20, a comprehensive benchmark dataset comprising 8,171 underwater images across 20 marine species reflecting real-world environmental challenges such as illumination, turbidity, occlusions, etc., providing a valuable resource for underwater visual understanding. Thirteen state-of-the-art deep learning models, including lightweight CNNs (SqueezeNet, MobileNetV2) and transformer-based architectures (ViT, ConvNeXt), were evaluated to benchmark their performance in classifying marine species under challenging conditions. Our experimental results show ConvNeXt achieving the best performance, with a Top-3 accuracy of 98.82% and a Top-1 accuracy of 90.69%, as well as the highest overall F1-score of 88.92% with moderately large parameter size. The results obtained from our other benchmark models also demonstrate trade-offs between complexity and performance. We also provide an extensive explainability analysis using GRAD-CAM and LIME for interpreting the strengths and pitfalls of the models. Our results reveal substantial room for improvement in underwater species recognition and demonstrate the value of AQUA20 as a foundation for future research in this domain. The dataset is publicly available at: https://huggingface.co/datasets/taufiktrf/AQUA20.