ShellfishNet: A Domain-Specific Benchmark for Visual Recognition of Marine Molluscs

TL;DR

ShellfishNet is a new comprehensive dataset and benchmark for evaluating AI models on real-world underwater images of marine molluscs, addressing ecological monitoring challenges.

Contribution

The paper introduces ShellfishNet, a large annotated dataset and evaluation framework tailored for marine benthic species recognition under complex underwater conditions.

Findings

Evaluated 80 neural network models on ShellfishNet dataset.

Assessed model robustness under simulated underwater image degradations.

Analyzed performance of fine-grained categorization and multimodal models.

Abstract

The decline of global shellfish biodiversity poses a severe threat to coastal ecosystems. Although artificial intelligence (AI) technologies show potential for automated ecological monitoring, existing marine benthic datasets often lack adaptation to the complexities of real underwater environments (e.g., variable lighting conditions and diverse species postures), posing challenges for the robust generalization of vision models in practical ecological monitoring. To address this problem, we construct ShellfishNet, a comprehensive image benchmark dataset designed specifically for real-world ecological monitoring constraints. Comprising 8,691 images across 32 taxa, this dataset includes a curated subset annotated with descriptive captions. It is constructed through field photography and web scraping, encompassing samples from complex real-world environments. Based on this benchmark, we systematically evaluate 80 representative neural network models, including Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), State Space Models (SSMs), and Self-Supervised Learning (SSL) methods. Furthermore, we evaluate the performance of fine-grained visual categorization (FGVC) models and investigate the image captioning capabilities of several mainstream multimodal large language models (MLLMs). Meanwhile, we introduce image corruption benchmark tests to simulate common underwater degradation scenarios (turbidity, severe weather) and assess the robustness of vision models, enabling trustworthy decisions on ecological protection in the wild. ShellfishNet is dedicated to providing a data foundation and a model-evaluation benchmark for the intelligent monitoring of benthic organisms.