Automated Marine Biofouling Assessment: Benchmarking Computer Vision and Multimodal LLMs on the Level of Fouling Scale

TL;DR

This paper evaluates computer vision and multimodal language models for automated marine biofouling severity classification, highlighting their strengths and limitations, and proposing hybrid approaches for scalable biofouling assessment.

Contribution

It introduces a benchmark dataset and compares traditional computer vision models with large multimodal language models for biofouling classification.

Findings

Computer vision models excel at extreme fouling levels but struggle with intermediate categories.

LLMs achieve competitive performance without training through structured prompts.

Hybrid methods combining segmentation and LLM reasoning show promise for scalable assessment.

Abstract

Marine biofouling on vessel hulls poses major ecological, economic, and biosecurity risks. Traditional survey methods rely on diver inspections, which are hazardous and limited in scalability. This work investigates automated classification of biofouling severity on the Level of Fouling (LoF) scale using both custom computer vision models and large multimodal language models (LLMs). Convolutional neural networks, transformer-based segmentation, and zero-shot LLMs were evaluated on an expert-labelled dataset from the New Zealand Ministry for Primary Industries. Computer vision models showed high accuracy at extreme LoF categories but struggled with intermediate levels due to dataset imbalance and image framing. LLMs, guided by structured prompts and retrieval, achieved competitive performance without training and provided interpretable outputs. The results demonstrate complementary strengths across approaches and suggest that hybrid methods integrating segmentation coverage with LLM reasoning offer a promising pathway toward scalable and interpretable biofouling assessment.