YH-MINER: Multimodal Intelligent System for Natural Ecological Reef Metric Extraction

TL;DR

The paper introduces YH-MINER, an intelligent multimodal system that enhances coral reef ecological monitoring by combining object detection, segmentation, and classification to improve accuracy and automation in underwater environments.

Contribution

It develops a novel multimodal framework utilizing large models for efficient coral detection, segmentation, and classification, enabling real-time ecological metric extraction.

Findings

Object detection mAP@0.5=0.78

Genus-level classification accuracy of 88%

Supports automation in underwater ecological monitoring

Abstract

Coral reefs, crucial for sustaining marine biodiversity and ecological processes (e.g., nutrient cycling, habitat provision), face escalating threats, underscoring the need for efficient monitoring. Coral reef ecological monitoring faces dual challenges of low efficiency in manual analysis and insufficient segmentation accuracy in complex underwater scenarios. This study develops the YH-MINER system, establishing an intelligent framework centered on the Multimodal Large Model (MLLM) for "object detection-semantic segmentation-prior input". The system uses the object detection module (mAP@0.5=0.78) to generate spatial prior boxes for coral instances, driving the segment module to complete pixel-level segmentation in low-light and densely occluded scenarios. The segmentation masks and finetuned classification instructions are fed into the Qwen2-VL-based multimodal model as prior inputs, achieving a genus-level classification accuracy of 88% and simultaneously extracting core ecological metrics. Meanwhile, the system retains the scalability of the multimodal model through standardized interfaces, laying a foundation for future integration into multimodal agent-based underwater robots and supporting the full-process automation of "image acquisition-prior generation-real-time analysis".