# Automated underwater plectropomus leopardus phenotype measurement through cylinder

## TL;DR

This paper introduces a non-invasive, image-based method to measure the body length of leopard coral trout underwater using corrected keypoint detection and stereo imaging.

## Contribution

A novel underwater phenotyping system using refraction-corrected keypoint detection and stereo matching for accurate, non-contact fish length measurement.

## Key findings

- The model achieved 98.6% accuracy in keypoint detection (AP@0.5:0.95).
- Underwater tests showed an average measurement error of 3.2 mm (MRPE=3.50%) with real-time processing at 28 FPS.
- The method reduces MRPE by 53.8% compared to baseline methods.

## Abstract

Accurate and non-invasive measurement of fish phenotypic characteristics in underwater environments is crucial for advancing aquaculture. Traditional manual methods require significant labor to anesthetize and capture fish, which not only raises ethical concerns but also risks causing injury to the animals. Alternative hardware-based approaches, such as acoustic technology and active structured light techniques, are often costly and may suffer from limited measurement accuracy. In contrast, image-based methods utilizing low-cost binocular cameras present a more affordable solution, although they face challenges such as light refraction between water and the waterproof enclosure, which can cause discrepancies between image coordinates and actual positions. To address these challenges, we have developed a fish keypoint detection dataset and trained both a fish object detection model and a keypoint detection model using the RTMDet and RTMPose architectures to identify keypoints on Plectropomus leopardus. Since the binocular camera must be housed in a waterproof enclosure, we correct for birefringence caused by the water and the enclosure by applying refraction corrections to the detected keypoint coordinates. This ensures that the keypoint coordinates obtained underwater are consistent with those in air, thereby improving the accuracy of subsequent stereo matching. Once the corrected keypoint coordinates are obtained, we apply the least squares method, in conjunction with binocular stereo imaging principles, to perform stereo matching and derive the actual 3D coordinates of the keypoints. We calculate the fish body length by measuring the 3D coordinates of the snout and tail. Our model achieved 98.6% accuracy in keypoints detection (AP@0.5:0.95). Underwater tests showed an average measurement error of approximately 3.2 mm (MRPE=3.50%) for fish in a tank, with real-time processing at 28 FPS on an NVIDIA GTX 1060 GPU. These results confirm that our method effectively detects keypoints on fish bodies and measures their length without physical contact or removal from the tank. By eliminating invasive procedures, our approach not only improves measurement efficiency but also aligns with ethical standards in aquaculture. Compared to existing techniques, our method offers enhanced accuracy (reducing MRPE by 53.8% compared to baseline methods) and practicality, making it a valuable tool for the aquaculture industry.