# End-to-End Segmentation and Classification of Zooplankton Using Shadowgraphy and Convolutional Neural Networks

## TL;DR

This paper presents a high-accuracy CNN-based system for classifying zooplankton in situ images, enabling efficient ecological studies across diverse ecosystems.

## Contribution

A novel niched CNN framework for modular and scalable zooplankton classification, achieving high accuracy across varied environments.

## Key findings

- Four CNN architectures achieved over 95% accuracy in classifying zooplankton from 70k images.
- The system maintains high accuracy with out-of-distribution data, showing adaptability to diverse ecosystems.
- A secondary classification level achieved 86–90% accuracy for finer taxonomic groupings.

## Abstract

What are the main findings?
Four CNN-based classification algorithms were successfully implemented for categorizing zooplankton with >95% accuracy using over 70k images.Niched networks are a tractable, modular and efficient avenue for specific taxonomic or functional classification.

Four CNN-based classification algorithms were successfully implemented for categorizing zooplankton with >95% accuracy using over 70k images.

Niched networks are a tractable, modular and efficient avenue for specific taxonomic or functional classification.

What are the implications of the main findings?
This segmentation and classification algorithm can potentially be used to study zooplankton interactions and various functional traits.Its high accuracy when given out-of-distribution data shows that the algorithm can be used in diverse ecosystems.

This segmentation and classification algorithm can potentially be used to study zooplankton interactions and various functional traits.

Its high accuracy when given out-of-distribution data shows that the algorithm can be used in diverse ecosystems.

With in situ imaging systems becoming more common, precise, and economically viable, use of these systems has grown dramatically, including both automated classification and biomass estimations. However, a rather large and overlooked portion of these efforts is reliable detection and classification of these organisms as they pass through the imaging device. This paper focuses on the development of an end-to-end classification CNN-based algorithm for marine zooplankton using the in situ Ichthyoplankton Imaging System (ISIIS-DPI) from Bellamare (La Jolla, CA, USA). Our novel approach considers many issues with automated segmentation and classification, including over-segmentation, noise segmentation, and organism size input. This allows for classifications in diverse water types, demonstrated by the comparison of three datasets created in conjunction with this project, each with very different water properties and zooplankton communities (Florida Gulf coast; Trondheimsfjord, Norway; Sargasso Sea). Our segmented image dataset contains 70,624 regions of interest (ROIs) across four organism classes—Chaetognath, Crustacean, Gelatinous, and Larvacean—with two classes dedicated to detritus. Four common network architectures—Resnet, Xception, GoogleNet, and Darknet—are trained on this dataset, with final test accuracies in the range of 95.94–96.09%. Following this initial training, a secondary level of classification is introduced. The base Gelatinous class is further divided into six groups. The same four CNN architectures are used once again, with final accuracies in the range of 86.12–90.40%, showing the ability to taxonomically classify down to the order level. The present work introduces a versatile, adaptable, scalable and autonomous segmentation and classification algorithm using niched networks mirroring taxonomy, and is fully contained in a publicly available MATLAB R2025a custom graphical user interface.