BeetleFlow: An Integrative Deep Learning Pipeline for Beetle Image Processing

TL;DR

BeetleFlow is a comprehensive deep learning pipeline that automates beetle detection, cropping, and segmentation in tray images, significantly aiding large-scale entomological data analysis.

Contribution

The paper introduces a novel multi-stage pipeline combining transformer-based detection and segmentation models specifically tailored for beetle image processing.

Findings

Achieved high accuracy in beetle segmentation with fine-tuned transformer models.

Developed an iterative detection process using open-vocabulary models.

Enhanced efficiency in processing large-scale beetle image datasets.

Abstract

In entomology and ecology research, biologists often need to collect a large number of insects, among which beetles are the most common species. A common practice for biologists to organize beetles is to place them on trays and take a picture of each tray. Given the images of thousands of such trays, it is important to have an automated pipeline to process the large-scale data for further research. Therefore, we develop a 3-stage pipeline to detect all the beetles on each tray, sort and crop the image of each beetle, and do morphological segmentation on the cropped beetles. For detection, we design an iterative process utilizing a transformer-based open-vocabulary object detector and a vision-language model. For segmentation, we manually labeled 670 beetle images and fine-tuned two variants of a transformer-based segmentation model to achieve fine-grained segmentation of beetles with relatively high accuracy. The pipeline integrates multiple deep learning methods and is specialized for beetle image processing, which can greatly improve the efficiency to process large-scale beetle data and accelerate biological research.