BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversity

TL;DR

This paper introduces BIOSCAN-5M, a large multimodal insect dataset with images, DNA sequences, and metadata, and establishes benchmark tasks to advance machine learning applications in insect biodiversity monitoring.

Contribution

The paper presents BIOSCAN-5M, a comprehensive multimodal insect dataset with new benchmark tasks for classification, clustering, and cross-modal learning.

Findings

Pretraining on DNA barcodes improves species classification accuracy.

Zero-shot clustering effectively groups insects across modalities.

Contrastive learning creates shared embeddings for multi-modal data.

Abstract

As part of an ongoing worldwide effort to comprehend and monitor insect biodiversity, this paper presents the BIOSCAN-5M Insect dataset to the machine learning community and establish several benchmark tasks. BIOSCAN-5M is a comprehensive dataset containing multi-modal information for over 5 million insect specimens, and it significantly expands existing image-based biological datasets by including taxonomic labels, raw nucleotide barcode sequences, assigned barcode index numbers, geographical, and size information. We propose three benchmark experiments to demonstrate the impact of the multi-modal data types on the classification and clustering accuracy. First, we pretrain a masked language model on the DNA barcode sequences of the BIOSCAN-5M dataset, and demonstrate the impact of using this large reference library on species- and genus-level classification performance. Second, we propose a zero-shot transfer learning task applied to images and DNA barcodes to cluster feature embeddings obtained from self-supervised learning, to investigate whether meaningful clusters can be derived from these representation embeddings. Third, we benchmark multi-modality by performing contrastive learning on DNA barcodes, image data, and taxonomic information. This yields a general shared embedding space enabling taxonomic classification using multiple types of information and modalities. The code repository of the BIOSCAN-5M Insect dataset is available at https://github.com/bioscan-ml/BIOSCAN-5M.