FungiTastic: A multi-modal dataset and benchmark for image categorization

TL;DR

FungiTastic introduces a comprehensive, multimodal fungal dataset and benchmark supporting various classification tasks, including DNA-verified labels, to advance research in fungal identification and ecology.

Contribution

The paper presents a new large-scale, multimodal fungal dataset with expert labels and DNA ground truth, along with a benchmark supporting diverse classification tasks.

Findings

Dataset contains 350,000 observations of 6,000 species.

Includes multimodal data: images, meteorological, satellite, and segmentation masks.

Provides baseline models and training framework for various tasks.

Abstract

We introduce a new, challenging benchmark and a dataset, FungiTastic, based on fungal records continuously collected over a twenty-year span. The dataset is labelled and curated by experts and consists of about 350k multimodal observations of 6k fine-grained categories (species). The fungi observations include photographs and additional data, e.g., meteorological and climatic data, satellite images, and body part segmentation masks. FungiTastic is one of the few benchmarks that include a test set with DNA-sequenced ground truth of unprecedented label reliability. The benchmark is designed to support (i) standard closed-set classification, (ii) open-set classification, (iii) multi-modal classification, (iv) few-shot learning, (v) domain shift, and many more. We provide tailored baselines for many use cases, a multitude of ready-to-use pre-trained models on https://huggingface.co/collections/BVRA/fungitastic-66a227ce0520be533dc6403b, and a framework for model training. The documentation and the baselines are available at https://github.com/BohemianVRA/FungiTastic/ and https://www.kaggle.com/datasets/picekl/fungitastic.