Evaluating the method reproducibility of deep learning models in the biodiversity domain

TL;DR

This study assesses the reproducibility of deep learning methods in biodiversity research by analyzing 61 publications, highlighting gaps in resource sharing and methodological transparency that affect reproducibility.

Contribution

It introduces a methodology for evaluating reproducibility in biodiversity-related deep learning studies and applies it to a curated dataset of publications.

Findings

47% of publications shared their datasets

Many publications lack detailed methodological information

Reproducibility is hindered by incomplete resource and randomness data

Abstract

Artificial Intelligence (AI) is revolutionizing biodiversity research by enabling advanced data analysis, species identification, and habitats monitoring, thereby enhancing conservation efforts. Ensuring reproducibility in AI-driven biodiversity research is crucial for fostering transparency, verifying results, and promoting the credibility of ecological findings.This study investigates the reproducibility of deep learning (DL) methods within the biodiversity domain. We design a methodology for evaluating the reproducibility of biodiversity-related publications that employ DL techniques across three stages. We define ten variables essential for method reproducibility, divided into four categories: resource requirements, methodological information, uncontrolled randomness, and statistical considerations. These categories subsequently serve as the basis for defining different levels of reproducibility. We manually extract the availability of these variables from a curated dataset comprising 61 publications identified using the keywords provided by biodiversity experts. Our study shows that the dataset is shared in 47% of the publications; however, a significant number of the publications lack comprehensive information on deep learning methods, including details regarding randomness.