BEANS: The Benchmark of Animal Sounds

TL;DR

BEANS is a comprehensive benchmark suite for bioacoustic machine learning tasks, providing datasets and baseline results across multiple species to standardize performance evaluation in the field.

Contribution

This paper introduces BEANS, the first public benchmark collection for bioacoustic ML tasks, covering multiple species and including baseline performance metrics.

Findings

12 diverse bioacoustic datasets included

Baseline ML methods evaluated and reported

Benchmark and code publicly available

Abstract

The use of machine learning (ML) based techniques has become increasingly popular in the field of bioacoustics over the last years. Fundamental requirements for the successful application of ML based techniques are curated, agreed upon, high-quality datasets and benchmark tasks to be learned on a given dataset. However, the field of bioacoustics so far lacks such public benchmarks which cover multiple tasks and species to measure the performance of ML techniques in a controlled and standardized way and that allows for benchmarking newly proposed techniques to existing ones. Here, we propose BEANS (the BEnchmark of ANimal Sounds), a collection of bioacoustics tasks and public datasets, specifically designed to measure the performance of machine learning algorithms in the field of bioacoustics. The benchmark proposed here consists of two common tasks in bioacoustics: classification and detection. It includes 12 datasets covering various species, including birds, land and marine mammals, anurans, and insects. In addition to the datasets, we also present the performance of a set of standard ML methods as the baseline for task performance. The benchmark and baseline code is made publicly available at \url{https://github.com/earthspecies/beans} in the hope of establishing a new standard dataset for ML-based bioacoustic research.