animal2vec and MeerKAT: A self-supervised transformer for rare-event raw audio input and a large-scale reference dataset for bioacoustics

TL;DR

This paper introduces animal2vec, a self-supervised transformer model for rare animal vocalizations, and releases MeerKAT, a large annotated meerkat vocalization dataset, advancing bioacoustic analysis with limited labeled data.

Contribution

The work presents a novel interpretable transformer model and a large annotated dataset, improving bioacoustic analysis of scarce and unbalanced animal vocalization data.

Findings

animal2vec outperforms existing methods on MeerKAT and NIPS4Bplus datasets.

The model performs well in few-shot learning scenarios.

MeerKAT is the largest labeled dataset for non-human terrestrial mammals.

Abstract

Bioacoustic research, vital for understanding animal behavior, conservation, and ecology, faces a monumental challenge: analyzing vast datasets where animal vocalizations are rare. While deep learning techniques are becoming standard, adapting them to bioacoustics remains difficult. We address this with animal2vec, an interpretable large transformer model, and a self-supervised training scheme tailored for sparse and unbalanced bioacoustic data. It learns from unlabeled audio and then refines its understanding with labeled data. Furthermore, we introduce and publicly release MeerKAT: Meerkat Kalahari Audio Transcripts, a dataset of meerkat (Suricata suricatta) vocalizations with millisecond-resolution annotations, the largest labeled dataset on non-human terrestrial mammals currently available. Our model outperforms existing methods on MeerKAT and the publicly available NIPS4Bplus birdsong dataset. Moreover, animal2vec performs well even with limited labeled data (few-shot learning). animal2vec and MeerKAT provide a new reference point for bioacoustic research, enabling scientists to analyze large amounts of data even with scarce ground truth information.