AVES: Animal Vocalization Encoder based on Self-Supervision

TL;DR

AVES is a self-supervised transformer model for encoding animal vocalizations, leveraging unannotated data to outperform supervised models in bioacoustics classification and detection tasks.

Contribution

The paper introduces AVES, a novel self-supervised transformer-based model that effectively encodes animal vocalizations using unannotated data, improving bioacoustic task performance.

Findings

AVES outperforms supervised baseline models.

Curating small task-specific datasets enhances model quality.

Open-source availability of AVES models.

Abstract

The lack of annotated training data in bioacoustics hinders the use of large-scale neural network models trained in a supervised way. In order to leverage a large amount of unannotated audio data, we propose AVES (Animal Vocalization Encoder based on Self-Supervision), a self-supervised, transformer-based audio representation model for encoding animal vocalizations. We pretrain AVES on a diverse set of unannotated audio datasets and fine-tune them for downstream bioacoustics tasks. Comprehensive experiments with a suite of classification and detection tasks have shown that AVES outperforms all the strong baselines and even the supervised "topline" models trained on annotated audio classification datasets. The results also suggest that curating a small training subset related to downstream tasks is an efficient way to train high-quality audio representation models. We open-source our models at \url{https://github.com/earthspecies/aves}.