Leveraging tropical reef, bird and unrelated sounds for superior transfer learning in marine bioacoustics

TL;DR

This paper demonstrates that combining bird, reef, and unrelated sounds during pretraining significantly improves transfer learning for marine bioacoustics, enabling effective ecological monitoring with minimal annotation.

Contribution

It introduces a novel pretraining strategy that leverages cross-domain audio mixing to enhance generalization in reef bioacoustics classification tasks.

Findings

Pretraining on bird sounds outperforms reef-only pretraining.

Cross-domain mixing maximizes reef sound classification accuracy.

SurfPerch network enables efficient marine sound analysis with minimal data.

Abstract

Machine learning has the potential to revolutionize passive acoustic monitoring (PAM) for ecological assessments. However, high annotation and compute costs limit the field's efficacy. Generalizable pretrained networks can overcome these costs, but high-quality pretraining requires vast annotated libraries, limiting its current applicability primarily to bird taxa. Here, we identify the optimum pretraining strategy for a data-deficient domain using coral reef bioacoustics. We assemble ReefSet, a large annotated library of reef sounds, though modest compared to bird libraries at 2% of the sample count. Through testing few-shot transfer learning performance, we observe that pretraining on bird audio provides notably superior generalizability compared to pretraining on ReefSet or unrelated audio alone. However, our key findings show that cross-domain mixing which leverages bird, reef and unrelated audio during pretraining maximizes reef generalizability. SurfPerch, our pretrained network, provides a strong foundation for automated analysis of marine PAM data with minimal annotation and compute costs.