Masked Autoencoders with Limited Data: Does It Work? A Fine-Grained Bioacoustics Case Study

TL;DR

This study evaluates the effectiveness of masked autoencoders (MAE) in fine-grained bioacoustic species classification, revealing that pretraining on diverse data generally outperforms domain-specific pretraining in limited-data scenarios.

Contribution

The paper systematically analyzes MAE pretraining in bioacoustics, showing that data scale outweighs domain-specific tuning for moderate-sized datasets.

Findings

Pretraining on diverse general audio data yields better transfer performance.

Additional domain-specific masked reconstruction pretraining offers limited or negative benefits.

Selective data filtering provides negligible gains when data is limited.

Abstract

Bioacoustic recognition requires fine-grained acoustic understanding to distinguish similar-sounding species. However, many large-scale data repositories such as iNaturalist are weakly annotated, often with only a single positive species label per recording, making supervised learning particularly challenging. Inspired by advances in computer vision, recent approaches have shifted toward self-supervised learning to capture the underlying structure of audio without relying on exhaustive annotations. In particular, masked autoencoders (MAE) have shown strong transferability on massive audio corpora, yet their effectiveness in more modest bioacoustic settings remains underexplored. In this work, we conduct a systematic study of MAE pretraining for species classification on iNatSounds, analyzing the impacts of pretraining data scale, domain specificity, data curation, and transfer strategies. Consistent with prior work, we find that models pretrained on diverse general audio data achieve the best transfer performance on iNatSounds. Contrary to observations from large-scale audio benchmarks, we find that (1) additional masked reconstruction pretraining on domain-specific data provides limited benefits and may even degrade performance relative to off-the-shelf models, and (2) selective data filtering offers a negligible advantage when the overall data scale is limited. Our results indicate that, in moderate-sized fine-grained bioacoustic settings, pretraining scale dominates objective design. These findings further clarify when MAE-based pretraining is effective and provide practical guidance for model selection under limited supervision.