BioME: A Resource-Efficient Bioacoustic Foundational Model for IoT Applications

TL;DR

BioME is a lightweight, resource-efficient bioacoustic encoder trained through distillation and multi-domain pretraining, achieving high performance in biodiversity monitoring tasks suitable for IoT devices.

Contribution

The paper introduces BioME, a novel bioacoustic encoder that reduces model size by 75% using layer-to-layer distillation and incorporates DSP-inspired features for improved ecological generalization.

Findings

BioME matches or exceeds larger models in bioacoustic tasks.

It reduces parameter count by 75% compared to its teacher model.

BioME is suitable for deployment on resource-constrained IoT platforms.

Abstract

Passive acoustic monitoring has become a key strategy in biodiversity assessment, conservation, and behavioral ecology, especially as Internet-of-Things (IoT) devices enable continuous in situ audio collection at scale. While recent self-supervised learning (SSL)-based audio encoders, such as BEATs and AVES, have shown strong performance in bioacoustic tasks, their computational cost and limited robustness to unseen environments hinder deployment on resource-constrained platforms. In this work, we introduce BioME, a resource-efficient audio encoder designed for bioacoustic applications. BioME is trained via layer-to-layer distillation from a high-capacity teacher model, enabling strong representational transfer while reducing the parameter count by 75%. To further improve ecological generalization, the model is pretrained on multi-domain data spanning speech, environmental sounds, and animal vocalizations. A key contribution is the integration of modulation-aware acoustic features via FiLM conditioning, injecting a DSP-inspired inductive bias that enhances feature disentanglement in low-capacity regimes. Across multiple bioacoustic tasks, BioME matches or surpasses the performance of larger models, including its teacher, while being suitable for resource-constrained IoT deployments. For reproducibility, code and pretrained checkpoints are publicly available.