Audio-to-Image Bird Species Retrieval without Audio-Image Pairs via Text Distillation

TL;DR

This paper introduces a data-efficient method for audio-to-image bird species retrieval that leverages text as an intermediary, enabling alignment without paired audio-image data and outperforming baselines on multiple benchmarks.

Contribution

The authors propose a novel text distillation approach that transfers visual semantics into audio representations, facilitating effective audio-to-image retrieval without requiring paired data.

Findings

Achieves strong audio-to-image retrieval performance on bioacoustic benchmarks.

Improves audio-text alignment while maintaining audio discriminative power.

Outperforms baselines based on zero-shot models and learned mappings.

Abstract

Audio-to-image retrieval offers an interpretable alternative to audio-only classification for bioacoustic species recognition, but learning aligned audio-image representations is challenging due to the scarcity of paired audio-image data. We propose a simple and data-efficient approach that enables audio-to-image retrieval without any audio-image supervision. Our proposed method uses text as a semantic intermediary: we distill the text embedding space of a pretrained image-text model (BioCLIP-2), which encodes rich visual and taxonomic structure, into a pretrained audio-text model (BioLingual) by fine-tuning its audio encoder with a contrastive objective. This distillation transfers visually grounded semantics into the audio representation, inducing emergent alignment between audio and image embeddings without using images during training. We evaluate the resulting model on multiple bioacoustic benchmarks. The distilled audio encoder preserves audio discriminative power while substantially improving audio-text alignment on focal recordings and soundscape datasets. Most importantly, on the SSW60 benchmark, the proposed approach achieves strong audio-to-image retrieval performance exceeding baselines based on zero-shot model combinations or learned mappings between text embeddings, despite not training on paired audio-image data. These results demonstrate that indirect semantic transfer through text is sufficient to induce meaningful audio-image alignment, providing a practical solution for visually grounded species recognition in data-scarce bioacoustic settings.