Attention-Based Audio Embeddings for Query-by-Example

TL;DR

This paper introduces a robust audio retrieval system using contrastive learning and attention mechanisms, significantly improving accuracy under high distortion while maintaining efficiency and scalability.

Contribution

The novel system employs a CNN with spectral-temporal attention and contrastive learning to generate noise-robust audio fingerprints for improved query matching.

Findings

Outperforms state-of-the-art systems at high distortion levels

Efficient in computation and memory usage

Scalable to larger databases

Abstract

An ideal audio retrieval system efficiently and robustly recognizes a short query snippet from an extensive database. However, the performance of well-known audio fingerprinting systems falls short at high signal distortion levels. This paper presents an audio retrieval system that generates noise and reverberation robust audio fingerprints using the contrastive learning framework. Using these fingerprints, the method performs a comprehensive search to identify the query audio and precisely estimate its timestamp in the reference audio. Our framework involves training a CNN to maximize the similarity between pairs of embeddings extracted from clean audio and its corresponding distorted and time-shifted version. We employ a channel-wise spectral-temporal attention mechanism to better discriminate the audio by giving more weight to the salient spectral-temporal patches in the signal. Experimental results indicate that our system is efficient in computation and memory usage while being more accurate, particularly at higher distortion levels, than competing state-of-the-art systems and scalable to a larger database.