Audio Retrieval with Natural Language Queries

TL;DR

This paper introduces new benchmarks for text-based audio retrieval using natural language queries, demonstrating the benefits of pre-training and aiming to inspire further research in cross-modal retrieval tasks.

Contribution

It presents challenging new benchmarks for text-based audio retrieval and establishes baseline results, highlighting the advantages of pre-training on diverse audio tasks.

Findings

Pre-training improves retrieval performance.

New benchmarks challenge existing methods.

Baseline results set a standard for future research.

Abstract

We consider the task of retrieving audio using free-form natural language queries. To study this problem, which has received limited attention in the existing literature, we introduce challenging new benchmarks for text-based audio retrieval using text annotations sourced from the Audiocaps and Clotho datasets. We then employ these benchmarks to establish baselines for cross-modal audio retrieval, where we demonstrate the benefits of pre-training on diverse audio tasks. We hope that our benchmarks will inspire further research into cross-modal text-based audio retrieval with free-form text queries.