PicoAudio: Enabling Precise Timestamp and Frequency Controllability of Audio Events in Text-to-audio Generation

TL;DR

PicoAudio is a novel framework that enables precise timestamp and frequency control in text-to-audio generation by integrating temporal information and fine-grained data processing, significantly outperforming existing models.

Contribution

The paper introduces PicoAudio, a new model that incorporates temporal information for controllable audio generation, addressing a key challenge in the field.

Findings

PicoAudio achieves superior controllability in timestamp and frequency.

Subjective and objective evaluations show significant improvements.

Generated samples demonstrate practical effectiveness.

Abstract

Recently, audio generation tasks have attracted considerable research interests. Precise temporal controllability is essential to integrate audio generation with real applications. In this work, we propose a temporal controlled audio generation framework, PicoAudio. PicoAudio integrates temporal information to guide audio generation through tailored model design. It leverages data crawling, segmentation, filtering, and simulation of fine-grained temporally-aligned audio-text data. Both subjective and objective evaluations demonstrate that PicoAudio dramantically surpasses current state-of-the-art generation models in terms of timestamp and occurrence frequency controllability. The generated samples are available on the demo website https://zeyuxie29.github.io/PicoAudio.github.io.