FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio Generation

TL;DR

FlashAudio introduces rectified flows with optimized time distribution and anchored guidance to enable fast, high-fidelity text-to-audio generation, surpassing traditional diffusion models in quality and speed.

Contribution

The paper proposes FlashAudio with rectified flows, Bifocal Samplers, immiscible flow, and Anchored Optimization to significantly improve one-step text-to-audio generation performance.

Findings

Outperforms diffusion models with hundreds of steps in audio quality.

Achieves 400x faster-than-real-time sampling on a single GPU.

Surpasses previous one-step methods in quality due to rectified flows and optimization techniques.

Abstract

Recent advancements in latent diffusion models (LDMs) have markedly enhanced text-to-audio generation, yet their iterative sampling processes impose substantial computational demands, limiting practical deployment. While recent methods utilizing consistency-based distillation aim to achieve few-step or single-step inference, their one-step performance is constrained by curved trajectories, preventing them from surpassing traditional diffusion models. In this work, we introduce FlashAudio with rectified flows to learn straight flow for fast simulation. To alleviate the inefficient timesteps allocation and suboptimal distribution of noise, FlashAudio optimizes the time distribution of rectified flow with Bifocal Samplers and proposes immiscible flow to minimize the total distance of data-noise pairs in a batch vias assignment. Furthermore, to address the amplified accumulation error caused by the classifier-free guidance (CFG), we propose Anchored Optimization, which refines the guidance scale by anchoring it to a reference trajectory. Experimental results on text-to-audio generation demonstrate that FlashAudio's one-step generation performance surpasses the diffusion-based models with hundreds of sampling steps on audio quality and enables a sampling speed of 400x faster than real-time on a single NVIDIA 4090Ti GPU. Code will be available at https://github.com/liuhuadai/FlashAudio.