AudioGAN: A Compact and Efficient Framework for Real-Time High-Fidelity Text-to-Audio Generation

TL;DR

AudioGAN is a novel GAN-based framework for real-time high-fidelity text-to-audio generation, achieving faster inference and fewer parameters than existing models, making it practical for media applications.

Contribution

This paper introduces AudioGAN, the first GAN-based TTA model that generates audio in a single pass with innovative attention mechanisms, reducing complexity and inference time.

Findings

Achieves state-of-the-art performance on AudioCaps dataset.

Uses 90% fewer parameters than previous models.

Runs 20 times faster, synthesizing audio in under one second.

Abstract

Text-to-audio (TTA) generation can significantly benefit the media industry by reducing production costs and enhancing work efficiency. However, most current TTA models (primarily diffusion-based) suffer from slow inference speeds and high computational costs. In this paper, we introduce AudioGAN, the first successful Generative Adversarial Networks (GANs)-based TTA framework that generates audio in a single pass, thereby reducing model complexity and inference time. To overcome the inherent difficulties in training GANs, we integrate multiple ,contrastive losses and propose innovative components Single-Double-Triple (SDT) Attention and Time-Frequency Cross-Attention (TF-CA). Extensive experiments on the AudioCaps dataset demonstrate that AudioGAN achieves state-of-the-art performance while using 90% fewer parameters and running 20 times faster, synthesizing audio in under one second. These results establish AudioGAN as a practical and powerful solution for real-time TTA.