SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Network

TL;DR

SpikeVoice demonstrates high-quality text-to-speech synthesis using spiking neural networks, effectively capturing long-term dependencies with novel attention mechanisms, achieving comparable results to traditional neural networks while significantly reducing energy consumption.

Contribution

This paper introduces SpikeVoice, the first TTS system based on SNNs, and proposes Spiking Temporal-Sequential Attention to address long-term dependency challenges.

Findings

Achieves TTS quality comparable to ANN-based systems.

Uses only 10.5% of the energy of traditional neural networks.

Effective across multiple languages and speaker scenarios.

Abstract

Brain-inspired Spiking Neural Network (SNN) has demonstrated its effectiveness and efficiency in vision, natural language, and speech understanding tasks, indicating their capacity to "see", "listen", and "read". In this paper, we design \textbf{SpikeVoice}, which performs high-quality Text-To-Speech (TTS) via SNN, to explore the potential of SNN to "speak". A major obstacle to using SNN for such generative tasks lies in the demand for models to grasp long-term dependencies. The serial nature of spiking neurons, however, leads to the invisibility of information at future spiking time steps, limiting SNN models to capture sequence dependencies solely within the same time step. We term this phenomenon "partial-time dependency". To address this issue, we introduce Spiking Temporal-Sequential Attention STSA in the SpikeVoice. To the best of our knowledge, SpikeVoice is the first TTS work in the SNN field. We perform experiments using four well-established datasets that cover both Chinese and English languages, encompassing scenarios with both single-speaker and multi-speaker configurations. The results demonstrate that SpikeVoice can achieve results comparable to Artificial Neural Networks (ANN) with only 10.5 energy consumption of ANN.