SIMD-size aware weight regularization for fast neural vocoding on CPU

TL;DR

This paper introduces SIMD-size aware regularization to improve the speed of neural vocoders on CPU by maintaining weight matrix order, enabling effective pruning without quality loss.

Contribution

It proposes a novel SIMD size aware regularization method that aligns weight matrices for fast CPU vocoding while preserving speech quality.

Findings

Achieves comparable naturalness with pruned models

Enables faster vocoding compared to conventional methods

Maintains quality despite 70% sparsity

Abstract

This paper proposes weight regularization for a faster neural vocoder. Pruning time-consuming DNN modules is a promising way to realize a real-time vocoder on a CPU (e.g. WaveRNN, LPCNet). Regularization that encourages sparsity is also effective in avoiding the quality degradation created by pruning. However, the orders of weight matrices must be contiguous in SIMD size for fast vocoding. To ensure this order, we propose explicit SIMD size aware regularization. Our proposed method reshapes a weight matrix into a tensor so that the weights are aligned by group size in advance, and then computes the group Lasso-like regularization loss. Experiments on 70% sparse subband WaveRNN show that pruning in conventional Lasso and column-wise group Lasso degrades the synthetic speech's naturalness. The vocoder with proposed regularization 1) achieves comparable naturalness to that without pruning and 2) performs meaningfully faster than other conventional vocoders using regularization.