Differentially Private Parameter-Efficient Fine-tuning for Large ASR Models

TL;DR

This paper introduces a differentially private parameter-efficient fine-tuning method for large ASR models, achieving strong privacy guarantees and improved performance with reduced computational costs.

Contribution

It presents a novel DP fine-tuning approach that balances privacy, efficiency, and accuracy for large automatic speech recognition models.

Findings

Achieved 4.6%/8.1% WER on LibriSpeech test sets.

Maintained (10, 3.52e-6)-DP during fine-tuning.

Set new performance benchmarks for privacy-preserving ASR models.

Abstract

Large ASR models can inadvertently leak sensitive information, which can be mitigated by formal privacy measures like differential privacy (DP). However, traditional DP training is computationally expensive, and can hurt model performance. Our study explores DP parameter-efficient fine-tuning as a way to mitigate privacy risks with smaller computation and performance costs for ASR models. Through extensive experimentation and progressive optimization, we achieve 4.6%/8.1% word error rate on LibriSpeech clean/other test-sets, setting a new performance benchmark while maintaining (10, 3.52e-6)-DP in fine-tuning a large ASR model with over 600M parameters.