Improving Domain Generalization for Sound Classification with Sparse Frequency-Regularized Transformer

TL;DR

This paper introduces FRITO, a regularization method for Transformers that enhances sound classification generalization by restricting attention along the frequency dimension, achieving state-of-the-art results with reduced inference time.

Contribution

The paper proposes a novel frequency-regularized attention mechanism for Transformers to improve out-of-distribution sound classification performance.

Findings

Achieves state-of-the-art generalization on TAU 2020 and Nsynth datasets.

Reduces inference time by 20%.

Improves model robustness to out-of-distribution data.

Abstract

Sound classification models' performance suffers from generalizing on out-of-distribution (OOD) data. Numerous methods have been proposed to help the model generalize. However, most either introduce inference overheads or focus on long-lasting CNN-variants, while Transformers has been proven to outperform CNNs on numerous natural language processing and computer vision tasks. We propose FRITO, an effective regularization technique on Transformer's self-attention, to improve the model's generalization ability by limiting each sequence position's attention receptive field along the frequency dimension on the spectrogram. Experiments show that our method helps Transformer models achieve SOTA generalization performance on TAU 2020 and Nsynth datasets while saving 20% inference time.