LaSAFT: Latent Source Attentive Frequency Transformation for Conditioned Source Separation

TL;DR

This paper introduces LaSAFT, a novel frequency transformation block, and GPoCM, a modulation technique, to enhance multi-source separation in spectrogram-based models, achieving state-of-the-art results on MUSDB18.

Contribution

The paper proposes LaSAFT and GPoCM, new methods for capturing source-dependent frequency patterns and modulating features, extending Conditioned-U-Net for improved multi-source separation.

Findings

LaSAFT and GPoCM improve separation performance.

Achieved state-of-the-art SDR on MUSDB18.

Enhanced multi-source separation accuracy.

Abstract

Recent deep-learning approaches have shown that Frequency Transformation (FT) blocks can significantly improve spectrogram-based single-source separation models by capturing frequency patterns. The goal of this paper is to extend the FT block to fit the multi-source task. We propose the Latent Source Attentive Frequency Transformation (LaSAFT) block to capture source-dependent frequency patterns. We also propose the Gated Point-wise Convolutional Modulation (GPoCM), an extension of Feature-wise Linear Modulation (FiLM), to modulate internal features. By employing these two novel methods, we extend the Conditioned-U-Net (CUNet) for multi-source separation, and the experimental results indicate that our LaSAFT and GPoCM can improve the CUNet's performance, achieving state-of-the-art SDR performance on several MUSDB18 source separation tasks.