Optimal Condition Training for Target Source Separation

TL;DR

This paper introduces an optimal condition training method for single-channel target source separation that leverages multiple semantic concepts and condition refinement to improve source disentanglement and achieve state-of-the-art results.

Contribution

The paper proposes a novel optimal condition training approach with condition refinement for improved source separation, outperforming existing models and handling diverse semantic concepts.

Findings

Significantly improves source separation efficiency.

Outperforms permutation invariant models with oracle assignment.

Achieves state-of-the-art in text-based source separation.

Abstract

Recent research has shown remarkable performance in leveraging multiple extraneous conditional and non-mutually exclusive semantic concepts for sound source separation, allowing the flexibility to extract a given target source based on multiple different queries. In this work, we propose a new optimal condition training (OCT) method for single-channel target source separation, based on greedy parameter updates using the highest performing condition among equivalent conditions associated with a given target source. Our experiments show that the complementary information carried by the diverse semantic concepts significantly helps to disentangle and isolate sources of interest much more efficiently compared to single-conditioned models. Moreover, we propose a variation of OCT with condition refinement, in which an initial conditional vector is adapted to the given mixture and transformed to a more amenable representation for target source extraction. We showcase the effectiveness of OCT on diverse source separation experiments where it improves upon permutation invariant models with oracle assignment and obtains state-of-the-art performance in the more challenging task of text-based source separation, outperforming even dedicated text-only conditioned models.