Complete and separate: Conditional separation with missing target source attribute completion

TL;DR

This paper introduces a method where a model learns to infer missing semantic information about sources in audio mixtures, significantly enhancing separation performance and matching specialized models without requiring complete semantic data.

Contribution

The work presents a novel approach for semantic attribute completion in source separation, improving multi-conditional separation by inferring missing information.

Findings

Separation performance approaches that of an oracle with full semantic info.

The method outperforms existing multi-conditional models.

Achieves comparable results to specialized single conditional models.

Abstract

Recent approaches in source separation leverage semantic information about their input mixtures and constituent sources that when used in conditional separation models can achieve impressive performance. Most approaches along these lines have focused on simple descriptions, which are not always useful for varying types of input mixtures. In this work, we present an approach in which a model, given an input mixture and partial semantic information about a target source, is trained to extract additional semantic data. We then leverage this pre-trained model to improve the separation performance of an uncoupled multi-conditional separation network. Our experiments demonstrate that the separation performance of this multi-conditional model is significantly improved, approaching the performance of an oracle model with complete semantic information. Furthermore, our approach achieves performance levels that are comparable to those of the best performing specialized single conditional models, thus providing an easier to use alternative.