Balancing Information Preservation and Disentanglement in Self-Supervised Music Representation Learning

TL;DR

This paper introduces a multi-view self-supervised learning framework that combines contrastive and reconstructive objectives to improve music audio representation disentanglement while preserving information fidelity.

Contribution

It proposes a novel unified SSL framework that balances information preservation and semantic disentanglement in music representations, with extensive evaluation of contrastive strategies.

Findings

Contrastive and reconstructive strategies exhibit trade-offs.

Combined strategies enable disentanglement without losing information.

Effective combination improves music attribute separation.

Abstract

Recent advances in self-supervised learning (SSL) methods offer a range of strategies for capturing useful representations from music audio without the need for labeled data. While some techniques focus on preserving comprehensive details through reconstruction, others favor semantic structure via contrastive objectives. Few works examine the interaction between these paradigms in a unified SSL framework. In this work, we propose a multi-view SSL framework for disentangling music audio representations that combines contrastive and reconstructive objectives. The architecture is designed to promote both information fidelity and structured semantics of factors in disentangled subspaces. We perform an extensive evaluation on the design choices of contrastive strategies using music audio representations in a controlled setting. We find that while reconstruction and contrastive strategies exhibit consistent trade-offs, when combined effectively, they complement each other; this enables the disentanglement of music attributes without compromising information integrity.