Hyperbolic Timbre Embedding for Musical Instrument Sound Synthesis Based on Variational Autoencoders

TL;DR

This paper introduces a novel VAE-based musical instrument sound synthesis method that employs hyperbolic space to better capture the hierarchical structure of timbre, improving latent representation efficiency.

Contribution

It proposes using hyperbolic space in VAE for timbre representation, addressing the hierarchical nature of instrument classification.

Findings

Hyperbolic space improves timbre representation efficiency.

The method captures hierarchical instrument structures effectively.

Experimental results outperform Euclidean-based approaches.

Abstract

In this paper, we propose a musical instrument sound synthesis (MISS) method based on a variational autoencoder (VAE) that has a hierarchy-inducing latent space for timbre. VAE-based MISS methods embed an input signal into a low-dimensional latent representation that captures the characteristics of the input. Adequately manipulating this representation leads to sound morphing and timbre replacement. Although most VAE-based MISS methods seek a disentangled representation of pitch and timbre, how to capture an underlying structure in timbre remains an open problem. To address this problem, we focus on the fact that musical instruments can be hierarchically classified on the basis of their physical mechanisms. Motivated by this hierarchy, we propose a VAE-based MISS method by introducing a hyperbolic space for timbre. The hyperbolic space can represent treelike data more efficiently than the Euclidean space owing to its exponential growth property. Results of experiments show that the proposed method provides an efficient latent representation of timbre compared with the method using the Euclidean space.