Linear combinations of latents in generative models: subspaces and beyond

TL;DR

This paper introduces LOL, a versatile method for forming linear combinations of latent variables in generative models, enabling more controlled and expressive data generation across different modalities.

Contribution

It proposes LOL, a general-purpose, easy-to-implement approach for linear combinations of latents that works broadly across models and data types.

Findings

LOL simplifies the creation of low-dimensional latent representations

It effectively constructs subspaces within the latent space

LOL enhances control over generative processes

Abstract

Sampling from generative models has become a crucial tool for applications like data synthesis and augmentation. Diffusion, Flow Matching and Continuous Normalising Flows have shown effectiveness across various modalities, and rely on latent variables for generation. For experimental design or creative applications that require more control over the generation process, it has become common to manipulate the latent variable directly. However, existing approaches for performing such manipulations (e.g. interpolation or forming low-dimensional representations) only work well in special cases or are network or data-modality specific. We propose Latent Optimal Linear combinations (LOL) as a general-purpose method to form linear combinations of latent variables that adhere to the assumptions of the generative model. As LOL is easy to implement and naturally addresses the broader task of forming any linear combinations, e.g. the construction of subspaces of the latent space, LOL dramatically simplifies the creation of expressive low-dimensional representations of high-dimensional objects.