Householder Projector for Unsupervised Latent Semantics Discovery

TL;DR

This paper introduces the Householder Projector, a novel orthogonal matrix parameterization method that improves unsupervised latent semantics discovery in StyleGANs by ensuring disentangled and meaningful semantic directions.

Contribution

The paper proposes a Householder-based low-rank orthogonal projector to enhance semantic attribute disentanglement in GAN latent spaces, addressing limitations of previous eigenvector-based methods.

Findings

Improves semantic disentanglement in StyleGANs

Achieves better attribute control with minimal fine-tuning

Maintains high image fidelity during semantic discovery

Abstract

Generative Adversarial Networks (GANs), especially the recent style-based generators (StyleGANs), have versatile semantics in the structured latent space. Latent semantics discovery methods emerge to move around the latent code such that only one factor varies during the traversal. Recently, an unsupervised method proposed a promising direction to directly use the eigenvectors of the projection matrix that maps latent codes to features as the interpretable directions. However, one overlooked fact is that the projection matrix is non-orthogonal and the number of eigenvectors is too large. The non-orthogonality would entangle semantic attributes in the top few eigenvectors, and the large dimensionality might result in meaningless variations among the directions even if the matrix is orthogonal. To avoid these issues, we propose Householder Projector, a flexible and general low-rank orthogonal matrix representation based on Householder transformations, to parameterize the projection matrix. The orthogonality guarantees that the eigenvectors correspond to disentangled interpretable semantics, while the low-rank property encourages that each identified direction has meaningful variations. We integrate our projector into pre-trained StyleGAN2/StyleGAN3 and evaluate the models on several benchmarks. Within only $1\%$ of the original training steps for fine-tuning, our projector helps StyleGANs to discover more disentangled and precise semantic attributes without sacrificing image fidelity.