Encouraging Disentangled and Convex Representation with Controllable Interpolation Regularization

TL;DR

This paper introduces Controllable Interpolation Regularization (CIR), a novel method that enhances disentangled and convex representations in controllable learning, improving downstream tasks like image synthesis and translation.

Contribution

The paper proposes CIR, a simple yet effective regularization technique that simultaneously promotes disentanglement and convexity, and demonstrates its compatibility with existing algorithms.

Findings

CIR improves disentanglement and convexity in latent space.

Enhanced representations lead to better controllable image synthesis.

CIR shows effectiveness across multiple algorithms and downstream tasks.

Abstract

We focus on controllable disentangled representation learning (C-Dis-RL), where users can control the partition of the disentangled latent space to factorize dataset attributes (concepts) for downstream tasks. Two general problems remain under-explored in current methods: (1) They lack comprehensive disentanglement constraints, especially missing the minimization of mutual information between different attributes across latent and observation domains. (2) They lack convexity constraints, which is important for meaningfully manipulating specific attributes for downstream tasks. To encourage both comprehensive C-Dis-RL and convexity simultaneously, we propose a simple yet efficient method: Controllable Interpolation Regularization (CIR), which creates a positive loop where disentanglement and convexity can help each other. Specifically, we conduct controlled interpolation in latent space during training, and we reuse the encoder to help form a 'perfect disentanglement' regularization. In that case, (a) disentanglement loss implicitly enlarges the potential understandable distribution to encourage convexity; (b) convexity can in turn improve robust and precise disentanglement. CIR is a general module and we merge CIR with three different algorithms: ELEGANT, I2I-Dis, and GZS-Net to show the compatibility and effectiveness. Qualitative and quantitative experiments show improvement in C-Dis-RL and latent convexity by CIR. This further improves downstream tasks: controllable image synthesis, cross-modality image translation, and zero-shot synthesis.