CoBELa: Steering Transparent Generation via Concept Bottlenecks on Energy Landscapes

TL;DR

CoBELa introduces an energy-based, decoder-free concept bottleneck framework for interpretable image generation, supporting compositional interventions and achieving improved accuracy and image quality on benchmark datasets.

Contribution

It proposes a novel energy-based approach that eliminates explicit bottleneck representations and enables post-hoc interpretability and compositional concept interventions without retraining.

Findings

Achieves 75.70%/82.42% concept accuracy on CelebA-HQ and CUB-200-2011.

Attains 6.47/5.37 FID scores, outperforming prior models.

Supports efficient multi-concept interventions through additive energy composition.

Abstract

Generative concept bottleneck models aim to enable interpretable generation by routing synthesis through explicit, user-facing concepts. In practice, prior approaches often rely on non-explicit bottleneck representations (e.g., vision cues or opaque concept embeddings) or black-box decoders to preserve image quality, which weakens the transparency. We propose CoBELa (Concept Bottlenecks on Energy Landscapes), a decoder-free, energy-based framework that eliminates non-explicit bottleneck representations by conditioning generation entirely through per-concept energy functions over the latent space of a frozen pretrained generator-requiring no generator retraining and enabling post-hoc interpretation. Because these concept energies compose additively, CoBELa naturally supports compositional concept interventions: concept conjunction and negation are realized by summing or subtracting per-concept energy terms without additional training. A diffusion-scheduled energy guidance scheme further replaces expensive MCMC chains with more stable, scheduled denoising for efficient concept-steered sampling. Experiments on CelebA-HQ and CUB-200-2011 demonstrate improvements over prior concept bottleneck generative models, achieving 75.70%/82.42% concept accuracy and 6.47/5.37 FID, respectively, while enabling reliable multi-concept interventions.