Directly Training Joint Energy-Based Models for Conditional Synthesis and Calibrated Prediction of Multi-Attribute Data

TL;DR

This paper introduces an extension of energy-based models that enables both accurate multi-attribute predictions and the generation of high-quality samples conditioned on novel attribute combinations, improving uncertainty quantification and synthesis.

Contribution

The authors propose a simple extension to EBMs that allows for conditional sample generation in multi-attribute data, enhancing their predictive and generative capabilities.

Findings

Models achieve accurate, calibrated predictions.

Models generate high-quality conditional samples.

Approach effectively handles high-dimensional data.

Abstract

Multi-attribute classification generalizes classification, presenting new challenges for making accurate predictions and quantifying uncertainty. We build upon recent work and show that architectures for multi-attribute prediction can be reinterpreted as energy-based models (EBMs). While existing EBM approaches achieve strong discriminative performance, they are unable to generate samples conditioned on novel attribute combinations. We propose a simple extension which expands the capabilities of EBMs to generating accurate conditional samples. Our approach, combined with newly developed techniques in energy-based model training, allows us to directly maximize the likelihood of data and labels under the unnormalized joint distribution. We evaluate our proposed approach on high-dimensional image data with high-dimensional binary attribute labels. We find our models are capable of both accurate, calibrated predictions and high-quality conditional synthesis of novel attribute combinations.