Constant-Target Energy Matching: A Unified Framework for Continuous and Discrete Density Estimation

TL;DR

The paper introduces CTEM, a unified energy-based framework for density estimation across various data types, improving performance and sample quality without complex ratio regression.

Contribution

It proposes a novel bounded energy-difference transform enabling a unified approach to continuous, discrete, and mixed-variable density estimation.

Findings

CTEM outperforms competitive baselines on multiple benchmarks.

It yields higher-quality samples under standard sampling procedures.

The method simplifies density estimation by avoiding partition-function estimation.

Abstract

Density estimation is a central primitive in probabilistic modeling, yet continuous, discrete, and mixed-variable domains are often treated by separate objectives, limiting the ability to exploit a common statistical structure across data types. Continuous score-based methods rely on log-density gradients, while discrete extensions typically use concrete score whose unbounded targets become unstable near low-probability states. We introduce Constant-Target Energy Matching (CTEM), a unified energy-based framework for density estimation on general state spaces. CTEM replaces ordinary density-ratio regression with a bounded energy-difference transform and derives from it a sample-only training objective with the constant target 1. The learned scalar potential recovers log p without partition-function estimation or explicit unbounded ratio regression. Across continuous, discrete, and mixed-variable benchmarks, CTEM substantially improves density estimation over competitive baselines and yields higher-quality samples under standard sampling procedures.