Hamiltonian Score Matching and Generative Flows

TL;DR

This paper introduces Hamiltonian Score Matching and Hamiltonian Generative Flows, leveraging deliberately designed force fields in Hamiltonian ODEs for improved score estimation and generative modeling, unifying several existing approaches.

Contribution

It proposes novel methods using Hamiltonian velocity predictors for score matching and generative flows, expanding the design space of force fields in Hamiltonian systems.

Findings

HSM serves as a new score matching metric.

HGFs perform competitively with state-of-the-art generative models.

Oscillation HGFs demonstrate the versatility of force field design.

Abstract

Classical Hamiltonian mechanics has been widely used in machine learning in the form of Hamiltonian Monte Carlo for applications with predetermined force fields. In this work, we explore the potential of deliberately designing force fields for Hamiltonian ODEs, introducing Hamiltonian velocity predictors (HVPs) as a tool for score matching and generative models. We present two innovations constructed with HVPs: Hamiltonian Score Matching (HSM), which estimates score functions by augmenting data via Hamiltonian trajectories, and Hamiltonian Generative Flows (HGFs), a novel generative model that encompasses diffusion models and flow matching as HGFs with zero force fields. We showcase the extended design space of force fields by introducing Oscillation HGFs, a generative model inspired by harmonic oscillators. Our experiments validate our theoretical insights about HSM as a novel score matching metric and demonstrate that HGFs rival leading generative modeling techniques.