INRFlow: Flow Matching for INRs in Ambient Space

TL;DR

INRFlow introduces a domain-agnostic flow matching approach that directly models diverse data modalities in ambient space, simplifying training and improving performance across images, 3D point clouds, and protein structures.

Contribution

It proposes INRFlow, a novel ambient space flow matching method inspired by INRs, enabling continuous predictions without domain-specific compressors.

Findings

Effective handling of multiple data modalities

Outperforms comparable approaches in experiments

Simplifies the training pipeline across domains

Abstract

Flow matching models have emerged as a powerful method for generative modeling on domains like images or videos, and even on irregular or unstructured data like 3D point clouds or even protein structures. These models are commonly trained in two stages: first, a data compressor is trained, and in a subsequent training stage a flow matching generative model is trained in the latent space of the data compressor. This two-stage paradigm sets obstacles for unifying models across data domains, as hand-crafted compressors architectures are used for different data modalities. To this end, we introduce INRFlow, a domain-agnostic approach to learn flow matching transformers directly in ambient space. Drawing inspiration from INRs, we introduce a conditionally independent point-wise training objective that enables INRFlow to make predictions continuously in coordinate space. Our empirical results demonstrate that INRFlow effectively handles different data modalities such as images, 3D point clouds and protein structure data, achieving strong performance in different domains and outperforming comparable approaches. INRFlow is a promising step towards domain-agnostic flow matching generative models that can be trivially adopted in different data domains.