PolyNODE: Variable-dimension Neural ODEs on M-polyfolds

TL;DR

PolyNODEs extend neural ODEs to variable-dimensional M-polyfolds, enabling flow-based models that handle changing dimensions, with applications in autoencoding and classification tasks.

Contribution

This work introduces PolyNODEs, the first neural ODE models capable of operating on variable-dimensional spaces called M-polyfolds, expanding geometric deep learning capabilities.

Findings

Successfully trained PolyNODE autoencoders on M-polyfolds with dimensional bottlenecks.

Extracted meaningful latent representations for downstream classification.

Demonstrated the model's ability to handle variable dimensions in flow-based learning.

Abstract

Neural ordinary differential equations (NODEs) are geometric deep learning models based on dynamical systems and flows generated by vector fields on manifolds. Despite numerous successful applications, particularly within the flow matching paradigm, all existing NODE models are fundamentally constrained to fixed-dimensional dynamics by the intrinsic nature of the manifold's dimension. In this paper, we extend NODEs to M-polyfolds (spaces that can simultaneously accommodate varying dimensions and a notion of differentiability) and introduce PolyNODEs, the first variable-dimensional flow-based model in geometric deep learning. As an example application, we construct explicit M-polyfolds featuring dimensional bottlenecks and PolyNODE autoencoders based on parametrised vector fields that traverse these bottlenecks. We demonstrate experimentally that our PolyNODE models can be trained to solve reconstruction tasks in these spaces, and that latent representations of the input can be extracted and used to solve downstream classification tasks. The code used in our experiments is publicly available at https://github.com/turbotage/PolyNODE .