Automatic and Structure-Aware Sparsification of Hybrid Neural ODEs

TL;DR

This paper introduces an automatic structure-aware sparsification method for hybrid neural ODEs, enhancing their efficiency and robustness in healthcare applications by reducing over-complexity while maintaining mechanistic plausibility.

Contribution

It presents a novel hybrid pipeline combining domain-informed graph modifications with data-driven regularization for automatic state selection and structure optimization in neural ODEs.

Findings

Improved predictive performance on synthetic and real-world data.

Enhanced robustness and stability of hybrid neural ODEs.

Achieved desired sparsity without sacrificing mechanistic plausibility.

Abstract

Hybrid neural ordinary differential equations (neural ODEs) integrate mechanistic models with neural ODEs, offering strong inductive bias and flexibility, and are particularly advantageous in data-scarce healthcare settings. However, excessive latent states and interactions from mechanistic models can lead to training inefficiency and over-fitting, limiting practical effectiveness of hybrid neural ODEs. In response, we propose a new hybrid pipeline for automatic state selection and structure optimization in mechanistic neural ODEs, combining domain-informed graph modifications with data-driven regularization to sparsify the model for improving predictive performance and stability while retaining mechanistic plausibility. Experiments on synthetic and real-world data show improved predictive performance and robustness with desired sparsity, establishing an effective solution for hybrid model reduction in healthcare applications.