Ill-Conditioning in Dictionary-Based Dynamic-Equation Learning: A Systems Biology Case Study

TL;DR

This paper analyzes how ill-conditioning impacts the sparse regression-based discovery of biological system dynamics, revealing challenges and potential solutions related to data sampling and basis functions.

Contribution

It systematically investigates the effects of ill-conditioning on model recovery in systems biology, highlighting the influence of data sampling and basis choices.

Findings

Strong multicollinearity can occur with few terms in the model.

Orthogonal polynomial bases do not always improve conditioning.

Proper data sampling aligned with basis functions enhances model recovery.

Abstract

Data-driven discovery of governing equations from time-series data provides a powerful framework for understanding complex biological systems. Library-based approaches that use sparse regression over candidate functions have shown considerable promise, but they face a critical challenge when candidate functions become strongly correlated: numerical ill-conditioning. Poor or restricted sampling, together with particular choices of candidate libraries, can produce strong multicollinearity and numerical instability. In such cases, measurement noise may lead to widely different recovered models, obscuring the true underlying dynamics and hindering accurate system identification. Although sparse regularization promotes parsimonious solutions and can partially mitigate conditioning issues, strong correlations may persist, regularization may bias the recovered models, and the regression problem may remain highly sensitive to small perturbations in the data. We present a systematic analysis of how ill-conditioning affects sparse identification of biological dynamics using benchmark models from systems biology. We show that combinations involving as few as two or three terms can already exhibit strong multicollinearity and extremely large condition numbers. We further show that orthogonal polynomial bases do not consistently resolve ill-conditioning and can perform worse than monomial libraries when the data distribution deviates from the weight function associated with the orthogonal basis. Finally, we demonstrate that when data are sampled from distributions aligned with the appropriate weight functions corresponding to the orthogonal basis, numerical conditioning improves, and orthogonal polynomial bases can yield improved model recovery accuracy across two baseline models.