Classes of ODE solutions: smoothness, covering numbers, implications for noisy function fitting, and the curse of smoothness phenomenon

TL;DR

This paper investigates the smoothness, complexity, and statistical properties of classes of solutions to ordinary differential equations (ODEs), revealing a 'curse of smoothness' and providing bounds relevant for noisy function recovery.

Contribution

It establishes novel bounds on the smoothness and covering numbers of ODE solution classes, and analyzes their implications for function fitting and the curse of smoothness phenomenon.

Findings

Solutions can have derivatives growing factorially fast, indicating a curse of smoothness.

Covering numbers of solution classes are larger than standard smooth function classes.

The minimax optimal convergence rate for noisy recovery is derived under certain conditions.

Abstract

Many numerical methods for recovering ODE solutions from data rely on approximating the solutions using basis functions or kernel functions under a least square criterion. The accuracy of this approach hinges on the smoothness of the solutions. This paper provides a theoretical foundation for these methods by establishing novel results on the smoothness and covering numbers of ODE solution classes (as a measure of their "size"). Our results provide answers to "how do the degree of smoothness and the "size" of a class of ODEs affect the "size" of the associated class of solutions?" We show that: (1) for $y^{'}=f\left(y\right)$ and $y^{'}=f\left(x,\,y\right)$, if the absolute values of all $k$th ($k\leq\beta+1$) order derivatives of $f$ are bounded by $1$, then the solution can end up with the $(k+1)$th derivative whose magnitude grows factorially fast in $k$ -- "a curse of smoothness"; (2) our upper bounds for the covering numbers of the $(\beta+2)-$degree smooth solution classes are greater than those of the "standard" $(\beta+2)-$degree smooth class of univariate functions; (3) the mean squared error of least squares fitting for noisy recovery has a convergence rate no larger than $\left(\frac{1}{n}\right)^{\frac{2\left(\beta+2\right)}{2\left(\beta+2\right)+1}}$ if $n=\Omega\left(\left(\beta\sqrt{\log\left(\beta\vee1\right)}\right)^{4\beta+10}\right)$, and under this condition, the rate $\left(\frac{1}{n}\right)^{\frac{2\left(\beta+2\right)}{2\left(\beta+2\right)+1}}$ is minimax optimal in the case of $y^{'}=f\left(x,\,y\right)$; (4) more generally, for the higher order Picard type ODEs, $y^{\left(m\right)}=f\left(x,\,y,\,y^{'},\,...,y^{\left(m-1\right)}\right)$, the covering number of the solution class is bounded from above by the product of the covering number of the class $\mathcal{F}$ that $f$ ranges over and the covering number of the set where initial values lie.