Goodness-of-fit tests on manifolds

TL;DR

This paper introduces a general framework for goodness-of-fit testing on non-linear models involving smooth submanifolds, with applications in machine learning and signal processing, by characterizing residuals as chi-squared distributions.

Contribution

It develops a theoretical foundation for goodness-of-fit tests on non-linear models with Gaussian noise, including a method for sequential model order selection.

Findings

Residuals follow a (possibly noncentral) chi-squared distribution.

Applicable to low-rank matrix and tensor rank determination.

Useful for source number estimation and neural network complexity analysis.

Abstract

We develop a general theory for the goodness-of-fit test to non-linear models. In particular, we assume that the observations are noisy samples of a submanifold defined by a \yao{sufficiently smooth non-linear map}. The observation noise is additive Gaussian. Our main result shows that the "residual" of the model fit, by solving a non-linear least-square problem, follows a (possibly noncentral) $\chi^2$ distribution. The parameters of the $\chi^2$ distribution are related to the model order and dimension of the problem. We further present a method to select the model orders sequentially. We demonstrate the broad application of the general theory in machine learning and signal processing, including determining the rank of low-rank (possibly complex-valued) matrices and tensors from noisy, partial, or indirect observations, determining the number of sources in signal demixing, and potential applications in determining the number of hidden nodes in neural networks.