Unsupervised learning of observation functions in state-space models by nonparametric moment methods

TL;DR

This paper introduces a nonparametric moment-based method for unsupervised learning of non-invertible observation functions in nonlinear state-space models, addressing identifiability and convergence issues.

Contribution

It proposes a novel generalized moment approach for estimating observation functions without invertibility, utilizing RKHS and constrained regression, with theoretical and numerical validation.

Findings

Identifiability characterized by RKHS closure.

Convergent estimators for various function types.

Limitations include symmetry and stationarity issues.

Abstract

We investigate the unsupervised learning of non-invertible observation functions in nonlinear state-space models. Assuming abundant data of the observation process along with the distribution of the state process, we introduce a nonparametric generalized moment method to estimate the observation function via constrained regression. The major challenge comes from the non-invertibility of the observation function and the lack of data pairs between the state and observation. We address the fundamental issue of identifiability from quadratic loss functionals and show that the function space of identifiability is the closure of a RKHS that is intrinsic to the state process. Numerical results show that the first two moments and temporal correlations, along with upper and lower bounds, can identify functions ranging from piecewise polynomials to smooth functions, leading to convergent estimators. The limitations of this method, such as non-identifiability due to symmetry and stationarity, are also discussed.