Theoretical Properties of Projection Based Multilayer Perceptrons with Functional Inputs

TL;DR

This paper investigates the theoretical properties of projection-based multilayer perceptrons (MLPs) with functional inputs, demonstrating their universality as approximators and their consistency in learning functional data mappings.

Contribution

It provides the first theoretical analysis showing that projection-based MLPs are universal approximators and are consistent learners for functional data.

Findings

MLPs with functional inputs are universal approximators.

Projection-based MLPs can approximate any continuous functional mapping.

They are consistent learners with decreasing mean square error as data increases.

Abstract

Many real world data are sampled functions. As shown by Functional Data Analysis (FDA) methods, spectra, time series, images, gesture recognition data, etc. can be processed more efficiently if their functional nature is taken into account during the data analysis process. This is done by extending standard data analysis methods so that they can apply to functional inputs. A general way to achieve this goal is to compute projections of the functional data onto a finite dimensional sub-space of the functional space. The coordinates of the data on a basis of this sub-space provide standard vector representations of the functions. The obtained vectors can be processed by any standard method. In our previous work, this general approach has been used to define projection based Multilayer Perceptrons (MLPs) with functional inputs. We study in this paper important theoretical properties of the proposed model. We show in particular that MLPs with functional inputs are universal approximators: they can approximate to arbitrary accuracy any continuous mapping from a compact sub-space of a functional space to R. Moreover, we provide a consistency result that shows that any mapping from a functional space to R can be learned thanks to examples by a projection based MLP: the generalization mean square error of the MLP decreases to the smallest possible mean square error on the data when the number of examples goes to infinity.