A Learning Theory in Linear Systems under Compositional Models

TL;DR

This paper develops a learning theory for training linear operators with compositional inputs, using Bayesian inversion to infer unknown variables from noisy outputs, and quantifies uncertainty propagation and convergence rates.

Contribution

It introduces a Bayesian approach for learning linear systems with compositional variables and derives explicit convergence rates under stochastic models.

Findings

Quantified uncertainty in trained operators.

Derived explicit convergence rates for training.

Demonstrated uncertainty propagation from noisy data.

Abstract

We present a learning theory for the training of a linear system operator having an input compositional variable and propose a Bayesian inversion method for inferring the unknown variable from an output of a noisy linear system. We assume that we have partial or even no knowledge of the operator but have training data of input and ouput. A compositional variable satisfies the constraints that the elements of the variable are all non-negative and sum to unity. We quantified the uncertainty in the trained operator and present the convergence rates of training in explicit forms for several interesting cases under stochastic compositional models. The trained linear operator with the covariance matrix, estimated from the training set of pairs of ground-truth input and noisy output data, is further used in evaluation of posterior uncertainty of the solution. This posterior uncertainty clearly demonstrates uncertainty propagation from noisy training data and addresses possible mismatch between the true operator and the estimated one in the final solution.