On Some Integrated Approaches to Inference

TL;DR

This paper advocates for a unified framework for continuous inference methods by explicitly partitioning prior and data information, enabling better comparison and development of optimal algorithms across various approaches.

Contribution

It introduces an extended continuous complexity model incorporating optimization-based algorithms, unifying neural networks, Monte Carlo, spline, and regularization methods under a common theoretical framework.

Findings

Proposes a standardized approach to compare inference methods.

Extends continuous complexity models with optimization algorithms.

Facilitates the development of optimal inference algorithms.

Abstract

We present arguments for the formulation of unified approach to different standard continuous inference methods from partial information. It is claimed that an explicit partition of information into a priori (prior knowledge) and a posteriori information (data) is an important way of standardizing inference approaches so that they can be compared on a normative scale, and so that notions of optimal algorithms become farther-reaching. The inference methods considered include neural network approaches, information-based complexity, and Monte Carlo, spline, and regularization methods. The model is an extension of currently used continuous complexity models, with a class of algorithms in the form of optimization methods, in which an optimization functional (involving the data) is minimized. This extends the family of current approaches in continuous complexity theory, which include the use of interpolatory algorithms in worst and average case settings.