Reconciled Polynomial Machine: A Unified Representation of Shallow and Deep Learning Models

TL;DR

This paper introduces the reconciled polynomial machine, a unified model that encompasses shallow and deep learning models, providing insights into their representations and learning errors.

Contribution

It presents a unified framework that represents various machine learning models, including deep neural networks, as reconciled polynomial machines, bridging shallow and deep learning.

Findings

All analyzed models can be reduced to the reconciled polynomial machine form.

Provides a unified perspective on the representation of shallow and deep models.

Analyzes learning errors based on function approximation of these models.

Abstract

In this paper, we aim at introducing a new machine learning model, namely reconciled polynomial machine, which can provide a unified representation of existing shallow and deep machine learning models. Reconciled polynomial machine predicts the output by computing the inner product of the feature kernel function and variable reconciling function. Analysis of several concrete models, including Linear Models, FM, MVM, Perceptron, MLP and Deep Neural Networks, will be provided in this paper, which can all be reduced to the reconciled polynomial machine representations. Detailed analysis of the learning error by these models will also be illustrated in this paper based on their reduced representations from the function approximation perspective.