Toward a `Standard Model' of Machine Learning

TL;DR

This paper proposes a unified formalism and a standard equation for machine learning that unifies various paradigms, facilitating standardized development and panoramic learning from diverse experiences.

Contribution

It introduces a standardized ML formalism with a unifying learning objective that encompasses multiple paradigms, enabling more systematic and reusable ML approach development.

Findings

Unified formalism covers supervised, unsupervised, reinforcement, and adversarial learning.

Provides a standard equation that models diverse ML algorithms as special cases.

Guides the design of new ML methods within a common framework.

Abstract

Machine learning (ML) is about computational methods that enable machines to learn concepts from experience. In handling a wide variety of experience ranging from data instances, knowledge, constraints, to rewards, adversaries, and lifelong interaction in an ever-growing spectrum of tasks, contemporary ML/AI (artificial intelligence) research has resulted in a multitude of learning paradigms and methodologies. Despite the continual progresses on all different fronts, the disparate narrowly focused methods also make standardized, composable, and reusable development of ML approaches difficult, and preclude the opportunity to build AI agents that panoramically learn from all types of experience. This article presents a standardized ML formalism, in particular a `standard equation' of the learning objective, that offers a unifying understanding of many important ML algorithms in the supervised, unsupervised, knowledge-constrained, reinforcement, adversarial, and online learning paradigms, respectively -- those diverse algorithms are encompassed as special cases due to different choices of modeling components. The framework also provides guidance for mechanical design of new ML approaches and serves as a promising vehicle toward panoramic machine learning with all experience.