Why Machine Learning Cannot Ignore Maximum Likelihood Estimation

TL;DR

This paper emphasizes the importance of integrating maximum likelihood estimation into machine learning to ensure statistical rigor and improve the estimation of functional parameters like prediction functions and densities.

Contribution

It highlights the necessity of incorporating maximum likelihood methods into machine learning for better theoretical foundations and practical impact.

Findings

Maximum likelihood is essential for statistical inference in machine learning.

Integrating MLE improves the estimation of prediction functions.

The paper advocates for a foundational shift towards MLE in ML research.

Abstract

The growth of machine learning as a field has been accelerating with increasing interest and publications across fields, including statistics, but predominantly in computer science. How can we parse this vast literature for developments that exemplify the necessary rigor? How many of these manuscripts incorporate foundational theory to allow for statistical inference? Which advances have the greatest potential for impact in practice? One could posit many answers to these queries. Here, we assert that one essential idea is for machine learning to integrate maximum likelihood for estimation of functional parameters, such as prediction functions and conditional densities.