Fundamentals of Regression

TL;DR

This chapter reviews classical regression tools, discusses the integration of physics-informed methods in machine learning, and explores combining traditional computational science with modern data-driven approaches.

Contribution

It provides a comprehensive overview of traditional and modern regression techniques, emphasizing the emerging physics-informed machine learning methods and their integration with computational science.

Findings

Classical regression tools are foundational in machine learning.

Physics-informed methods enhance regression by incorporating physical laws.

Combining machine learning with numerical methods improves modeling of physical systems.

Abstract

This chapter opens with a review of classic tools for regression, a subset of machine learning that seeks to find relationships between variables. With the advent of scientific machine learning this field has moved from a purely data-driven (statistical) formalism to a constrained or ``physics-informed'' formalism, which integrates physical knowledge and methods from traditional computational engineering. In the first part, we introduce the general concepts and the statistical flavor of regression versus other forms of curve fitting. We then move to an overview of traditional methods from machine learning and their classification and ways to link these to traditional computational science. Finally, we close with a note on methods to combine machine learning and numerical methods for physics