The Gauss-Markov Adjunction Provides Categorical Semantics of Residuals in Supervised Learning

TL;DR

This paper introduces a categorical semantics framework for residuals in supervised learning, specifically multiple linear regression, using the Gauss-Markov Adjunction to formalize the relationship between parameters and residuals.

Contribution

It develops a novel categorical formulation of supervised learning models, formalizing the interplay between residuals and parameters through the Gauss-Markov Adjunction.

Findings

Categorical modeling clarifies residual-parameter relationships.

The Gauss-Markov Adjunction captures the structure of linear regression.

This framework links residuals and estimators via functor limits.

Abstract

Enhancing the intelligibility and interpretability of machine learning is a crucial task in responding to the demand for Explicability as an AI principle, and in promoting the better social implementation of AI. The aim of our research is to contribute to this improvement by reformulating machine learning models through the lens of category theory, thereby developing a semantic framework for structuring and understanding AI systems. Our categorical modeling in this paper clarifies and formalizes the structural interplay between residuals and parameters in supervised learning. The present paper focuses on the multiple linear regression model, which represents the most basic form of supervised learning. By defining two Lawvere-enriched categories corresponding to parameters and data, along with an adjoint pair of functors between them, we introduce our categorical formulation of supervised learning. We show that the essential structure of this framework is captured by what we call the Gauss-Markov Adjunction. Within this setting, the dual flow of information can be explicitly described as a correspondence between variations in parameters and residuals. The ordinary least squares estimator for the parameters and the minimum residual are related via the preservation of limits by the right adjoint functor. Furthermore, we position this formulation as an instance of extended denotational semantics for supervised learning, and propose applying a semantic perspective developed in theoretical computer science as a formal foundation for Explicability in AI.