Linear regression in the Bayesian framework

TL;DR

This paper reviews various linear regression methods and unifies them within the Bayesian framework, providing formulas and model selection criteria to enhance understanding and application.

Contribution

It offers a unified Bayesian perspective on linear regression methods like least squares, ridge, and LASSO, including derivations and model selection criteria.

Findings

Unified formulas for linear regression methods in Bayesian framework

Derivation of KIC for model selection

Clarification of different regression strategies

Abstract

These notes aim at clarifying different strategies to perform linear regression from given dataset. Methods like the weighted and ordinary least squares, ridge regression or LASSO are proposed in the literature. The present article is my understanding of these methods which are, according to me, better unified in the Bayesian framework. The formulas to address linear regression with these methods are derived. The KIC for model selection is also derived in the end of the document.