Best subset selection in linear regression via bi-objective mixed integer linear programming

TL;DR

This paper introduces a bi-objective mixed integer linear programming method for best subset selection in linear regression, effectively balancing bias minimization and predictor count, and demonstrates its computational advantages.

Contribution

It presents a novel bi-objective optimization framework for subset selection, addressing limitations of existing single-objective methods.

Findings

Proposed approach outperforms traditional methods in computational efficiency.

Effectively balances bias reduction and model simplicity.

Shows promising results in computational experiments.

Abstract

We study the problem of choosing the best subset of p features in linear regression given n observations. This problem naturally contains two objective functions including minimizing the amount of bias and minimizing the number of predictors. The existing approaches transform the problem into a single-objective optimization problem. We explain the main weaknesses of existing approaches, and to overcome their drawbacks, we propose a bi-objective mixed integer linear programming approach. A computational study shows the efficacy of the proposed approach.