Automated Model Selection for Generalized Linear Models

TL;DR

This paper introduces a novel optimization-based approach that automates feature selection and model fitting for generalized linear models by directly optimizing information criteria and addressing multicollinearity.

Contribution

It presents a mixed-integer conic optimization framework that integrates feature selection with generalized linear models, incorporating new correlation constraints to handle multicollinearity.

Findings

Effective automated model selection using mixed-integer conic optimization.

Improved handling of multicollinearity through novel correlation constraints.

Direct optimization of AIC and BIC criteria for model selection.

Abstract

In this paper, we show how mixed-integer conic optimization can be used to combine feature subset selection with holistic generalized linear models to fully automate the model selection process. Concretely, we directly optimize for the Akaike and Bayesian information criteria while imposing constraints designed to deal with multicollinearity in the feature selection task. Specifically, we propose a novel pairwise correlation constraint that combines the sign coherence constraint with ideas from classical statistical models like Ridge regression and the OSCAR model.