Convex Techniques for Model Selection

TL;DR

This paper introduces a convex algorithm for automatic regularization parameter selection in model selection, demonstrated on ridge regression with potential extension to complex models, aiming to reduce manual tuning.

Contribution

It presents a novel convex approach for automating regularization parameter tuning in model selection, applicable to various models including ridge regression.

Findings

The convex method performs comparably or better than standard methods.

It effectively automates the parameter tuning process.

The approach extends to complex models beyond ridge regression.

Abstract

We develop a robust convex algorithm to select the regularization parameter in model selection. In practice this would be automated in order to save practitioners time from having to tune it manually. In particular, we implement and test the convex method for $K$-fold cross validation on ridge regression, although the same concept extends to more complex models. We then compare its performance with standard methods.