Convex Hull Approximation of Nearly Optimal Lasso Solutions

TL;DR

This paper introduces a method to approximate the set of nearly optimal Lasso solutions using convex hulls, enabling diverse feature selection beyond a single optimal solution.

Contribution

It proposes a novel algorithm that efficiently approximates the solution set of nearly optimal Lasso solutions, increasing diversity in feature selection.

Findings

The algorithm effectively approximates the set of nearly optimal solutions.

It produces diverse Lasso solutions with high approximation accuracy.

Experimental results validate the method's efficiency and diversity enhancement.

Abstract

In an ordinary feature selection procedure, a set of important features is obtained by solving an optimization problem such as the Lasso regression problem, and we expect that the obtained features explain the data well. In this study, instead of the single optimal solution, we consider finding a set of diverse yet nearly optimal solutions. To this end, we formulate the problem as finding a small number of solutions such that the convex hull of these solutions approximates the set of nearly optimal solutions. The proposed algorithm consists of two steps: First, we randomly sample the extreme points of the set of nearly optimal solutions. Then, we select a small number of points using a greedy algorithm. The experimental results indicate that the proposed algorithm can approximate the solution set well. The results also indicate that we can obtain Lasso solutions with a large diversity.