Piecewise-Linear Approximation for Feature Subset Selection in a Sequential Logit Model

TL;DR

This paper introduces a piecewise-linear approximation method for feature subset selection in sequential logit models, improving over quadratic approximation by better capturing the logistic loss and resulting in more effective feature selection.

Contribution

The paper presents a novel piecewise-linear approximation approach that transforms the feature selection problem into a mixed integer linear optimization, outperforming previous quadratic approximation methods.

Findings

The piecewise-linear approach finds better feature subsets than quadratic approximation.

The method effectively minimizes the information criterion in feature selection.

Computational results validate the superiority of the proposed approximation.

Abstract

This paper concerns a method of selecting a subset of features for a sequential logit model. Tanaka and Nakagawa (2014) proposed a mixed integer quadratic optimization formulation for solving the problem based on a quadratic approximation of the logistic loss function. However, since there is a significant gap between the logistic loss function and its quadratic approximation, their formulation may fail to find a good subset of features. To overcome this drawback, we apply a piecewise-linear approximation to the logistic loss function. Accordingly, we frame the feature subset selection problem of minimizing an information criterion as a mixed integer linear optimization problem. The computational results demonstrate that our piecewise-linear approximation approach found a better subset of features than the quadratic approximation approach.