Structured variable selection and estimation

TL;DR

This paper introduces non-negative garrote methods for structured variable selection in linear regression, effectively incorporating hierarchical relationships among predictors, with strong theoretical backing and practical extensions to generalized linear models.

Contribution

It proposes a novel, computationally simple approach for structured variable selection that respects predictor relationships, extending to generalized linear models.

Findings

Methods are easy to compute and theoretically sound.

Simulations demonstrate improved variable selection accuracy.

Real data examples confirm practical effectiveness.

Abstract

In linear regression problems with related predictors, it is desirable to do variable selection and estimation by maintaining the hierarchical or structural relationships among predictors. In this paper we propose non-negative garrote methods that can naturally incorporate such relationships defined through effect heredity principles or marginality principles. We show that the methods are very easy to compute and enjoy nice theoretical properties. We also show that the methods can be easily extended to deal with more general regression problems such as generalized linear models. Simulations and real examples are used to illustrate the merits of the proposed methods.