Topological Techniques in Model Selection

TL;DR

This paper introduces a novel topological approach to model selection in polynomial regression, combining Betti number-based complexity measures with validation error to improve sparsity and prediction accuracy.

Contribution

It proposes a new algorithm that integrates topological complexity measures with statistical criteria for better model selection in hierarchical polynomial models.

Findings

Produces sparser models with lower prediction errors

Outperforms several other statistical methods in simulations

Effectively handles higher-order interaction models

Abstract

The LASSO is an attractive regularisation method for linear regression that combines variable selection with an efficient computation procedure. This paper is concerned with enhancing the performance of LASSO for square-free hierarchical polynomial models when combining validation error with a measure of model complexity. The measure of the complexity is the sum of Betti numbers of the model which is seen as a simplicial complex, and we describe the model in terms of components and cycles, borrowing from recent developments in computational topology. We study and propose an algorithm which combines statistical and topological criteria. This compound criterion would allow us to deal with model selection problems in polynomial regression models containing higher-order interactions. Simulation results demonstrate that the compound criteria produce sparser models with lower prediction errors than the estimators of several other statistical methods for higher order interaction models.