Sparse hierarchical interaction learning with epigraphical projection

TL;DR

This paper introduces a primal-dual proximal algorithm with epigraphical projection for hierarchical interaction learning, demonstrating improved efficiency and effectiveness over existing methods on synthetic and real datasets.

Contribution

It proposes a novel optimization algorithm for hierarchical interaction learning that outperforms state-of-the-art methods and enables fair comparison of hierarchical penalizations.

Findings

The proposed method outperforms FISTA and ADMM in experiments.

It provides a flexible framework for comparing hierarchical regularizations.

The algorithm is effective on both synthetic and real data.

Abstract

This work focuses on learning optimization problems with quadratical interactions between variables, which go beyond the additive models of traditional linear learning. We investigate more specifically two different methods encountered in the literature to deal with this problem: "hierNet" and structured-sparsity regularization, and study their connections. We propose a primal-dual proximal algorithm based on an epigraphical projection to optimize a general formulation of these learning problems. The experimental setting first highlights the improvement of the proposed procedure compared to state-of-the-art methods based on fast iterative shrinkage-thresholding algorithm (i.e. FISTA) or alternating direction method of multipliers (i.e. ADMM), and then, using the proposed flexible optimization framework, we provide fair comparisons between the different hierarchical penalizations and their improvement over the standard $\ell_1$-norm penalization. The experiments are conducted both on synthetic and real data, and they clearly show that the proposed primal-dual proximal algorithm based on epigraphical projection is efficient and effective to solve and investigate the problem of hierarchical interaction learning.