Globally Convergent Accelerated Algorithms for Multilinear Sparse Logistic Regression with $\ell_0$-constraints

TL;DR

This paper introduces a novel accelerated optimization algorithm for multilinear sparse logistic regression with $ ext{l}_0$-constraints, ensuring convergence and demonstrating superior accuracy and speed on synthetic and real datasets.

Contribution

The paper proposes APALM$^+$, an accelerated proximal algorithm with adaptive momentum, providing convergence guarantees for the challenging $ ext{l}_0$-constrained multilinear logistic regression model.

Findings

APALM$^+$$ guarantees convergence to a critical point.

Algorithm outperforms state-of-the-art methods in accuracy.

Empirical results show faster convergence and better feature selection.

Abstract

Tensor data represents a multidimensional array. Regression methods based on low-rank tensor decomposition leverage structural information to reduce the parameter count. Multilinear logistic regression serves as a powerful tool for the analysis of multidimensional data. To improve its efficacy and interpretability, we present a Multilinear Sparse Logistic Regression model with $\ell_0$-constraints ($\ell_0$-MLSR). In contrast to the $\ell_1$-norm and $\ell_2$-norm, the $\ell_0$-norm constraint is better suited for feature selection. However, due to its nonconvex and nonsmooth properties, solving it is challenging and convergence guarantees are lacking. Additionally, the multilinear operation in $\ell_0$-MLSR also brings non-convexity. To tackle these challenges, we propose an Accelerated Proximal Alternating Linearized Minimization with Adaptive Momentum (APALM$^+$) method to solve the $\ell_0$-MLSR model. We provide a proof that APALM$^+$ can ensure the convergence of the objective function of $\ell_0$-MLSR. We also demonstrate that APALM$^+$ is globally convergent to a first-order critical point as well as establish convergence rate by using the Kurdyka-Lojasiewicz property. Empirical results obtained from synthetic and real-world datasets validate the superior performance of our algorithm in terms of both accuracy and speed compared to other state-of-the-art methods.