S2MAM: Semi-supervised Meta Additive Model for Robust Estimation and Variable Selection

TL;DR

The paper introduces S2MAM, a semi-supervised learning model that automatically selects variables and updates similarity measures for robust, interpretable predictions using a bilevel optimization scheme.

Contribution

It proposes a novel bilevel optimization-based semi-supervised model that enhances variable selection, similarity matrix updating, and interpretability, with theoretical guarantees.

Findings

Validated robustness across synthetic and real datasets.

Achieved interpretable predictions with improved variable selection.

Demonstrated convergence and generalization bounds for the method.

Abstract

Semi-supervised learning with manifold regularization is a classical framework for jointly learning from both labeled and unlabeled data, where the key requirement is that the support of the unknown marginal distribution has the geometric structure of a Riemannian manifold. Typically, the Laplace-Beltrami operator-based manifold regularization can be approximated empirically by the Laplacian regularization associated with the entire training data and its corresponding graph Laplacian matrix. However, the graph Laplacian matrix depends heavily on the prespecified similarity metric and may lead to inappropriate penalties when dealing with redundant or noisy input variables. To address the above issues, this paper proposes a new Semi-Supervised Meta Additive Model (S$^2$MAM) based on a bilevel optimization scheme that automatically identifies informative variables, updates the similarity matrix, and simultaneously achieves interpretable predictions. Theoretical guarantees are provided for S$^2$MAM, including the computing convergence and the statistical generalization bound. Experimental assessments across 4 synthetic and 12 real-world datasets, with varying levels and categories of corruption, validate the robustness and interpretability of the proposed approach.