# Structure learning via unstructured kernel-based M-regression

## TL;DR

This paper introduces a versatile kernel-based M-regression framework in RKHS for uncovering true target function structures, including sparsity and interactions, applicable across various loss functions with proven asymptotic properties and demonstrated effectiveness.

## Contribution

It presents a novel, general framework for structure learning in statistical models using unstructured M-regression in RKHS, accommodating diverse loss functions and providing theoretical guarantees.

## Key findings

- Framework effectively recovers true structures in simulations.
- Applicable to multiple loss functions including regression and classification.
- Demonstrates superior performance in real case study.

## Abstract

In statistical learning, identifying underlying structures of true target functions based on observed data plays a crucial role to facilitate subsequent modeling and analysis. Unlike most of those existing methods that focus on some specific settings under certain model assumptions, this paper proposes a general and novel framework for recovering true structures of target functions by using unstructured M-regression in a reproducing kernel Hilbert space (RKHS). The proposed framework is inspired by the fact that gradient functions can be employed as a valid tool to learn underlying structures, including sparse learning, interaction selection and model identification, and it is easy to implement by taking advantage of the nice properties of the RKHS. More importantly, it admits a wide range of loss functions, and thus includes many commonly used methods, such as mean regression, quantile regression, likelihood-based classification, and margin-based classification, which is also computationally efficient by solving convex optimization tasks. The asymptotic results of the proposed framework are established within a rich family of loss functions without any explicit model specifications. The superior performance of the proposed framework is also demonstrated by a variety of simulated examples and a real case study.