Robust Finite Mixture Regression for Heterogeneous Targets

TL;DR

This paper introduces a robust finite mixture regression model that effectively handles heterogeneous, incomplete, and outlier-prone data with shared feature selection, clustering, and anomaly detection capabilities, validated by theoretical bounds and empirical results.

Contribution

The paper presents a novel FMR model that jointly clusters samples, models multiple targets, performs shared feature selection, and detects anomalies, with proven theoretical guarantees.

Findings

Achieves state-of-the-art performance on synthetic and real data

Effectively handles outliers and incomplete targets

Provides non-asymptotic oracle performance bounds

Abstract

Finite Mixture Regression (FMR) refers to the mixture modeling scheme which learns multiple regression models from the training data set. Each of them is in charge of a subset. FMR is an effective scheme for handling sample heterogeneity, where a single regression model is not enough for capturing the complexities of the conditional distribution of the observed samples given the features. In this paper, we propose an FMR model that 1) finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously, 2) achieves shared feature selection among tasks and cluster components, and 3) detects anomaly tasks or clustered structure among tasks, and accommodates outlier samples. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The proposed model is evaluated on both synthetic and real-world data sets. The results show that our model can achieve state-of-the-art performance.