Robust mixture of experts modeling using the $t$ distribution

TL;DR

This paper introduces a robust mixture of experts model using the $t$ distribution to effectively handle heavy-tailed and noisy data in regression and clustering tasks, validated through simulations and real-world applications.

Contribution

The paper develops a $t$ distribution-based MoE model with a dedicated EM algorithm, enhancing robustness over traditional Gaussian-based models.

Findings

Effective in modeling non-linear regression functions

Robust in handling heavy-tailed and noisy data

Successful application to real-world datasets

Abstract

Mixture of Experts (MoE) is a popular framework for modeling heterogeneity in data for regression, classification, and clustering. For regression and cluster analyses of continuous data, MoE usually use normal experts following the Gaussian distribution. However, for a set of data containing a group or groups of observations with heavy tails or atypical observations, the use of normal experts is unsuitable and can unduly affect the fit of the MoE model. We introduce a robust MoE modeling using the $t$ distribution. The proposed $t$ MoE (TMoE) deals with these issues regarding heavy-tailed and noisy data. We develop a dedicated expectation-maximization (EM) algorithm to estimate the parameters of the proposed model by monotonically maximizing the observed data log-likelihood. We describe how the presented model can be used in prediction and in model-based clustering of regression data. The proposed model is validated on numerical experiments carried out on simulated data, which show the effectiveness and the robustness of the proposed model in terms of modeling non-linear regression functions as well as in model-based clustering. Then, it is applied to the real-world data of tone perception for musical data analysis, and the one of temperature anomalies for the analysis of climate change data. The obtained results show the usefulness of the TMoE model for practical applications.