Federated Latent Class Regression for Hierarchical Data

TL;DR

This paper introduces FEDHLCR, a federated learning approach using hierarchical latent class regression to improve accuracy and robustness on noisy, hierarchical, and non-IID datasets, with strong theoretical guarantees.

Contribution

It proposes a novel probabilistic federated regression model, FEDHLCR, combining mixture models with Bayesian inference for better accuracy and convergence in heterogeneous data environments.

Findings

FEDHLCR achieves fast convergence on non-IID datasets.

The model maintains analytical properties and avoids overfitting.

Experimental results demonstrate improved accuracy over traditional methods.

Abstract

Federated Learning (FL) allows a number of agents to participate in training a global machine learning model without disclosing locally stored data. Compared to traditional distributed learning, the heterogeneity (non-IID) of the agents slows down the convergence in FL. Furthermore, many datasets, being too noisy or too small, are easily overfitted by complex models, such as deep neural networks. Here, we consider the problem of using FL regression on noisy, hierarchical and tabular datasets in which user distributions are significantly different. Inspired by Latent Class Regression (LCR), we propose a novel probabilistic model, Hierarchical Latent Class Regression (HLCR), and its extension to Federated Learning, FEDHLCR. FEDHLCR consists of a mixture of linear regression models, allowing better accuracy than simple linear regression, while at the same time maintaining its analytical properties and avoiding overfitting. Our inference algorithm, being derived from Bayesian theory, provides strong convergence guarantees and good robustness to overfitting. Experimental results show that FEDHLCR offers fast convergence even in non-IID datasets.