Federated Automatic Latent Variable Selection in Multi-output Gaussian Processes

TL;DR

This paper introduces a federated learning method for multi-output Gaussian processes that automatically determines the number of latent processes, enhancing privacy and computational efficiency while maintaining transfer learning capabilities.

Contribution

It proposes a hierarchical model with spike-and-slab priors and a federated variational inference algorithm for automatic latent process selection without centralized data sharing.

Findings

Effective latent process selection demonstrated on real datasets

Reduces privacy risks by avoiding data centralization

Improves computational efficiency in federated settings

Abstract

This paper explores a federated learning approach that automatically selects the number of latent processes in multi-output Gaussian processes (MGPs). The MGP has seen great success as a transfer learning tool when data is generated from multiple sources/units/entities. A common approach in MGPs to transfer knowledge across units involves gathering all data from each unit to a central server and extracting common independent latent processes to express each unit as a linear combination of the shared latent patterns. However, this approach poses key challenges in (i) determining the adequate number of latent processes and (ii) relying on centralized learning which leads to potential privacy risks and significant computational burdens on the central server. To address these issues, we propose a hierarchical model that places spike-and-slab priors on the coefficients of each latent process. These priors help automatically select only needed latent processes by shrinking the coefficients of unnecessary ones to zero. To estimate the model while avoiding the drawbacks of centralized learning, we propose a variational inference-based approach, that formulates model inference as an optimization problem compatible with federated settings. We then design a federated learning algorithm that allows units to jointly select and infer the common latent processes without sharing their data. We also discuss an efficient learning approach for a new unit within our proposed federated framework. Simulation and case studies on Li-ion battery degradation and air temperature data demonstrate the advantageous features of our proposed approach.