Interpret Federated Learning with Shapley Values

TL;DR

This paper explores how to interpret feature importance in vertical Federated Learning using Shapley values, balancing model transparency with data privacy concerns.

Contribution

It introduces a novel method employing Shapley values to interpret feature importance while preserving data privacy in vertical Federated Learning.

Findings

Effective interpretation of host features using Shapley values.

Unified importance measure for guest features.

Robust and informative interpretation results.

Abstract

Federated Learning is introduced to protect privacy by distributing training data into multiple parties. Each party trains its own model and a meta-model is constructed from the sub models. In this way the details of the data are not disclosed in between each party. In this paper we investigate the model interpretation methods for Federated Learning, specifically on the measurement of feature importance of vertical Federated Learning where feature space of the data is divided into two parties, namely host and guest. For host party to interpret a single prediction of vertical Federated Learning model, the interpretation results, namely the feature importance, are very likely to reveal the protected data from guest party. We propose a method to balance the model interpretability and data privacy in vertical Federated Learning by using Shapley values to reveal detailed feature importance for host features and a unified importance value for federated guest features. Our experiments indicate robust and informative results for interpreting Federated Learning models.