Federated Learning of Molecular Properties with Graph Neural Networks in a Heterogeneous Setting

TL;DR

This paper introduces a federated learning framework for molecular property prediction using graph neural networks in a heterogeneous data setting, addressing data privacy and distribution challenges in chemistry research.

Contribution

It presents a new benchmark FedChem for heterogeneous federated learning in chemistry and proposes FLIT(+) to improve model performance across diverse client data.

Findings

FLIT(+) outperforms other federated learning schemes on FedChem

Heterogeneous molecular data poses significant learning challenges

The FedChem benchmark enables future research in federated chemistry AI

Abstract

Chemistry research has both high material and computational costs to conduct experiments. Institutions thus consider chemical data to be valuable and there have been few efforts to construct large public datasets for machine learning. Another challenge is that different intuitions are interested in different classes of molecules, creating heterogeneous data that cannot be easily joined by conventional distributed training. In this work, we introduce federated heterogeneous molecular learning to address these challenges. Federated learning allows end-users to build a global model collaboratively while keeping the training data distributed over isolated clients. Due to the lack of related research, we first simulate a heterogeneous federated learning benchmark (FedChem) by jointly performing scaffold splitting and latent Dirichlet allocation on existing datasets for heterogeneously distributed client data. Our results on FedChem show that significant learning challenges arise when working with heterogeneous molecules across clients. We then propose a method to alleviate the problem, namely Federated Learning by Instance reweighTing (FLIT(+)). FLIT(+) can align the local training across heterogeneous clients by improving the performance for uncertain samples. Comprehensive experiments conducted on our new benchmark FedChem validate the advantages of this method over other federated learning schemes. FedChem should enable a new type of collaboration for improving AI in chemistry that mitigates concerns about valuable chemical data.