Enhancing Molecular Property Prediction via Mixture of Collaborative Experts

TL;DR

This paper introduces GNN-MoCE, a novel architecture for molecular property prediction that leverages a mixture of collaborative experts with specialized projections and loss functions, improving performance especially on limited or imbalanced datasets.

Contribution

The paper proposes the GNN-MoCE model with expert-specific projections and loss, enhancing expert collaboration and diversity in molecular property prediction tasks.

Findings

Outperforms traditional methods on 24 MPP datasets

Effective in limited data and high imbalance scenarios

Demonstrates superior predictive accuracy

Abstract

Molecular Property Prediction (MPP) task involves predicting biochemical properties based on molecular features, such as molecular graph structures, contributing to the discovery of lead compounds in drug development. To address data scarcity and imbalance in MPP, some studies have adopted Graph Neural Networks (GNN) as an encoder to extract commonalities from molecular graphs. However, these approaches often use a separate predictor for each task, neglecting the shared characteristics among predictors corresponding to different tasks. In response to this limitation, we introduce the GNN-MoCE architecture. It employs the Mixture of Collaborative Experts (MoCE) as predictors, exploiting task commonalities while confronting the homogeneity issue in the expert pool and the decision dominance dilemma within the expert group. To enhance expert diversity for collaboration among all experts, the Expert-Specific Projection method is proposed to assign a unique projection perspective to each expert. To balance decision-making influence for collaboration within the expert group, the Expert-Specific Loss is presented to integrate individual expert loss into the weighted decision loss of the group for more equitable training. Benefiting from the enhancements of MoCE in expert creation, dynamic expert group formation, and experts' collaboration, our model demonstrates superior performance over traditional methods on 24 MPP datasets, especially in tasks with limited data or high imbalance.