Multi-task Joint Strategies of Self-supervised Representation Learning on Biomedical Networks for Drug Discovery

TL;DR

This paper introduces a multi-task self-supervised learning framework, MSSL2drug, that combines various SSL tasks on biomedical networks to improve drug discovery, demonstrating the effectiveness of multimodal and local-global task combinations.

Contribution

It proposes a novel multi-task SSL strategy with a graph attention-based framework for biomedical networks, exploring optimal task combinations for drug discovery.

Findings

Multimodal task combinations outperform single-task models.

Local-global task combinations yield higher performance than random pairs.

Multimodal and local-global strategies serve as guidelines for multi-task SSL in drug discovery.

Abstract

Self-supervised representation learning (SSL) on biomedical networks provides new opportunities for drug discovery. However, how to effectively combine multiple SSL models is still challenging and has been rarely explored. Therefore, we propose multi-task joint strategies of self-supervised representation learning on biomedical networks for drug discovery, named MSSL2drug. We design six basic SSL tasks inspired by various modality features including structures, semantics, and attributes in heterogeneous biomedical networks. Importantly, fifteen combinations of multiple tasks are evaluated by a graph attention-based multi-task adversarial learning framework in two drug discovery scenarios. The results suggest two important findings. (1) Combinations of multimodal tasks achieve the best performance compared to other multi-task joint models. (2) The local-global combination models yield higher performance than random two-task combinations when there are the same size of modalities. Therefore, we conjecture that the multimodal and local-global combination strategies can be treated as the guideline of multi-task SSL for drug discovery.