Graph Representation Learning for Multi-Task Settings: a Meta-Learning Approach

TL;DR

This paper introduces a meta-learning approach for Graph Neural Networks to generate versatile node embeddings capable of supporting multiple tasks efficiently, outperforming traditional single-task models.

Contribution

It proposes a novel meta-learning framework enabling GNNs to produce adaptable multi-task node embeddings with minimal fine-tuning.

Findings

Meta-trained GNNs achieve multi-task performance comparable to single-task models.

Meta-learning enhances the generality and adaptability of node embeddings.

Experimental results show improved or comparable performance across multiple tasks.

Abstract

Graph Neural Networks (GNNs) have become the state-of-the-art method for many applications on graph structured data. GNNs are a model for graph representation learning, which aims at learning to generate low dimensional node embeddings that encapsulate structural and feature-related information. GNNs are usually trained in an end-to-end fashion, leading to highly specialized node embeddings. While this approach achieves great results in the single-task setting, the generation of node embeddings that can be used to perform multiple tasks (with performance comparable to single-task models) is still an open problem. We propose the use of meta-learning to allow the training of a GNN model capable of producing multi-task node embeddings. In particular, we exploit the properties of optimization-based meta-learning to learn GNNs that can produce general node representations by learning parameters that can quickly (i.e. with a few steps of gradient descent) adapt to multiple tasks. Our experiments show that the embeddings produced by a model trained with our purposely designed meta-learning procedure can be used to perform multiple tasks with comparable or, surprisingly, even higher performance than both single-task and multi-task end-to-end models.