Variational models for signal processing with Graph Neural Networks

TL;DR

This paper explores variational models integrated with Graph Neural Networks for unsupervised signal processing on point-clouds, highlighting their efficiency and potential for applications with limited labeled data.

Contribution

It demonstrates that variational algorithms on graphs can be reformulated as Message Passing Networks, enabling efficient unsupervised learning of GNNs for signal processing.

Findings

Variational algorithms can be expressed as Message Passing Networks.

Unsupervised GNN training can be achieved via inverse problem optimization.

Variational-based GNNs are computationally efficient compared to traditional methods.

Abstract

This paper is devoted to signal processing on point-clouds by means of neural networks. Nowadays, state-of-the-art in image processing and computer vision is mostly based on training deep convolutional neural networks on large datasets. While it is also the case for the processing of point-clouds with Graph Neural Networks (GNN), the focus has been largely given to high-level tasks such as classification and segmentation using supervised learning on labeled datasets such as ShapeNet. Yet, such datasets are scarce and time-consuming to build depending on the target application. In this work, we investigate the use of variational models for such GNN to process signals on graphs for unsupervised learning. Our contributions are two-fold. We first show that some existing variational-based algorithms for signals on graphs can be formulated as Message Passing Networks (MPN), a particular instance of GNN, making them computationally efficient in practice when compared to standard gradient-based machine learning algorithms. Secondly, we investigate the unsupervised learning of feed-forward GNN, either by direct optimization of an inverse problem or by model distillation from variational-based MPN. Keywords:Graph Processing. Neural Network. Total Variation. Variational Methods. Message Passing Network. Unsupervised learning