LaplaceNet: A Hybrid Graph-Energy Neural Network for Deep Semi-Supervised Classification

TL;DR

LaplaceNet introduces a hybrid graph-energy neural network framework for semi-supervised classification that reduces model complexity and outperforms existing methods on benchmark datasets.

Contribution

It proposes a novel semi-supervised learning approach using Laplacian energy minimization for pseudo-labeling, with theoretical justification and improved generalization.

Findings

Outperforms state-of-the-art methods on benchmark datasets

Multi-sampling augmentation improves generalization

Reduces model complexity compared to existing approaches

Abstract

Semi-supervised learning has received a lot of recent attention as it alleviates the need for large amounts of labelled data which can often be expensive, requires expert knowledge and be time consuming to collect. Recent developments in deep semi-supervised classification have reached unprecedented performance and the gap between supervised and semi-supervised learning is ever-decreasing. This improvement in performance has been based on the inclusion of numerous technical tricks, strong augmentation techniques and costly optimisation schemes with multi-term loss functions. We propose a new framework, LaplaceNet, for deep semi-supervised classification that has a greatly reduced model complexity. We utilise a hybrid approach where pseudolabels are produced by minimising the Laplacian energy on a graph. These pseudo-labels are then used to iteratively train a neural-network backbone. Our model outperforms state-of-the art methods for deep semi-supervised classification, over several benchmark datasets. Furthermore, we consider the application of strong-augmentations to neural networks theoretically and justify the use of a multi-sampling approach for semi-supervised learning. We demonstrate, through rigorous experimentation, that a multi-sampling augmentation approach improves generalisation and reduces the sensitivity of the network to augmentation.