Noisy Node Classification by Bi-level Optimization based Multi-teacher Distillation

TL;DR

This paper introduces BO-NNC, a novel multi-teacher distillation approach using bi-level optimization to enhance noisy node classification in graph neural networks, outperforming existing methods.

Contribution

The paper presents a new bi-level optimization-based multi-teacher distillation framework for noisy graph node classification, incorporating dynamic teacher weighting and label quality improvement.

Findings

Achieves superior accuracy on real datasets compared to state-of-the-art methods.

Effectively handles noisy labels in graph neural network training.

Demonstrates robustness and adaptability through extensive experiments.

Abstract

Previous graph neural networks (GNNs) usually assume that the graph data is with clean labels for representation learning, but it is not true in real applications. In this paper, we propose a new multi-teacher distillation method based on bi-level optimization (namely BO-NNC), to conduct noisy node classification on the graph data. Specifically, we first employ multiple self-supervised learning methods to train diverse teacher models, and then aggregate their predictions through a teacher weight matrix. Furthermore, we design a new bi-level optimization strategy to dynamically adjust the teacher weight matrix based on the training progress of the student model. Finally, we design a label improvement module to improve the label quality. Extensive experimental results on real datasets show that our method achieves the best results compared to state-of-the-art methods.