Training Lightweight Graph Convolutional Networks with Phase-field Models

TL;DR

This paper introduces a novel method for training lightweight graph convolutional networks using phase-field models, enabling joint topology and weight optimization for improved generalization and pruning.

Contribution

It proposes a phase-field regularizer that allows end-to-end training of GCN topology and weights, enhancing model efficiency and performance.

Findings

Outperforms other regularizers in skeleton-based recognition tasks

Enables effective topology selection with targeted pruning rates

Improves generalization of lightweight GCNs

Abstract

In this paper, we design lightweight graph convolutional networks (GCNs) using a particular class of regularizers, dubbed as phase-field models (PFMs). PFMs exhibit a bi-phase behavior using a particular ultra-local term that allows training both the topology and the weight parameters of GCNs as a part of a single "end-to-end" optimization problem. Our proposed solution also relies on a reparametrization that pushes the mask of the topology towards binary values leading to effective topology selection and high generalization while implementing any targeted pruning rate. Both masks and weights share the same set of latent variables and this further enhances the generalization power of the resulting lightweight GCNs. Extensive experiments conducted on the challenging task of skeleton-based recognition show the outperformance of PFMs against other staple regularizers as well as related lightweight design methods.