Patch-level Neighborhood Interpolation: A General and Effective Graph-based Regularization Strategy

TL;DR

This paper introduces Patch-level Neighborhood Interpolation (Pani), a graph-based regularization strategy that enhances neural network training by leveraging non-local patch relationships, improving performance and efficiency.

Contribution

It proposes a novel regularizer that constructs patch-level graphs for non-local interpolation, and adapts it into VAT and MixUp, demonstrating improved effectiveness and efficiency.

Findings

Pani improves regularization in supervised and semi-supervised learning.

Pani VAT introduces non-local adversarial smoothness.

Pani MixUp outperforms traditional MixUp methods.

Abstract

Regularization plays a crucial role in machine learning models, especially for deep neural networks. The existing regularization techniques mainly rely on the i.i.d. assumption and only consider the knowledge from the current sample, without the leverage of the neighboring relationship between samples. In this work, we propose a general regularizer called \textbf{Patch-level Neighborhood Interpolation~(Pani)} that conducts a non-local representation in the computation of networks. Our proposal explicitly constructs patch-level graphs in different layers and then linearly interpolates neighborhood patch features, serving as a general and effective regularization strategy. Further, we customize our approach into two kinds of popular regularization methods, namely Virtual Adversarial Training (VAT) and MixUp as well as its variants. The first derived \textbf{Pani VAT} presents a novel way to construct non-local adversarial smoothness by employing patch-level interpolated perturbations. The second derived \textbf{Pani MixUp} method extends the MixUp, and achieves superiority over MixUp and competitive performance over state-of-the-art variants of MixUp method with a significant advantage in computational efficiency. Extensive experiments have verified the effectiveness of our Pani approach in both supervised and semi-supervised settings.