Tangent-Normal Adversarial Regularization for Semi-supervised Learning

TL;DR

This paper introduces tangent-normal adversarial regularization (TNAR), enhancing semi-supervised learning by leveraging data manifold geometry through tangent and normal adversarial training, leading to improved classifier robustness and accuracy.

Contribution

The paper proposes TNAR, combining tangent and normal adversarial regularization, extending VAT by incorporating data manifold structure for better semi-supervised learning performance.

Findings

TNAR outperforms existing semi-supervised methods on artificial datasets.

TAR enforces local invariance along the data manifold.

NAR enhances robustness against noise orthogonal to the manifold.

Abstract

Compared with standard supervised learning, the key difficulty in semi-supervised learning is how to make full use of the unlabeled data. A recently proposed method, virtual adversarial training (VAT), smartly performs adversarial training without label information to impose a local smoothness on the classifier, which is especially beneficial to semi-supervised learning. In this work, we propose tangent-normal adversarial regularization (TNAR) as an extension of VAT by taking the data manifold into consideration. The proposed TNAR is composed by two complementary parts, the tangent adversarial regularization (TAR) and the normal adversarial regularization (NAR). In TAR, VAT is applied along the tangent space of the data manifold, aiming to enforce local invariance of the classifier on the manifold, while in NAR, VAT is performed on the normal space orthogonal to the tangent space, intending to impose robustness on the classifier against the noise causing the observed data deviating from the underlying data manifold. Demonstrated by experiments on both artificial and practical datasets, our proposed TAR and NAR complement with each other, and jointly outperforms other state-of-the-art methods for semi-supervised learning.