Nuisance-Label Supervision: Robustness Improvement by Free Labels

TL;DR

This paper introduces a Nuisance-label Supervision (NLS) module that enhances model robustness to irrelevant nuisance factors by explicitly supervising with nuisance labels, which can be obtained freely through data augmentation and synthetic data, leading to improved performance in real-world scenarios.

Contribution

The paper proposes a novel NLS module that leverages freely available nuisance labels to improve model invariance and robustness against nuisance factors.

Findings

Improved robustness to image corruption.

Enhanced invariance to appearance changes.

Effective use of synthetic data for nuisance labels.

Abstract

In this paper, we present a Nuisance-label Supervision (NLS) module, which can make models more robust to nuisance factor variations. Nuisance factors are those irrelevant to a task, and an ideal model should be invariant to them. For example, an activity recognition model should perform consistently regardless of the change of clothes and background. But our experiments show existing models are far from this capability. So we explicitly supervise a model with nuisance labels to make extracted features less dependent on nuisance factors. Although the values of nuisance factors are rarely annotated, we demonstrate that besides existing annotations, nuisance labels can be acquired freely from data augmentation and synthetic data. Experiments show consistent improvement in robustness towards image corruption and appearance change in action recognition.