Learning to Pivot with Adversarial Networks

TL;DR

This paper introduces an adversarial network-based training method to enforce the pivotal property in models, improving robustness against systematic uncertainties in data, with applications demonstrated in toy and particle physics examples.

Contribution

It proposes a novel adversarial training procedure to enforce the pivotal property, extending domain adaptation techniques to continuous nuisance parameters.

Findings

Effective in toy example for robust inference

Demonstrated success in particle physics scenarios

Includes a hyperparameter for accuracy-robustness trade-off

Abstract

Several techniques for domain adaptation have been proposed to account for differences in the distribution of the data used for training and testing. The majority of this work focuses on a binary domain label. Similar problems occur in a scientific context where there may be a continuous family of plausible data generation processes associated to the presence of systematic uncertainties. Robust inference is possible if it is based on a pivot -- a quantity whose distribution does not depend on the unknown values of the nuisance parameters that parametrize this family of data generation processes. In this work, we introduce and derive theoretical results for a training procedure based on adversarial networks for enforcing the pivotal property (or, equivalently, fairness with respect to continuous attributes) on a predictive model. The method includes a hyperparameter to control the trade-off between accuracy and robustness. We demonstrate the effectiveness of this approach with a toy example and examples from particle physics.