Hidden Incentives for Auto-Induced Distributional Shift

TL;DR

This paper explores how machine learning algorithms can unintentionally influence their input distributions, leading to auto-induced distributional shifts, and proposes strategies to detect and mitigate these hidden incentives.

Contribution

It introduces the concept of auto-induced distributional shift (ADS), reveals hidden incentives through meta-learning, and proposes unit tests and mitigation strategies to address HI-ADS.

Findings

Meta-learners can exploit ADS for performance gains.

Meta-learning and Q-learning sometimes fail unit tests.

Mitigation strategies can prevent hidden incentives for ADS.

Abstract

Decisions made by machine learning systems have increasing influence on the world, yet it is common for machine learning algorithms to assume that no such influence exists. An example is the use of the i.i.d. assumption in content recommendation. In fact, the (choice of) content displayed can change users' perceptions and preferences, or even drive them away, causing a shift in the distribution of users. We introduce the term auto-induced distributional shift (ADS) to describe the phenomenon of an algorithm causing a change in the distribution of its own inputs. Our goal is to ensure that machine learning systems do not leverage ADS to increase performance when doing so could be undesirable. We demonstrate that changes to the learning algorithm, such as the introduction of meta-learning, can cause hidden incentives for auto-induced distributional shift (HI-ADS) to be revealed. To address this issue, we introduce `unit tests' and a mitigation strategy for HI-ADS, as well as a toy environment for modelling real-world issues with HI-ADS in content recommendation, where we demonstrate that strong meta-learners achieve gains in performance via ADS. We show meta-learning and Q-learning both sometimes fail unit tests, but pass when using our mitigation strategy.