Analysis of hidden feedback loops in continuous machine learning systems

TL;DR

This paper explores the complexities of feedback loops in continuous AI systems, illustrating how they influence environments and user behavior, and discusses potential solutions to manage these effects.

Contribution

It identifies and analyzes implicit feedback loops in lifelong learning AI systems, providing a preliminary model and discussing conditions and solutions for these phenomena.

Findings

Feedback loops can significantly alter system behavior and environment.

Conditions for feedback loop emergence are identified.

Potential approaches to mitigate feedback effects are discussed.

Abstract

In this concept paper, we discuss intricacies of specifying and verifying the quality of continuous and lifelong learning artificial intelligence systems as they interact with and influence their environment causing a so-called concept drift. We signify a problem of implicit feedback loops, demonstrate how they intervene with user behavior on an exemplary housing prices prediction system. Based on a preliminary model, we highlight conditions when such feedback loops arise and discuss possible solution approaches.