Regression Under Human Assistance

TL;DR

This paper develops a novel approach for optimizing ridge regression models to operate effectively under varying levels of human assistance, addressing the challenge of decision-making in hybrid human-AI systems.

Contribution

It introduces the problem of ridge regression under human assistance, proves its NP-hardness, and proposes an efficient greedy algorithm with approximation guarantees based on submodularity properties.

Findings

The algorithm effectively outsources high-error samples to humans.

It outperforms several baseline methods in experiments.

The approach is robust across different settings.

Abstract

Decisions are increasingly taken by both humans and machine learning models. However, machine learning models are currently trained for full automation -- they are not aware that some of the decisions may still be taken by humans. In this paper, we take a first step towards the development of machine learning models that are optimized to operate under different automation levels. More specifically, we first introduce the problem of ridge regression under human assistance and show that it is NP-hard. Then, we derive an alternative representation of the corresponding objective function as a difference of nondecreasing submodular functions. Building on this representation, we further show that the objective is nondecreasing and satisfies $\alpha$-submodularity, a recently introduced notion of approximate submodularity. These properties allow a simple and efficient greedy algorithm to enjoy approximation guarantees at solving the problem. Experiments on synthetic and real-world data from two important applications -- medical diagnosis and content moderation-demonstrate that our algorithm outsources to humans those samples in which the prediction error of the ridge regression model would have been the highest if it had to make a prediction, it outperforms several competitive baselines, and its performance is robust with respect to several design choices and hyperparameters used in the experiments.