Learning When to Advise Human Decision Makers

TL;DR

This paper introduces a novel AI advising system that interacts with humans to provide advice only when beneficial, significantly improving decision quality and supporting human learning across various domains.

Contribution

It proposes an interactive advising framework that determines optimal timing for advice, outperforming fixed advising methods in large-scale experiments.

Findings

Improved decision accuracy with adaptive advising

Enhanced human learning and responsiveness

Effective in diverse real-world domains

Abstract

Artificial intelligence (AI) systems are increasingly used for providing advice to facilitate human decision making in a wide range of domains, such as healthcare, criminal justice, and finance. Motivated by limitations of the current practice where algorithmic advice is provided to human users as a constant element in the decision-making pipeline, in this paper we raise the question of when should algorithms provide advice? We propose a novel design of AI systems in which the algorithm interacts with the human user in a two-sided manner and aims to provide advice only when it is likely to be beneficial for the user in making their decision. The results of a large-scale experiment show that our advising approach manages to provide advice at times of need and to significantly improve human decision making compared to fixed, non-interactive, advising approaches. This approach has additional advantages in facilitating human learning, preserving complementary strengths of human decision makers, and leading to more positive responsiveness to the advice.