Evaluating and Improving Interactions with Hazy Oracles

TL;DR

This paper introduces a formal framework and evaluation metric for deferred inference in AI systems that interact with hazy or ambiguous human inputs, demonstrating significant error reduction across multiple tasks.

Contribution

It formalizes the concept of deferred inference, proposes the DEV metric for evaluation, and applies these to improve AI performance without altering models.

Findings

Error reduction of up to 48% in tested tasks

DEV metric balances error reduction and human effort

Applicable across diverse AI tasks

Abstract

Many AI systems integrate sensor inputs, world knowledge, and human-provided information to perform inference. While such systems often treat the human input as flawless, humans are better thought of as hazy oracles whose input may be ambiguous or outside of the AI system's understanding. In such situations it makes sense for the AI system to defer its inference while it disambiguates the human-provided information by, for example, asking the human to rephrase the query. Though this approach has been considered in the past, current work is typically limited to application-specific methods and non-standardized human experiments. We instead introduce and formalize a general notion of deferred inference. Using this formulation, we then propose a novel evaluation centered around the Deferred Error Volume (DEV) metric, which explicitly considers the tradeoff between error reduction and the additional human effort required to achieve it. We demonstrate this new formalization and an innovative deferred inference method on the disparate tasks of Single-Target Video Object Tracking and Referring Expression Comprehension, ultimately reducing error by up to 48% without any change to the underlying model or its parameters.