Are Akpans Trick or Treat: Unveiling Helpful Biases in Assistant Systems

TL;DR

This paper investigates the helpfulness and fairness of AI assistant dialogue systems, revealing biases favoring questions about highly-developed countries and proposing computational measures for helpfulness evaluation.

Contribution

It introduces methods for automatic helpfulness assessment and highlights fairness issues in current AI assistants through empirical analysis.

Findings

Existing systems favor questions about highly-developed countries.

Biases in helpfulness can lead to fairness concerns.

Proposed models enable automated helpfulness evaluation.

Abstract

Information-seeking AI assistant systems aim to answer users' queries about knowledge in a timely manner. However, both the human-perceived helpfulness of information-seeking assistant systems and its fairness implication are under-explored. In this paper, we study computational measurements of helpfulness. We collect human annotations on the helpfulness of dialogue responses, develop models for automatic helpfulness evaluation, and then propose to use the helpfulness level of a dialogue system towards different user queries to gauge the fairness of a dialogue system. Experiments with state-of-the-art dialogue systems, including ChatGPT, under three information-seeking scenarios reveal that existing systems tend to be more helpful for questions regarding concepts from highly-developed countries than less-developed countries, uncovering potential fairness concerns underlying the current information-seeking assistant systems.