Dimensions of Disagreement: Unpacking Divergence and Misalignment in Cognitive Science and Artificial Intelligence

TL;DR

This paper explores the different dimensions of disagreement between humans and artificial agents, emphasizing the importance of understanding representational differences for improving collaboration and alignment.

Contribution

It introduces a framework for analyzing divergence and misalignment in cognitive and AI systems, highlighting the role of representational differences in disagreements.

Findings

Disagreement can stem from evaluative divergence or representational misalignment.

Tools exist to quantify representational overlap between agents.

Understanding these interactions aids in developing better resolution strategies.

Abstract

The increasing prevalence of artificial agents creates a correspondingly increasing need to manage disagreements between humans and artificial agents, as well as between artificial agents themselves. Considering this larger space of possible agents exposes an opportunity for furthering our understanding of the nature of disagreement: past studies in psychology have often cast disagreement as two agents forming diverging evaluations of the same object, but disagreement can also arise from differences in how agents represent that object. AI research on human-machine alignment and recent work in computational cognitive science have focused on this latter kind of disagreement, and have developed tools that can be used to quantify the extent of representational overlap between agents. Understanding how divergence and misalignment interact to produce disagreement, and how resolution strategies depend on this interaction, is key to promoting effective collaboration between diverse types of agents.