Hidden Complexities in the Computational Modeling of Proportionality for Robotic Norm Violation Response

TL;DR

This paper introduces a mathematical model for how robots should balance moral and social considerations when responding to norm violations, highlighting the complexity of modeling proportionality in human-robot interactions.

Contribution

The paper presents a naive mathematical model for proportionality in norm violation responses and discusses the complexities involved in developing morally and socially competent robots.

Findings

Model explains balancing moral and social considerations

Highlights hidden complexities in proportionality modeling

Identifies key research directions for robot social competence

Abstract

Language-capable robots hold unique persuasive power over humans, and thus can help regulate people's behavior and preserve a better moral ecosystem, by rejecting unethical commands and calling out norm violations. However, miscalibrated norm violation responses (when the harshness of a response does not match the actual norm violation severity) may not only decrease the effectiveness of human-robot communication, but may also damage the rapport between humans and robots. Therefore, when robots respond to norm violations, it is crucial that they consider both the moral value of their response (by considering how much positive moral influence their response could exert) and the social value (by considering how much face threat might be imposed by their utterance). In this paper, we present a simple (naive) mathematical model of proportionality which could explain how moral and social considerations should be balanced in multi-agent norm violation response generation. But even more importantly, we use this model to start a discussion about the hidden complexity of modeling proportionality, and use this discussion to identify key research directions that must be explored in order to develop socially and morally competent language-capable robots.