On the Pros and Cons of Active Learning for Moral Preference Elicitation

TL;DR

This paper critically examines the effectiveness of active learning for moral preference elicitation, highlighting its limitations under common assumptions and suggesting cautious application due to potential violations in moral judgment contexts.

Contribution

It identifies key assumptions underlying active learning in moral preference elicitation and demonstrates their potential violations through simulations, providing nuanced insights into its practical viability.

Findings

Active learning can perform worse than random queries when assumptions are violated.

Performance depends on stability, noise level, and hypothesis class suitability.

Cautious use of active learning is recommended for moral preference elicitation.

Abstract

Computational preference elicitation methods are tools used to learn people's preferences quantitatively in a given context. Recent works on preference elicitation advocate for active learning as an efficient method to iteratively construct queries (framed as comparisons between context-specific cases) that are likely to be most informative about an agent's underlying preferences. In this work, we argue that the use of active learning for moral preference elicitation relies on certain assumptions about the underlying moral preferences, which can be violated in practice. Specifically, we highlight the following common assumptions (a) preferences are stable over time and not sensitive to the sequence of presented queries, (b) the appropriate hypothesis class is chosen to model moral preferences, and (c) noise in the agent's responses is limited. While these assumptions can be appropriate for preference elicitation in certain domains, prior research on moral psychology suggests they may not be valid for moral judgments. Through a synthetic simulation of preferences that violate the above assumptions, we observe that active learning can have similar or worse performance than a basic random query selection method in certain settings. Yet, simulation results also demonstrate that active learning can still be viable if the degree of instability or noise is relatively small and when the agent's preferences can be approximately represented with the hypothesis class used for learning. Our study highlights the nuances associated with effective moral preference elicitation in practice and advocates for the cautious use of active learning as a methodology to learn moral preferences.