Requirements for Aligned, Dynamic Resolution of Conflicts in Operational Constraints

TL;DR

This paper explores how autonomous AI systems can dynamically resolve conflicts in operational constraints by integrating normative, pragmatic, and situational knowledge to align actions with human values in complex environments.

Contribution

It characterizes the knowledge requirements for decision making in conflict resolution and provides empirical case studies illustrating how agents can achieve better alignment.

Findings

Agents need to incorporate normative and situational understanding.

Dynamic conflict resolution improves alignment with human values.

Empirical case studies demonstrate practical decision-making approaches.

Abstract

Deployed, autonomous AI systems must often evaluate multiple plausible courses of action (extended sequences of behavior) in novel or under-specified contexts. Despite extensive training, these systems will inevitably encounter scenarios where no available course of action fully satisfies all operational constraints (e.g., operating procedures, rules, laws, norms, and goals). To achieve goals in accordance with human expectations and values, agents must go beyond their trained policies and instead construct, evaluate, and justify candidate courses of action. These processes require contextual "knowledge" that may lie outside prior (policy) training. This paper characterizes requirements for agent decision making in these contexts. It also identifies the types of knowledge agents require to make decisions robust to agent goals and aligned with human expectations. Drawing on both analysis and empirical case studies, we examine how agents need to integrate normative, pragmatic, and situational understanding to select and then to pursue more aligned courses of action in complex, real-world environments.