Toward Constraint Compliant Goal Formulation and Planning

TL;DR

This paper investigates how different ethical frameworks influence goal formulation and planning in agents, demonstrating the impact of constraint encoding, framing, and metacognitive judgments on ethical behavior.

Contribution

It introduces a systematic exploration of encoding ethical constraints in goal planning, comparing deontological and utilitarian framings, and highlights the role of metacognition in ethical decision-making.

Findings

Different ethical framings lead to distinct agent behaviors.

Agents can satisfy and satisfice with mixed hard and soft constraints.

Metacognitive judgments are crucial for resolving ethical conflicts.

Abstract

One part of complying with norms, rules, and preferences is incorporating constraints (such as knowledge of ethics) into one's goal formulation and planning processing. We explore in a simple domain how the encoding of knowledge in different ethical frameworks influences an agent's goal formulation and planning processing and demonstrate ability of an agent to satisfy and satisfice when its collection of relevant constraints includes a mix of "hard" and "soft" constraints of various types. How the agent attempts to comply with ethical constraints depends on the ethical framing and we investigate tradeoffs between deontological framing and utilitarian framing for complying with an ethical norm. Representative scenarios highlight how performing the same task with different framings of the same norm leads to different behaviors. Our explorations suggest an important role for metacognitive judgments in resolving ethical conflicts during goal formulation and planning.