Using Counterfactuals in Knowledge-Based Programming

TL;DR

This paper extends knowledge-based programming frameworks by incorporating counterfactual reasoning, enabling more nuanced protocols where agents act based on hypothetical scenarios, which was difficult in previous models.

Contribution

It introduces a formal method to include counterfactuals in knowledge-based programs, allowing for more realistic modeling of agent behavior involving hypothetical considerations.

Findings

Counterfactuals enable modeling of agents' hypothetical reasoning.

Protocols can specify agents' actions based on what would happen under different scenarios.

Formalization avoids counterintuitive behaviors seen in previous approaches.

Abstract

This paper adds counterfactuals to the framework of knowledge-based programs of Fagin, Halpern, Moses, and Vardi. The use of counterfactuals is illustrated by designing a protocol in which an agent stops sending messages once it knows that it is safe to do so. Such behavior is difficult to capture in the original framework because it involves reasoning about counterfactual executions, including ones that are not consistent with the protocol. Attempts to formalize these notions without counterfactuals are shown to lead to rather counterintuitive behavior.