Enhancing LLM Agent Safety via Causal Influence Prompting

TL;DR

This paper presents CIP, a novel method using causal influence diagrams to improve the safety of LLM-based autonomous agents by anticipating and mitigating harmful outcomes.

Contribution

Introducing CIP, a new approach that employs causal influence diagrams to enhance safety in LLM agents through structured decision-making and iterative refinement.

Findings

Effective safety improvements in code execution tasks

Enhanced safety in mobile device control tasks

Causal influence diagrams enable better risk mitigation

Abstract

As autonomous agents powered by large language models (LLMs) continue to demonstrate potential across various assistive tasks, ensuring their safe and reliable behavior is crucial for preventing unintended consequences. In this work, we introduce CIP, a novel technique that leverages causal influence diagrams (CIDs) to identify and mitigate risks arising from agent decision-making. CIDs provide a structured representation of cause-and-effect relationships, enabling agents to anticipate harmful outcomes and make safer decisions. Our approach consists of three key steps: (1) initializing a CID based on task specifications to outline the decision-making process, (2) guiding agent interactions with the environment using the CID, and (3) iteratively refining the CID based on observed behaviors and outcomes. Experimental results demonstrate that our method effectively enhances safety in both code execution and mobile device control tasks.