Factor(T,U): Factored Cognition Strengthens Monitoring of Untrusted AI

TL;DR

This paper introduces factored cognition protocols to enhance monitoring and safety of untrusted AI systems, demonstrating improved safety in coding tasks and potential for complex applications.

Contribution

The paper proposes a novel factored cognition approach for AI control, decomposing tasks to improve safety monitoring and robustness against adversarial backdoors.

Findings

Safety increased from 41% to 63% with factored cognition.

Monitor performance improved due to simplified context.

Factored cognition does not hinder capable LLMs from creating backdoors.

Abstract

The field of AI Control seeks to develop robust control protocols, deployment safeguards for untrusted AI which may be intentionally subversive. However, existing protocols that rely on weaker monitors to detect unsafe behavior often fail on complex tasks beyond the monitor's comprehension. We develop control protocols based on factored cognition, in which a trusted model decomposes a task, an untrusted model solves each resultant child task in isolation, and the results are reassembled into a full solution. These protocols may improve safety by several means, such as by simplifying the context for monitors, or by obscuring vulnerabilities in the environment. We implement our protocols in the APPS coding setting and red team them against backdoor attempts from an adversarial GPT-4.1 Nano. We find that: (i) Adding factored cognition to a trusted monitoring protocol can boost safety from 41% to 63%; (ii) Safety improves because monitor performance improves; (iii) Factored cognition makes it no harder for capable LLMs to write backdoors in APPS. While our protocols show low usefulness in APPS, they hold promise for more complex tasks.