AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitor

TL;DR

AutoMonitor-Bench is a comprehensive benchmark for assessing the reliability of LLM-based misbehavior monitors across various tasks, revealing performance variability and the challenges in achieving dependable monitoring.

Contribution

The paper introduces AutoMonitor-Bench, the first systematic benchmark for evaluating LLM misbehavior monitors, including a large annotated dataset and analysis of performance trade-offs.

Findings

Significant variability in monitor performance across models.

Trade-off observed between Miss Rate and False Alarm Rate.

Training on easy misbehavior datasets does not fully improve detection of complex misbehaviors.

Abstract

We introduce AutoMonitor-Bench, the first benchmark designed to systematically evaluate the reliability of LLM-based misbehavior monitors across diverse tasks and failure modes. AutoMonitor-Bench consists of 3,010 carefully annotated test samples spanning question answering, code generation, and reasoning, with paired misbehavior and benign instances. We evaluate monitors using two complementary metrics: Miss Rate (MR) and False Alarm Rate (FAR), capturing failures to detect misbehavior and oversensitivity to benign behavior, respectively. Evaluating 12 proprietary and 10 open-source LLMs, we observe substantial variability in monitoring performance and a consistent trade-off between MR and FAR, revealing an inherent safety-utility tension. To further explore the limits of monitor reliability, we construct a large-scale training corpus of 153,581 samples and fine-tune Qwen3-4B-Instruction to investigate whether training on known, relatively easy-to-construct misbehavior datasets improves monitoring performance on unseen and more implicit misbehaviors. Our results highlight the challenges of reliable, scalable misbehavior monitoring and motivate future work on task-aware designing and training strategies for LLM-based monitors.