Large language models require a new form of oversight: capability-based monitoring

TL;DR

This paper introduces a new capability-based monitoring framework for large language models in healthcare, focusing on shared capabilities rather than task-specific performance to improve oversight and safety.

Contribution

It proposes a scalable, capability-centered approach for monitoring LLMs, addressing limitations of traditional task-based oversight in dynamic healthcare environments.

Findings

Capability-based monitoring enables detection of systemic weaknesses.

It allows cross-task identification of long-tail errors.

Supports safe, adaptive oversight of generalist AI models.

Abstract

The rapid adoption of large language models (LLMs) in healthcare has been accompanied by scrutiny of their oversight. Existing monitoring approaches, inherited from traditional machine learning (ML), are task-based and founded on assumed performance degradation arising from dataset drift. In contrast, with LLMs, inevitable model degradation due to changes in populations compared to the training dataset cannot be assumed, because LLMs were not trained for any specific task in any given population. We therefore propose a new organizing principle guiding generalist LLM monitoring that is scalable and grounded in how these models are developed and used in practice: capability-based monitoring. Capability-based monitoring is motivated by the fact that LLMs are generalist systems whose overlapping internal capabilities are reused across numerous downstream tasks. Instead of evaluating each downstream task independently, this approach organizes monitoring around shared model capabilities, such as summarization, reasoning, translation, or safety guardrails, in order to enable cross-task detection of systemic weaknesses, long-tail errors, and emergent behaviors that task-based monitoring may miss. We describe considerations for developers, organizational leaders, and professional societies for implementing a capability-based monitoring approach. Ultimately, capability-based monitoring will provide a scalable foundation for safe, adaptive, and collaborative monitoring of LLMs and future generalist artificial intelligence models in healthcare.