Answer Only as Precisely as Justified: Calibrated Claim-Level Specificity Control for Agentic Systems

TL;DR

This paper introduces compositional selective specificity (CSS), a post-generation method that calibrates claim-level specificity in agentic systems to better express uncertainty and improve response quality.

Contribution

It proposes CSS as a novel post-generation layer that decomposes answers into claims and calibrates their specificity to control overcommitment.

Findings

CSS improves the risk-utility trade-off in answer generation.

CSS raises overcommitment-aware utility from 0.846 to 0.913.

CSS achieves 0.938 specificity retention.

Abstract

Agentic systems often fail not by being entirely wrong, but by being too precise: a response may be generally useful while particular claims exceed what the evidence supports. We study this failure mode as overcommitment control and introduce compositional selective specificity (CSS), a post-generation layer that decomposes an answer into claims, proposes coarser backoffs, and emits each claim at the most specific calibrated level that appears admissible. The method is designed to express uncertainty as a local semantic backoff rather than as a whole-answer refusal. Across a full LongFact run and HotpotQA pilots, calibrated CSS improves the risk-utility trade-off of fixed drafts. On the full LongFact run, it raises overcommitment-aware utility from 0.846 to 0.913 relative to the no-CSS output while achieving 0.938 specificity retention. These results suggest that claim-level specificity control is a useful uncertainty interface for agentic systems and a target for future distribution-free validity layers.