Task-Aware Delegation Cues for LLM Agents

TL;DR

This paper introduces a task-aware signaling framework for LLM agents that enhances transparency, trust, and collaboration by providing interpretable cues and adaptive delegation protocols based on task-specific reliability and risk assessments.

Contribution

It presents a novel signaling layer that converts offline preference evaluations into online, user-facing primitives, enabling more transparent and adaptive human-agent collaboration.

Findings

Task taxonomy derived from pairwise comparisons improves interpretability.

Capability Profiles predict win rates and inform delegation decisions.

Coordination-Risk Cues help assess disagreement and guide routing.

Abstract

LLM agents increasingly present as conversational collaborators, yet human--agent teamwork remains brittle due to information asymmetry: users lack task-specific reliability cues, and agents rarely surface calibrated uncertainty or rationale. We propose a task-aware collaboration signaling layer that turns offline preference evaluations into online, user-facing primitives for delegation. Using Chatbot Arena pairwise comparisons, we induce an interpretable task taxonomy via semantic clustering, then derive (i) Capability Profiles as task-conditioned win-rate maps and (ii) Coordination-Risk Cues as task-conditioned disagreement (tie-rate) priors. These signals drive a closed-loop delegation protocol that supports common-ground verification, adaptive routing (primary vs.\ primary+auditor), explicit rationale disclosure, and privacy-preserving accountability logs. Two predictive probes validate that task typing carries actionable structure: cluster features improve winner prediction accuracy and reduce difficulty prediction error under stratified 5-fold cross-validation. Overall, our framework reframes delegation from an opaque system default into a visible, negotiable, and auditable collaborative decision, providing a principled design space for adaptive human--agent collaboration grounded in mutual awareness and shared accountability.