Risk-Sensitive Agent Compositions
Guruprerana Shabadi, Rajeev Alur
TL;DR
This paper formalizes agentic workflows as directed acyclic graphs and introduces an efficient algorithm to select agent compositions that minimize risk of safety and fairness violations, with proven near-optimality and practical evaluation.
Contribution
It presents a novel formalization of agent compositions as graphs and develops a dynamic programming algorithm to minimize risk measures like value-at-risk.
Findings
Algorithm effectively approximates value-at-risk in agent compositions.
Proven near-optimality of the risk approximation for practical loss functions.
Successful application to video game-like control benchmarks.
Abstract
From software development to robot control, modern agentic systems decompose complex objectives into a sequence of subtasks and choose a set of specialized AI agents to complete them. We formalize agentic workflows as directed acyclic graphs, called agent graphs, where edges represent AI agents and paths correspond to feasible compositions of agents. Real-world deployment requires selecting agent compositions that not only maximize task success but also minimize violations of safety, fairness, and privacy requirements which demands a careful analysis of the low-probability (tail) behaviors of compositions of agents. In this work, we consider risk minimization over the set of feasible agent compositions and seek to minimize the value-at-risk and the conditional value-at-risk of the loss distribution of the agent composition where the loss quantifies violations of these requirements. We…
Peer Reviews
Decision·ICLR 2026 Poster
1. Clear formulation & motivation: Risk of max loss is appropriate for safety/privacy violations in composed systems. 2. Simple, scalable idea: Union-bound budgeting with DP avoids path enumeration; complexity $O\left(n(d+1)^2|V|^2\right)$. 3. Theory with interpretable slack: Asymptotically, the selected path is within an $\alpha^2 / 2$ quantile-level slack under independence (Thm. 2) 4. Empirical evidence: Tight coverage across benchmarks; agreement with exhaustive baseline; clear sensitivit
1. Choice of VaR over CVaR: VaR ignores tail severity; CVaR is often preferred for safety. The paper mentions CVaR as future work but gives no partial result or empirical check. 2. Missing baselines and ablations. No comparison to chance-constrained shortest path or to dependence-aware relaxations. No experiments that explicitly vary tail dependence (e.g., shared noise seeds, correlated disturbances) to show robustness/failure modes. 3. Finite-sample conservativeness is unclear. Thm. 1 shows t
- This work connects risk-sensitive optimization and compositional agent systems using directed acyclic graphs. The presented BucketedVaR algorithm is polynomial in the number of agents, thereby avoiding exponential enumeration (as seen in baselines). - The topic presented is relevant and has significance for measuring risk sensitivity in safety-critical AI agent composition. The experiments in various RL benchmarks demonstrate that BucketedVaR successfully identifies the same optimal path as t
- In real AI agent chains, losses are often correlated via shared context or sequential dependence; the paper should analyze or empirically test how violation of independence could affect performance. - Since the work only tested on RL benchmarks but described LLM examples as potential use cases, missing validation on LLM agentic pipelines where sampling cost and judge noise matter is concerning. For LLMs, losses (e.g., amount of hallucinated information) may be subjective, noisy, or non-numer
1. The paper presents a technically sound and mathematically coherent formulation of risk minimization for agent compositions. 2. The theoretical development is detailed and carefully reasoned, with clear proofs supporting the approximation guarantees. 3. Overall, the work provides both a mathematical framework and an algorithmic contribution that meaningfully advance the study of risk-aware agent systems.
1. The paper’s presentation is quite dense, and the abstract and introduction do not clearly communicate the main ideas, which makes it difficult to identify the core contribution without considerable effort. 2. The approach also depends on several strong assumptions, such as the DAG workflow structure and the independence of agent losses, which limit its generality and practical relevance. 3. Finally, the work remains mostly theoretical, and it is not clear how the proposed framework could
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Ethics and Social Impacts of AI · Artificial Intelligence in Games
MethodsSparse Evolutionary Training