Quantifying Trust: Financial Risk Management for Trustworthy AI Agents

TL;DR

This paper introduces the Agentic Risk Standard (ARS), a financial-inspired framework for managing trust and risk in autonomous AI agents, emphasizing end-to-end outcomes and contractual guarantees over internal model properties.

Contribution

It proposes a novel risk management framework for trustworthy AI, integrating assessment, underwriting, and compensation to provide enforceable guarantees for end-user trust.

Findings

ARS enables predefined compensation for failures and misalignments.

Simulation shows ARS improves social benefits and user trust.

Framework shifts trust from model properties to product guarantees.

Abstract

Prior work on trustworthy AI emphasizes model-internal properties such as bias mitigation, adversarial robustness, and interpretability. As AI systems evolve into autonomous agents deployed in open environments and increasingly connected to payments or assets, the operational meaning of trust shifts to end-to-end outcomes: whether an agent completes tasks, follows user intent, and avoids failures that cause material or psychological harm. These risks are fundamentally product-level and cannot be eliminated by technical safeguards alone because agent behavior is inherently stochastic. To address this gap between model-level reliability and user-facing assurance, we propose a complementary framework based on risk management. Drawing inspiration from financial underwriting, we introduce the \textbf{Agentic Risk Standard (ARS)}, a payment settlement standard for AI-mediated transactions. ARS integrates risk assessment, underwriting, and compensation into a single transaction framework that protects users when interacting with agents. Under ARS, users receive predefined and contractually enforceable compensation in cases of execution failure, misalignment, or unintended outcomes. This shifts trust from an implicit expectation about model behavior to an explicit, measurable, and enforceable product guarantee. We also present a simulation study analyzing the social benefits of applying ARS to agentic transactions. ARS's implementation can be found at https://github.com/t54-labs/AgenticRiskStandard.