Trust via Reputation of Conviction

TL;DR

This paper introduces a mathematical framework for trust based on the conviction of sources, emphasizing reputation as a measure of likelihood that claims are vindicated by consensus, applicable to AI agents and beyond.

Contribution

It formalizes conviction as the core of trust, develops a reputation model based on expected conviction, and applies it to AI agents for robust trust assessment.

Findings

Reputation is modeled as expected weighted signed conviction.

Conviction is regime-independent and rewards genuine contribution.

Continuous verification is essential for reputation development.

Abstract

The question of \emph{knowledge}, \emph{truth} and \emph{trust} is explored via a mathematical formulation of claims and sources. We define truth as the reproducibly perceived subset of knowledge, formalize sources as having both generative and discriminative roles, and develop a framework for reputation grounded in the \emph{conviction} -- the likelihood that a source's stance is vindicated by independent consensus. We argue that conviction, rather than correctness or faithfulness, is the principled basis for trust: it is regime-independent, rewards genuine contribution, and demands the transparent and self-sufficient perceptions that make external verification possible. We formalize reputation as the expected weighted signed conviction over a realm of claims, characterize its behavior across source-claim regimes, and identify continuous verification as both a theoretical necessity and a practical mechanism through which reputation accrues. The framework is applied to AI agents, which are identified as capable but error-prone sources for whom verifiable conviction and continuously accrued reputation constitute the only robust foundation for trust.