Measuring LLM Trust Allocation Across Conflicting Software Artifacts

TL;DR

This paper introduces TRACE, a framework for evaluating how well LLMs assess trustworthiness across conflicting software artifacts, revealing strengths and weaknesses in their artifact-level trust reasoning.

Contribution

The paper presents TRACE, a novel structured evaluation framework for artifact-level trust assessment in LLMs, highlighting their capabilities and blind spots in software artifact evaluation.

Findings

Models detect explicit documentation bugs with 67-94% accuracy.

Detection of contradictions between Javadoc and implementation ranges from 50-91%.

Models poorly calibrate confidence and struggle with subtle implementation drift.

Abstract

LLM-based software engineering assistants fail not only by producing incorrect outputs, but also by allocating trust to the wrong artifact when code, documentation, and tests disagree. Existing evaluations focus mainly on downstream outcomes and therefore cannot reveal whether a model recognized degraded evidence, identified the unreliable source, or calibrated its trust across artifacts. We present TRACE (Trust Reasoning over Artifacts for Calibrated Evaluation), a framework that elicits structured artifact-level trust traces over Javadoc, method signatures, implementations, and test prefixes under blind perturbations. Using 22,339 valid traces from seven models on 456 curated Java method bundles, we evaluate per-artifact quality assessment, inconsistency detection, affected artifact attribution, and source prioritization. Across all models, quality penalties are largely localized to the perturbed artifact and increase with severity, but sensitivity is asymmetric across artifact types: documentation bugs induce a substantially larger heavy-to-subtle gap than implementation faults (0.152-0.253 vs. 0.049-0.123). Models detect explicit documentation bugs well (67-94%) and Javadoc and implementation contradictions at 50-91%, yet show a systematic blind spot when only the implementation drifts while the documentation remains plausible, with detection dropping by 7-42 percentage points. Confidence is poorly calibrated for six of seven models. These findings suggest that current LLMs are better at auditing natural-language specifications than at detecting subtle code-level drift, motivating explicit artifact-level trust reasoning before correctness-critical downstream use.