Transitive Expert Error and Routing Problems in Complex AI Systems

TL;DR

This paper identifies systematic vulnerabilities at domain boundaries caused by expert biases and social influences, leading to errors in AI routing systems that produce plausible but causally incorrect outputs, and proposes interventions to mitigate these issues.

Contribution

It introduces Transitive Expert Error (TEE), a novel framework explaining boundary errors in AI systems and suggests architectural interventions for mitigation.

Findings

TEE causes confident, causally incorrect outputs at domain boundaries.

Routing failures can be detected through specific signatures like confidence-accuracy dissociations.

Architectural interventions can address black-box human cognition limitations.

Abstract

Domain expertise enhances judgment within boundaries but creates systematic vulnerabilities specifically at borders. We term this Transitive Expert Error (TEE), distinct from Dunning-Kruger effects, requiring calibrated expertise as precondition. Mechanisms enabling reliable within-domain judgment become liabilities when structural similarity masks causal divergence. Two core mechanisms operate: structural similarity bias causes experts to overweight surface features (shared vocabulary, patterns, formal structure) while missing causal architecture differences; authority persistence maintains confidence across competence boundaries through social reinforcement and metacognitive failures (experts experience no subjective uncertainty as pattern recognition operates smoothly on familiar-seeming inputs.) These mechanism intensify under three conditions: shared vocabulary masking divergent processes, social pressure for immediate judgment, and delayed feedback. These findings extend to AI routing architectures (MoE systems, multi-model orchestration, tool-using agents, RAG systems) exhibiting routing-induced failures (wrong specialist selected) and coverage-induced failures (no appropriate specialist exists). Both produce a hallucination phenotype: confident, coherent, structurally plausible but causally incorrect outputs at domain boundaries. In human systems where mechanisms are cognitive black boxes; AI architectures make them explicit and addressable. We propose interventions: multi-expert activation with disagreement detection (router level), boundary-aware calibration (specialist level), and coverage gap detection (training level). TEE has detectable signatures (routing patterns, confidence-accuracy dissociations, domain-inappropriate content) enabling monitoring and mitigation. What remains intractable in human cognition becomes addressable through architectural design.