UX in the Age of AI: Rethinking Evaluation Metrics Through a Statistical Lens

TL;DR

This paper proposes ADUX-Stat, a new probabilistic framework for evaluating AI-driven user experiences, addressing the limitations of traditional metrics in stochastic, context-sensitive AI systems.

Contribution

It introduces three novel constructs—IEI, TDC, and BUCS—that enable a more accurate, dynamic, and uncertainty-aware assessment of UX in AI products.

Findings

Validated conceptually across five AI product categories.

Addresses the gap between classical UX metrics and AI system variability.

Provides a reproducible methodology for UX evaluation in AI contexts.

Abstract

The rapid proliferation of artificial intelligence (AI) in consumer-facing digital products has disrupted the assumptions underlying classical user experience (UX) evaluation frameworks. Legacy metrics such as the System Usability Scale (SUS), Net Promoter Score (NPS), and task completion rate were engineered for deterministic, rule-based interfaces where identical inputs yield identical outputs. In AI-mediated systems -- spanning conversational agents, generative interfaces, and recommendation engines -- outputs are stochastic, context-sensitive, and temporally variable, rendering these metrics structurally insufficient. This paper introduces the Adaptive Dynamic UX Statistical Framework (ADUX-Stat), a novel evaluation model that reconceptualises usability as a probabilistic signal distribution rather than a static scalar score. ADUX-Stat integrates three original constructs: (1) Interaction Entropy Index (IEI), quantifying the unpredictability of AI responses from a user perception standpoint; (2) Temporal Drift Coefficient (TDC), measuring longitudinal degradation or improvement of perceived usability over interaction sessions; and (3) Bayesian Usability Confidence Score (BUCS), producing credible interval estimates of usability quality under uncertainty. The framework is validated conceptually against five established AI product categories. ADUX-Stat addresses a critical gap at the intersection of HCI research, statistical modelling, and AI product evaluation, offering a reproducible, field-deployable methodology for UX practitioners and researchers alike.