Trusting Your AI Agent Emotionally and Cognitively: Development and Validation of a Semantic Differential Scale for AI Trust

TL;DR

This paper develops and validates a semantic differential scale to measure both emotional and cognitive trust in AI agents, addressing a gap in trust assessment tools for human-AI interactions.

Contribution

It introduces a validated 27-item scale for affective and cognitive trust in AI, enabling better measurement of trust dimensions in human-AI interaction research.

Findings

Emotional and cognitive trust interact to influence overall trust.

The scale effectively captures trust dimensions in scenarios involving LLM-based AI.

Empirical results show how trust components are shaped by AI capabilities.

Abstract

Trust is not just a cognitive issue but also an emotional one, yet the research in human-AI interactions has primarily focused on the cognitive route of trust development. Recent work has highlighted the importance of studying affective trust towards AI, especially in the context of emerging human-like LLMs-powered conversational agents. However, there is a lack of validated and generalizable measures for the two-dimensional construct of trust in AI agents. To address this gap, we developed and validated a set of 27-item semantic differential scales for affective and cognitive trust through a scenario-based survey study. We then further validated and applied the scale through an experiment study. Our empirical findings showed how the emotional and cognitive aspects of trust interact with each other and collectively shape a person's overall trust in AI agents. Our study methodology and findings also provide insights into the capability of the state-of-art LLMs to foster trust through different routes.