Evaluating Trust in the Context of Conversational Information Systems for new users of the Internet

TL;DR

This paper investigates how different voice-based chat-bots influence trust among new Internet users with low English proficiency, emphasizing the importance of understanding and response accuracy in building trust.

Contribution

The study develops four personalized voice chat-bots on Google Assistant and evaluates their impact on user trust and preferences among first-time Internet users.

Findings

Users preferred the female, personalized bot.

Trust correlated with the bot's understanding and response accuracy.

Personalization influenced user preferences more than gender or friendliness.

Abstract

Most online information sources are text-based and in Western Languages like English. However, many new and first time users of the Internet are in contexts with low English proficiency and are unable to access vital information online. Several researchers have focused on building conversational information systems over voice for this demographic, and also highlighted the importance of building trust towards the information source. In this work we develop four versions of a voice based chat-bot on the Google Assistant platform in which we vary the gender, friendliness and personalisation of the bot. We find that the users rank the female version of the bot with more personalisations over the others; however when rating the bots individually, the ratings depend on the ability of the bot to understand the users' spoken query and respond accurately.