WaLLM -- Insights from an LLM-Powered Chatbot deployment via WhatsApp

TL;DR

This paper presents WaLLM, an AI chatbot on WhatsApp designed for developing regions, analyzing user interactions over six months to understand engagement, trust, and information-seeking behaviors.

Contribution

The paper introduces WaLLM, a novel WhatsApp-based AI chatbot tailored for developing regions, and provides a systematic analysis of user interactions and engagement patterns.

Findings

55% of queries seek factual information

Health and well-being is the most popular topic (28%)

Users engaging with the leaderboard interact 3x more

Abstract

Recent advances in generative AI, such as ChatGPT, have transformed access to information in education, knowledge-seeking, and everyday decision-making. However, in many developing regions, access remains a challenge due to the persistent digital divide. To help bridge this gap, we developed WaLLM - a custom AI chatbot over WhatsApp, a widely used communication platform in developing regions. Beyond answering queries, WaLLM offers several features to enhance user engagement: a daily top question, suggested follow-up questions, trending and recent queries, and a leaderboard-based reward system. Our service has been operational for over 6 months, amassing over 14.7K queries from approximately 100 users. In this paper, we present WaLLM's design and a systematic analysis of logs to understand user interactions. Our results show that 55% of user queries seek factual information. "Health and well-being" was the most popular topic (28%), including queries about nutrition and disease, suggesting users view WaLLM as a reliable source. Two-thirds of users' activity occurred within 24 hours of the daily top question. Users who accessed the "Leaderboard" interacted with WaLLM 3x as those who did not. We conclude by discussing implications for culture-based customization, user interface design, and appropriate calibration of users' trust in AI systems for developing regions.