Could AI Leapfrog the Web? Evidence from Teachers in Sierra Leone

TL;DR

This study shows that AI-powered WhatsApp chatbots can provide cost-effective, relevant, and helpful educational information to teachers in Sierra Leone, surpassing traditional web search in accessibility and efficiency.

Contribution

The paper provides empirical evidence that AI chatbots can outperform web search in low-bandwidth settings, offering a scalable solution for information access in developing regions.

Findings

AI responses are more relevant and helpful than web search results.

Querying AI is 98% cheaper than loading web pages.

More teachers relied on AI than web search over time.

Abstract

Only 37% of sub-Saharan Africans use the internet, and those who do seldom rely on traditional web search. A major reason is that bandwidth is scarce and costly. We study whether an AI-powered WhatsApp chatbot can bridge this gap by analyzing 40,350 queries submitted by 529 Sierra Leonean teachers over 17 months. Each month, more teachers relied on AI than web search for teaching assistance. We compare the AI responses to the top results from google.com.sl, which mostly returns web pages formatted for foreign users: just 2% of pages originate in-country. Also, each web page consumes 3,107 times more bandwidth than an AI response on average. As a result, querying AI through WhatsApp is 98% less expensive than loading a web page, even including AI compute costs. In blinded evaluations, an independent sample of teachers rate AI responses as more relevant, helpful, and correct answers to queries than web search results. These findings suggest that AI can provide cost-effective access to information in low-connectivity environments.