AI Telephone Surveying: Automating Quantitative Data Collection with an AI Interviewer

TL;DR

This paper presents an AI telephone surveying system using large language models, enabling more natural and adaptive interviews for quantitative data collection, and demonstrates its effectiveness through pilot studies.

Contribution

It introduces a novel AI system for telephone surveys that improves respondent experience and data quality compared to traditional IVR methods.

Findings

Higher survey completion rates with AI interviewer

Reduced break-off rates in pilot studies

Improved respondent satisfaction scores

Abstract

With the rise of voice-enabled artificial intelligence (AI) systems, quantitative survey researchers have access to a new data-collection mode: AI telephone surveying. By using AI to conduct phone interviews, researchers can scale quantitative studies while balancing the dual goals of human-like interactivity and methodological rigor. Unlike earlier efforts that used interactive voice response (IVR) technology to automate these surveys, voice AI enables a more natural and adaptive respondent experience as it is more robust to interruptions, corrections, and other idiosyncrasies of human speech. We built and tested an AI system to conduct quantitative surveys based on large language models (LLM), automatic speech recognition (ASR), and speech synthesis technologies. The system was specifically designed for quantitative research, and strictly adhered to research best practices like question order randomization, answer order randomization, and exact wording. To validate the system's effectiveness, we deployed it to conduct two pilot surveys with the SSRS Opinion Panel and followed-up with a separate human-administered survey to assess respondent experiences. We measured three key metrics: the survey completion rates, break-off rates, and respondent satisfaction scores. Our results suggest that shorter instruments and more responsive AI interviewers may contribute to improvements across all three metrics studied.