Facilitating the Integration of LLMs Into Online Experiments With Simple Chat

TL;DR

Simple Chat is an open-source tool that simplifies integrating large language models into online experiments, enabling ecologically valid, real-time human-LLM interaction studies with improved control and participant experience.

Contribution

It introduces a unified, easy-to-use chat interface that connects various LLM providers to online survey platforms, streamlining experimental setup and execution.

Findings

Facilitates real-time, conversational LLM interactions in online experiments.

Reduces technical barriers for researchers integrating LLMs.

Demonstrates utility through two case studies.

Abstract

As large language models (LLMs) become increasingly prevalent, understanding human-LLM interactions is emerging as a central priority in psychological research. Online experiments offer an efficient means to study human-LLM interactions, yet integrating LLMs into established survey platforms remains technically demanding, particularly when aiming for ecologically valid, real-time conversational experiences with strong experimental control. We introduce Simple Chat, an open-source, research-focused chat interface that streamlines LLM integration for platforms such as Qualtrics, oTree, and LimeSurvey, while presenting a unified participant experience across conditions. Simple Chat connects to both commercial providers and open-weights models, supports streaming responses to preserve conversational flow, and offers an administrative interface for fine-grained control of prompts and interface features. By reducing technical barriers, standardizing interfaces, and improving participant experience, Simple Chat helps advance the study of human-LLM interaction. In this article, we outline Simple Chat's key features, provide a step-by-step tutorial, and demonstrate its utility through two illustrative case studies.