DiSCoKit: An Open-Source Toolkit for Deploying Live LLM Experiences in Survey Research

TL;DR

DiSCoKit is an open-source toolkit designed to facilitate the deployment of live large-language model experiences in online survey research, addressing technical challenges and enabling experimental manipulation of AI behaviors.

Contribution

It introduces a novel toolkit that simplifies integrating live LLMs into survey platforms, expanding research capabilities in human-AI interaction studies.

Findings

Enables real-time deployment of LLMs in surveys

Supports manipulation of AI behaviors for experiments

Addresses technical barriers in survey-based AI research

Abstract

Advancing social-scientific research of human-AI interaction dynamics and outcomes often requires researchers to deliver experiences with live large-language models (LLMs) to participants through online survey platforms. However, technical and practical challenges (from logging chat data to manipulating AI behaviors for experimental designs) often inhibit survey-based deployment of AI stimuli. We developed DiSCoKit--an open-source toolkit for deploying live LLM experiences (e.g., ones based on models delivered through Microsoft Azure portal) through JavaScript-enabled survey platforms (e.g., Qualtrics). This paper introduces that toolkit, explaining its scientific impetus, describes its architecture and operation, as well as its deployment possibilities and limitations.