ECHO: An Open Research Platform for Evaluation of Chat, Human Behavior, and Outcomes

TL;DR

ECHO is an open platform that facilitates reproducible, mixed-method research on human interactions with conversational AI and search engines, integrating various experimental components within a unified framework.

Contribution

It introduces a low-coding, flexible platform supporting end-to-end human-AI interaction studies with detailed logging and structured data export capabilities.

Findings

Supports diverse experimental workflows including surveys, chat, and search sessions.

Enables scalable, reproducible human-centered AI evaluations across disciplines.

Reduces technical barriers for complex human-AI interaction research.

Abstract

ECHO (Evaluation of Chat, Human behavior, and Outcomes) is an open research platform designed to support reproducible, mixed-method studies of human interaction with both conversational AI systems and Web search engines. It enables researchers from varying disciplines to orchestrate end-to-end experimental workflows that integrate consent and background surveys, chat-based and search-based information-seeking sessions, writing or judgment tasks, and pre- and post-task evaluations within a unified, low-coding-load framework. ECHO logs fine-grained interaction traces and participant responses, and exports structured datasets for downstream analysis. By supporting both chat and search alongside flexible evaluation instruments, ECHO lowers technical barriers for studying learning, decision making, and user experience across different information access paradigms, empowering researchers from information retrieval, HCI, and the social sciences to conduct scalable and reproducible human-centered AI evaluations.