From SERPs to Agents: A Platform for Comparative Studies of Information Interaction

TL;DR

UXLab is an open-source platform that simplifies the deployment of comparative user studies across diverse information access systems, enabling researchers to analyze user behavior without technical barriers.

Contribution

The paper introduces UXLab, a no-code, web-based system for designing and managing comparative studies of information interaction methods.

Findings

UXLab facilitates easy setup of user studies across different information systems.

A case study demonstrates UXLab's effectiveness in comparing RAG and autonomous agents.

Supports future multi-modal interaction research.

Abstract

The diversification of information access systems, from RAG to autonomous agents, creates a critical need for comparative user studies. However, the technical overhead to deploy and manage these distinct systems is a major barrier. We present UXLab, an open-source system for web-based user studies that addresses this challenge. Its core is a web-based dashboard enabling the complete, no-code configuration of complex experimental designs. Researchers can visually manage the full study, from recruitment to comparing backends like traditional search, vector databases, and LLMs. We demonstrate UXLab's value via a micro case study comparing user behavior with RAG versus an autonomous agent. UXLab allows researchers to focus on experimental design and analysis, supporting future multi-modal interaction research.