LEXI: Large Language Models Experimentation Interface

TL;DR

LEXI is an open-source interface that facilitates the deployment and testing of large language models in social interaction experiments, addressing current limitations in experimental setups and data collection.

Contribution

The paper introduces LEXI, a novel open-source tool that enables researchers to easily build, deploy, and evaluate LLM-powered social agents in experimental settings.

Findings

LEXI demonstrated high usability and low mental workload in usability tests.

A proof-of-concept study showed empathetic agents are perceived as more social.

People wrote longer and more positive messages to empathetic agents.

Abstract

The recent developments in Large Language Models (LLM), mark a significant moment in the research and development of social interactions with artificial agents. These agents are widely deployed in a variety of settings, with potential impact on users. However, the study of social interactions with agents powered by LLM is still emerging, limited by access to the technology and to data, the absence of standardised interfaces, and challenges to establishing controlled experimental setups using the currently available business-oriented platforms. To answer these gaps, we developed LEXI, LLMs Experimentation Interface, an open-source tool enabling the deployment of artificial agents powered by LLM in social interaction behavioural experiments. Using a graphical interface, LEXI allows researchers to build agents, and deploy them in experimental setups along with forms and questionnaires while collecting interaction logs and self-reported data. The outcomes of usability testing indicate LEXI's broad utility, high usability and minimum mental workload requirement, with distinctive benefits observed across disciplines. A proof-of-concept study exploring the tool's efficacy in evaluating social HAIs was conducted, resulting in high-quality data. A comparison of empathetic versus neutral agents indicated that people perceive empathetic agents as more social, and write longer and more positive messages towards them.