LLMartini: Seamless and Interactive Leveraging of Multiple LLMs through Comparison and Composition

TL;DR

LLMartini is an interactive system that simplifies comparing and combining outputs from multiple large language models, reducing user effort and improving response quality through semantic alignment and visual cues.

Contribution

The paper introduces LLMartini, a novel system enabling seamless comparison and composition of multiple LLM outputs with semantic alignment and user-friendly features.

Findings

Significantly reduced task completion time

Lowered cognitive load for users

Increased user satisfaction

Abstract

The growing diversity of large language models (LLMs) means users often need to compare and combine outputs from different models to obtain higher-quality or more comprehensive responses. However, switching between separate interfaces and manually integrating outputs is inherently inefficient, leading to a high cognitive burden and fragmented workflows. To address this, we present LLMartini, a novel interactive system that supports seamless comparison, selection, and intuitive cross-model composition tools. The system decomposes responses into semantically aligned segments based on task-specific criteria, automatically merges consensus content, and highlights model differences through color coding while preserving unique contributions. In a user study (N=18), LLMartini significantly outperformed conventional manual methods across all measured metrics, including task completion time, cognitive load, and user satisfaction. Our work highlights the importance of human-centered design in enhancing the efficiency and creativity of multi-LLM interactions and offers practical implications for leveraging the complementary strengths of various language models.