"I Want It That Way": Enabling Interactive Decision Support Using Large Language Models and Constraint Programming

TL;DR

This paper presents a hybrid approach combining Large Language Models and Constraint Programming to improve interactive decision support, specifically in meeting scheduling, by enabling iterative preference elicitation and collaborative decision-making.

Contribution

It introduces a novel hybrid framework integrating LLMs with constraint programming for interactive decision support, demonstrated through studies on meeting scheduling.

Findings

Effective preference elicitation through hybrid LLM-constraint programming approach

Improved system performance in meeting scheduling tasks

Positive user feedback on collaborative decision-making process

Abstract

A critical factor in the success of decision support systems is the accurate modeling of user preferences. Psychology research has demonstrated that users often develop their preferences during the elicitation process, highlighting the pivotal role of system-user interaction in developing personalized systems. This paper introduces a novel approach, combining Large Language Models (LLMs) with Constraint Programming to facilitate interactive decision support. We study this hybrid framework through the lens of meeting scheduling, a time-consuming daily activity faced by a multitude of information workers. We conduct three studies to evaluate the novel framework, including a diary study (n=64) to characterize contextual scheduling preferences, a quantitative evaluation of the system's performance, and a user study (n=10) with a prototype system. Our work highlights the potential for a hybrid LLM and optimization approach for iterative preference elicitation and design considerations for building systems that support human-system collaborative decision-making processes.