U-Define: Designing User Workflows for Hard and Soft Constraints in LLM-Based Planning

TL;DR

U-Define introduces a user-friendly system enabling natural language constraint specification with distinct verification methods, enhancing reliability and control in LLM-based planning.

Contribution

It proposes a novel workflow that categorizes constraints as hard or soft, verified through formal methods and LLM evaluation, improving user intent expression.

Findings

User-defined constraint types increase perceived usefulness.

The system improves planning performance and user satisfaction.

Constraints verification maintains usability while enhancing reliability.

Abstract

LLMs are increasingly used for end-user task planning, yet their black-box nature limits users' ability to ensure reliability and control. While recent systems incorporate verification techniques, it remains unclear how users can effectively apply such rigid constraints to represent intent or adapt to real-world variability. For example, prior work finds that hard-only constraints are too rigid, and numeric flexibility weights confuse users. We investigate how interaction workflows can better support users in applying constraints to guide LLM-generated plans, examining whether abstracting strictness into high-level types (i.e., hard and soft) paired with distinct verification mechanisms helps users more reliably express and align intent. We present U-Define, a system that lets users define constraints in natural language and categorize them as either hard rules that must not be violated or soft preferences that allow flexibility. U-Define verifies these types through complementary methods: formal model checking for hard constraints and LLM-as-judge evaluation for soft ones. Through a technical evaluation and user studies with general and expert participants, we find that user-defined constraint types improve perceived usefulness, performance, and satisfaction while maintaining usability. These findings provide insights for designing flexible yet reliable constraint-based workflows.