Optimizing Large Language Models for Dynamic Constraints through Human-in-the-Loop Discriminators

TL;DR

This paper introduces a flexible human-in-the-loop framework for optimizing large language models to better handle dynamic constraints, demonstrated through a travel planner case study with significant performance improvements.

Contribution

The paper presents a novel framework enabling LLMs to interact with system interfaces and collaborate with humans to optimize performance under complex constraints, enhancing generalizability.

Findings

7.78% pass rate with human discriminator after one iteration

40.2% improvement over baseline in performance

Effective application to a travel planning task

Abstract

Large Language Models (LLMs) have recently demonstrated impressive capabilities across various real-world applications. However, due to the current text-in-text-out paradigm, it remains challenging for LLMs to handle dynamic and complex application constraints, let alone devise general solutions that meet predefined system goals. Current common practices like model finetuning and reflection-based reasoning often address these issues case-by-case, limiting their generalizability. To address this issue, we propose a flexible framework that enables LLMs to interact with system interfaces, summarize constraint concepts, and continually optimize performance metrics by collaborating with human experts. As a case in point, we initialized a travel planner agent by establishing constraints from evaluation interfaces. Then, we employed both LLM-based and human discriminators to identify critical cases and continuously improve agent performance until the desired outcomes were achieved. After just one iteration, our framework achieved a $7.78\%$ pass rate with the human discriminator (a $40.2\%$ improvement over baseline) and a $6.11\%$ pass rate with the LLM-based discriminator. Given the adaptability of our proposal, we believe this framework can be applied to a wide range of constraint-based applications and lay a solid foundation for model finetuning with performance-sensitive data samples.