LIBRA: Language Model Informed Bandit Recourse Algorithm for Personalized Treatment Planning

TL;DR

This paper introduces LIBRA, a novel bandit algorithm that integrates large language models with decision-making processes to improve personalized treatment planning, ensuring efficiency, robustness, and reduced reliance on LLMs.

Contribution

The paper proposes LIBRA, a new framework combining LLMs with bandit algorithms for recourse in high-stakes decisions, with theoretical guarantees and empirical validation.

Findings

LIBRA reduces initial regret with near-optimal LLM recommendations.

LIBRA consults LLMs only O(log^2 T) times, ensuring efficiency.

LIBRA outperforms standard bandits and LLM-only methods in experiments.

Abstract

We introduce a unified framework that seamlessly integrates algorithmic recourse, contextual bandits, and large language models (LLMs) to support sequential decision-making in high-stakes settings such as personalized medicine. We first introduce the recourse bandit problem, where a decision-maker must select both a treatment action and a feasible, minimal modification to mutable patient features. To address this problem, we develop the Generalized Linear Recourse Bandit (GLRB) algorithm. Building on this foundation, we propose LIBRA, a Language Model-Informed Bandit Recourse Algorithm that strategically combines domain knowledge from LLMs with the statistical rigor of bandit learning. LIBRA offers three key guarantees: (i) a warm-start guarantee, showing that LIBRA significantly reduces initial regret when LLM recommendations are near-optimal; (ii) an LLM-effort guarantee, proving that the algorithm consults the LLM only $O(\log^2 T)$ times, where $T$ is the time horizon, ensuring long-term autonomy; and (iii) a robustness guarantee, showing that LIBRA never performs worse than a pure bandit algorithm even when the LLM is unreliable. We further establish matching lower bounds that characterize the fundamental difficulty of the recourse bandit problem and demonstrate the near-optimality of our algorithms. Experiments on synthetic environments and a real hypertension-management case study confirm that GLRB and LIBRA improve regret, treatment quality, and sample efficiency compared with standard contextual bandits and LLM-only benchmarks. Our results highlight the promise of recourse-aware, LLM-assisted bandit algorithms for trustworthy LLM-bandits collaboration in personalized high-stakes decision-making.