medR: Reward Engineering for Clinical Offline Reinforcement Learning via Tri-Drive Potential Functions

TL;DR

This paper introduces medR, an automated reward engineering framework for clinical offline reinforcement learning that uses large language models to design and verify reward functions, improving policy performance across diverse diseases.

Contribution

We propose a novel LLM-driven pipeline for automated reward design in clinical RL, incorporating potential functions and quantitative metrics for optimal policy learning.

Findings

Automated reward design improves policy performance.

The framework generalizes across multiple diseases.

Quantitative metrics effectively evaluate reward structures.

Abstract

Reinforcement Learning (RL) offers a powerful framework for optimizing dynamic treatment regimes (DTRs). However, clinical RL is fundamentally bottlenecked by reward engineering: the challenge of defining signals that safely and effectively guide policy learning in complex, sparse offline environments. Existing approaches often rely on manual heuristics that fail to generalize across diverse pathologies. To address this, we propose an automated pipeline leveraging Large Language Models (LLMs) for offline reward design and verification. We formulate the reward function using potential functions consisted of three core components: survival, confidence, and competence. We further introduce quantitative metrics to rigorously evaluate and select the optimal reward structure prior to deployment. By integrating LLM-driven domain knowledge, our framework automates the design of reward functions for specific diseases while significantly enhancing the performance of the resulting policies.