Advancing Healthcare Automation: Multi-Agent System for Medical Necessity Justification

TL;DR

This paper presents a multi-agent system utilizing specialized Large Language Model agents, particularly GPT-4, to automate and improve the accuracy, explainability, and efficiency of medical prior authorization processes.

Contribution

It introduces a novel multi-agent framework that leverages LLMs for automating complex medical authorization tasks and systematically evaluates prompting strategies and model performance.

Findings

GPT-4 achieves 86.2% accuracy in checklist item judgments

GPT-4 achieves 95.6% accuracy in overall checklist judgment

The system enhances explainability and trust in medical authorization automation

Abstract

Prior Authorization delivers safe, appropriate, and cost-effective care that is medically justified with evidence-based guidelines. However, the process often requires labor-intensive manual comparisons between patient medical records and clinical guidelines, that is both repetitive and time-consuming. Recent developments in Large Language Models (LLMs) have shown potential in addressing complex medical NLP tasks with minimal supervision. This paper explores the application of Multi-Agent System (MAS) that utilize specialized LLM agents to automate Prior Authorization task by breaking them down into simpler and manageable sub-tasks. Our study systematically investigates the effects of various prompting strategies on these agents and benchmarks the performance of different LLMs. We demonstrate that GPT-4 achieves an accuracy of 86.2% in predicting checklist item-level judgments with evidence, and 95.6% in determining overall checklist judgment. Additionally, we explore how these agents can contribute to explainability of steps taken in the process, thereby enhancing trust and transparency in the system.