Retrieval Augmented Thought Process for Private Data Handling in Healthcare

TL;DR

This paper introduces Retrieval-Augmented Thought Process (RATP), a method that enhances privacy-preserving healthcare question-answering by combining external knowledge retrieval with decision-based reasoning, improving accuracy on private medical data.

Contribution

The paper proposes RATP, a novel approach that integrates multi-step decision processes with retrieval and learning techniques to handle private healthcare data securely and effectively.

Findings

RATP achieves 35% higher accuracy on private medical records.

It effectively combines retrieval with decision processes for improved reasoning.

The method maintains privacy by excluding private data from training.

Abstract

Large Language Models (LLMs) have demonstrated the strong potential to assist both clinicians and the general public with their extensive medical knowledge. However, their application in healthcare is constrained due to concerns about the privacy of data used in training, which prevents the integration of private and personal information because of security and ethical issues. Moreover, if their capabilities can be enhanced with information retrieval to access up-to-date knowledge, the current integration of LLMs with Information retrieval lacks robustness to imperfect retrieval, which can hinder their effectiveness and even reduce overall performance. In this work, we address this challenge by introducing the Retrieval-Augmented Thought Process (RATP). Given access to external knowledge, RATP formulates the thought generation of LLMs as a multiple-step decision process. To optimise such a thought process, RATP leverages Monte-Carlo Tree Search and learns a proxy reward function that permits cost-efficient inference. On a private dataset of electronic medical records, deliberately excluded from any LLM training set, RATP achieves 35% additional accuracy compared to in-context retrieval-augmented generation for the question-answering task.