Aspect-Oriented Summarization for Psychiatric Short-Term Readmission Prediction

TL;DR

This paper proposes an aspect-oriented summarization approach using LLMs to improve 30-day psychiatric readmission prediction, effectively capturing diverse information signals from summaries to enhance model performance.

Contribution

It introduces a novel aspect-oriented summarization method and integration techniques to improve predictive accuracy in complex clinical tasks.

Findings

Enhanced prediction accuracy for psychiatric readmission.

Effective use of multiple aspect-oriented summaries.

Validated on real-world multi-hospital data.

Abstract

Recent progress in large language models (LLMs) has enabled the automated processing of lengthy documents even without supervised training on a task-specific dataset. Yet, their zero-shot performance in complex tasks as opposed to straightforward information extraction tasks remains suboptimal. One feasible approach for tasks with lengthy, complex input is to first summarize the document and then apply supervised fine-tuning to the summary. However, the summarization process inevitably results in some loss of information. In this study we present a method for processing the summaries of long documents aimed to capture different important aspects of the original document. We hypothesize that LLM summaries generated with different aspect-oriented prompts contain different information signals, and we propose methods to measure these differences. We introduce approaches to effectively integrate signals from these different summaries for supervised training of transformer models. We validate our hypotheses on a high-impact task -- 30-day readmission prediction from a psychiatric discharge -- using real-world data from four hospitals, and show that our proposed method increases the prediction performance for the complex task of predicting patient outcome.