Integrating Supervised Extractive and Generative Language Models for Suicide Risk Evidence Summarization

TL;DR

This paper presents a novel approach combining supervised extractive and generative language models to improve evidence summarization for suicide risk detection, achieving top performance in a shared task.

Contribution

It introduces an integrated method that leverages BERT-based risk estimation and MentaLLaMa for summarization, advancing suicide risk evidence summarization techniques.

Findings

Achieved 1st place in highlight extraction based on recall.

Ranked 10th in summary generation based on consistency metrics.

Demonstrated effective integration of extractive and generative models for mental health applications.

Abstract

We propose a method that integrates supervised extractive and generative language models for providing supporting evidence of suicide risk in the CLPsych 2024 shared task. Our approach comprises three steps. Initially, we construct a BERT-based model for estimating sentence-level suicide risk and negative sentiment. Next, we precisely identify high suicide risk sentences by emphasizing elevated probabilities of both suicide risk and negative sentiment. Finally, we integrate generative summaries using the MentaLLaMa framework and extractive summaries from identified high suicide risk sentences and a specialized dictionary of suicidal risk words. SophiaADS, our team, achieved 1st place for highlight extraction and ranked 10th for summary generation, both based on recall and consistency metrics, respectively.