StethoLM: Audio Language Model for Cardiopulmonary Analysis Across Clinical Tasks

TL;DR

StethoLM is a novel audio-language model designed for comprehensive cardiopulmonary auscultation analysis, integrating audio encoding with medical language understanding to assist clinicians across various tasks.

Contribution

It introduces the first specialized audio-language model for auscultation, trained on a large benchmark with diverse clinical tasks, enhancing interpretability and robustness.

Findings

Achieves improved performance on multiple clinical tasks

Demonstrates robustness on out-of-distribution data

Establishes a foundation for AI in clinical auscultation

Abstract

Listening to heart and lung sounds - auscultation - is one of the first and most fundamental steps in a clinical examination. Despite being fast and non-invasive, it demands years of experience to interpret subtle audio cues. Recent deep learning methods have made progress in automating cardiopulmonary sound analysis, yet most are restricted to simple classification and offer little clinical interpretability or decision support. We present StethoLM, the first audio-language model specialized for cardiopulmonary auscultation, capable of performing instruction-driven clinical tasks across the full spectrum of auscultation analysis. StethoLM integrates audio encoding with a medical language model backbone and is trained on StethoBench, a comprehensive benchmark comprising 77,027 instruction-response pairs synthesized from 16,125 labeled cardiopulmonary recordings spanning seven clinical task categories: binary classification, detection, reporting, reasoning, differential diagnosis, comparison, and location-based analysis. Through multi-stage training that combines supervised fine-tuning and direct preference optimization, StethoLM achieves substantial gains in performance and robustness on out-of-distribution data. Our work establishes a foundation for instruction-following AI systems in clinical auscultation.