CNSight: Evaluation of Clinical Note Segmentation Tools

TL;DR

This paper evaluates various clinical note segmentation tools, highlighting the superior performance of large language models like GPT-5-mini in accurately identifying section boundaries in unstructured medical notes.

Contribution

It provides a comprehensive comparison of rule-based, transformer-based, and large language models for clinical note segmentation on a curated dataset.

Findings

Large language models achieve highest F1 scores (~72.4)

Lightweight baselines perform well on structured tasks but poorly on unstructured text

Results guide method selection for downstream clinical NLP tasks

Abstract

Clinical notes are often stored in unstructured or semi-structured formats after extraction from electronic medical record (EMR) systems, which complicates their use for secondary analysis and downstream clinical applications. Reliable identification of section boundaries is a key step toward structuring these notes, as sections such as history of present illness, medications, and discharge instructions each provide distinct clinical contexts. In this work, we evaluate rule-based baselines, domain-specific transformer models, and large language models for clinical note segmentation using a curated dataset of 1,000 notes from MIMIC-IV. Our experiments show that large API-based models achieve the best overall performance, with GPT-5-mini reaching a best average F1 of 72.4 across sentence-level and freetext segmentation. Lightweight baselines remain competitive on structured sentence-level tasks but falter on unstructured freetext. Our results provide guidance for method selection and lay the groundwork for downstream tasks such as information extraction, cohort identification, and automated summarization.