Automatic Mapping of Anatomical Landmarks from Free-Text Using Large Language Models: Insights from Llama-2

TL;DR

This study explores how large language models like Llama-2 can automatically map anatomical landmarks from free-text radiology reports to their spatial positions in images, potentially improving medical imaging workflows.

Contribution

It demonstrates that Llama-2 models can linearly encode anatomical landmark positions, showing robustness across prompts, which is a novel application in medical imaging analysis.

Findings

Llama-2 models can linearly represent spatial positions of landmarks.

Models show robustness to different prompts.

Potential to improve medical imaging accuracy and efficiency.

Abstract

Anatomical landmarks are vital in medical imaging for navigation and anomaly detection. Modern large language models (LLMs), like Llama-2, offer promise for automating the mapping of these landmarks in free-text radiology reports to corresponding positions in image data. Recent studies propose LLMs may develop coherent representations of generative processes. Motivated by these insights, we investigated whether LLMs accurately represent the spatial positions of anatomical landmarks. Through experiments with Llama-2 models, we found that they can linearly represent anatomical landmarks in space with considerable robustness to different prompts. These results underscore the potential of LLMs to enhance the efficiency and accuracy of medical imaging workflows.