What Do LLMs Know About Alzheimer's Disease? Fine-Tuning, Probing, and Data Synthesis for AD Detection

TL;DR

This paper explores fine-tuning large language models for early Alzheimer's detection, analyzing internal representations, and creating synthetic data to improve diagnostic performance.

Contribution

It introduces a novel approach combining fine-tuning, probing, and data synthesis with special markers for AD detection using LLMs.

Findings

Probing reveals significant changes in internal representations after fine-tuning.

Task-aware markers enhance the quality of synthetic diagnostic samples.

Synthetic data improves downstream AD detection performance.

Abstract

Reliable early detection of Alzheimer's disease (AD) is challenging, particularly due to limited availability of labeled data. While large language models (LLMs) have shown strong transfer capabilities across domains, adapting them to the AD domain through supervised fine-tuning remains largely unexplored. In this work, we fine-tune an LLM for AD detection and investigate how task-relevant information is encoded within its internal representations. We employ probing techniques to analyze intermediate activations across transformer layers, and we observe that, after fine-tuning, the probing values of specific words and special markers change substantially, indicating that these elements assume a crucial role in the model's improved detection performance. Guided by this insight, we design a curated set of task-aware special markers and train a sequence-to-sequence model as a data-synthesis tool that leverages these markers to generate structurally consistent and diagnostically informative synthetic samples. We evaluate the synthesized data both intrinsically and by incorporating it into downstream training pipelines.