PathAgent: Toward Interpretable Analysis of Whole-slide Pathology Images via Large Language Model-based Agentic Reasoning

TL;DR

PathAgent introduces an LLM-based framework that mimics pathologists' reasoning to analyze whole-slide images, providing transparent, step-by-step predictions that enhance interpretability and generalization in pathology diagnostics.

Contribution

The paper presents PathAgent, a novel, training-free LLM-based agent that explicitly models the reasoning process in pathology image analysis, improving interpretability and zero-shot performance.

Findings

Outperforms task-specific baselines in multiple datasets

Provides fully interpretable, stepwise diagnostic reasoning

Shows strong zero-shot generalization capabilities

Abstract

Analyzing whole-slide images (WSIs) requires an iterative, evidence-driven reasoning process that parallels how pathologists dynamically zoom, refocus, and self-correct while collecting the evidence. However, existing computational pipelines often lack this explicit reasoning trajectory, resulting in inherently opaque and unjustifiable predictions. To bridge this gap, we present PathAgent, a training-free, large language model (LLM)-based agent framework that emulates the reflective, stepwise analytical approach of human experts. PathAgent can autonomously explore WSI, iteratively and precisely locating significant micro-regions using the Navigator module, extracting morphology visual cues using the Perceptor, and integrating these findings into the continuously evolving natural language trajectories in the Executor. The entire sequence of observations and decisions forms an explicit chain-of-thought, yielding fully interpretable predictions. Evaluated across five challenging datasets, PathAgent exhibits strong zero-shot generalization, surpassing task-specific baselines in both open-ended and constrained visual question-answering tasks. Moreover, a collaborative evaluation with human pathologists confirms PathAgent's promise as a transparent and clinically grounded diagnostic assistant.