Step-CoT: Stepwise Visual Chain-of-Thought for Medical Visual Question Answering

TL;DR

Step-CoT introduces a structured, expert-curated reasoning dataset for medical VQA, enhancing model interpretability and accuracy by guiding multi-step clinical reasoning aligned with diagnostic workflows.

Contribution

It presents a large-scale, structured reasoning dataset with expert annotations and a novel teacher-student framework for improved medical VQA reasoning.

Findings

Improved reasoning accuracy over baseline models

Enhanced interpretability of model decisions

Effective grounding in radiographic evidence

Abstract

Chain-of-thought (CoT) reasoning has advanced medical visual question answering (VQA), yet most existing CoT rationales are free-form and fail to capture the structured reasoning process clinicians actually follow. This work asks: Can traceable, multi-step reasoning supervision improve reasoning accuracy and the interpretability of Medical VQA? To this end, we introduce Step-CoT, a large-scale medical reasoning dataset with expert-curated, structured multi-step CoT aligned to clinical diagnostic workflows, implicitly grounding the model's reasoning in radiographic evidence. Step-CoT comprises more than 10K real clinical cases and 70K VQA pairs organized around diagnostic workflows, providing supervised intermediate steps that guide models to follow valid reasoning trajectories. To effectively learn from Step-CoT, we further introduce a teacher-student framework with a dynamic graph-structured focusing mechanism that prioritizes diagnostically informative steps while filtering out less relevant contexts. Our experiments show that using Step-CoT can improve reasoning accuracy and interpretability. Benchmark: github.com/hahaha111111/Step-CoT. Dataset Card: huggingface.co/datasets/fl-15o/Step-CoT