PathCoT: Chain-of-Thought Prompting for Zero-shot Pathology Visual Reasoning
Junjie Zhou, Yingli Zuo, Shichang Feng, Peng Wan, Qi Zhu, Daoqiang Zhang, and Wei Shao

TL;DR
PathCoT enhances pathology visual reasoning in multimodal large language models by integrating domain-specific expert knowledge and self-evaluation, significantly improving accuracy in zero-shot scenarios.
Contribution
This paper introduces PathCoT, a novel zero-shot chain-of-thought prompting method that incorporates pathology expert knowledge and self-evaluation to improve reasoning accuracy.
Findings
PathCoT outperforms baseline models on the PathMMU dataset.
Incorporating expert knowledge reduces reasoning errors.
Self-evaluation improves answer reliability.
Abstract
With the development of generative artificial intelligence and instruction tuning techniques, multimodal large language models (MLLMs) have made impressive progress on general reasoning tasks. Benefiting from the chain-of-thought (CoT) methodology, MLLMs can solve the visual reasoning problem step-by-step. However, existing MLLMs still face significant challenges when applied to pathology visual reasoning tasks: (1) LLMs often underperforms because they lack domain-specific information, which can lead to model hallucinations. (2) The additional reasoning steps in CoT may introduce errors, leading to the divergence of answers. To address these limitations, we propose PathCoT, a novel zero-shot CoT prompting method which integrates the pathology expert-knowledge into the reasoning process of MLLMs and incorporates self-evaluation to mitigate divergence of answers. Specifically, PathCoT…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Topic Modeling · Advanced Graph Neural Networks
