Diagnosis Assistant for Liver Cancer Utilizing a Large Language Model with Three Types of Knowledge

TL;DR

This paper presents a specialized AI diagnostic assistant for liver cancer that combines large and small models, improving image segmentation, knowledge integration, and mimicking experienced doctors' reasoning to aid less experienced clinicians.

Contribution

It introduces a novel framework integrating multi-scale segmentation, personalized knowledge bases, and Chain of Thought prompting with retrieval-augmented generation for improved liver cancer diagnosis.

Findings

Small models enhance segmentation accuracy for tumors and vessels.

Large model scores over 1 point higher in doctor evaluations.

Method improves interpretability and reliability of AI-assisted diagnosis.

Abstract

Liver cancer has a high incidence rate, but primary healthcare settings often lack experienced doctors. Advances in large models and AI technologies offer potential assistance. This work aims to address limitations in liver cancer diagnosis models, such as poor understanding of medical images, insufficient consideration of liver blood vessels, and ensuring accurate medical information. We propose a specialized diagnostic assistant to improve the diagnostic capabilities of less experienced doctors. Our framework combines large and small models, using optimized small models for precise patient image perception. Specifically, a segmentation network iteratively removes ambiguous pixels for liver tumor segmentation, and a multi-scale, multi-level differential network segments liver vessels. Features from these segmentations and medical records form a patient's personalized knowledge base. For diagnosis, Chain of Thought (COT) technology designs prompts mimicking experienced doctors' thought patterns, and Retrieval-Augmented Generation (RAG) technology provides answers based on reliable domain knowledge and trusted cases. Our small model methods improve liver tumor and vessel segmentation performance, resulting in more accurate information extraction. The large model component scores over 1 point higher on a 10-point scale in evaluations by doctors compared to control methods. Our method enhances semantic perception of medical images, improves classification of ambiguous pixels, and optimizes small object perception. It considers blood vessel positions for specific treatments and improves response credibility and interpretability by mimicking experienced doctors' thought processes using reliable resources. This approach has been recognized by doctors and benefits liver cancer auxiliary diagnosis.