First Place Solution of 2023 Global Artificial Intelligence Technology Innovation Competition Track 1

TL;DR

This paper presents a top-performing solution for medical imaging report generation in an AI competition, utilizing a modified CPT-BASE model with innovative training and retrieval strategies to enhance report quality.

Contribution

The authors introduce a novel training approach and retrieval augmentation techniques for CPT-BASE, achieving state-of-the-art results in medical report generation.

Findings

Achieved first place in the competition leaderboard.

Enhanced report quality through retrieval augmentation and noise-aware prompts.

Single model performance surpassed previous benchmarks.

Abstract

In this paper, we present our champion solution to the Global Artificial Intelligence Technology Innovation Competition Track 1: Medical Imaging Diagnosis Report Generation. We select CPT-BASE as our base model for the text generation task. During the pre-training stage, we delete the mask language modeling task of CPT-BASE and instead reconstruct the vocabulary, adopting a span mask strategy and gradually increasing the number of masking ratios to perform the denoising auto-encoder pre-training task. In the fine-tuning stage, we design iterative retrieval augmentation and noise-aware similarity bucket prompt strategies. The retrieval augmentation constructs a mini-knowledge base, enriching the input information of the model, while the similarity bucket further perceives the noise information within the mini-knowledge base, guiding the model to generate higher-quality diagnostic reports based on the similarity prompts. Surprisingly, our single model has achieved a score of 2.321 on leaderboard A, and the multiple model fusion scores are 2.362 and 2.320 on the A and B leaderboards respectively, securing first place in the rankings.