# Generation of Multimodal Justification Using Visual Word Constraint Model for Explainable Computer-Aided Diagnosis

## TL;DR

This paper introduces a deep learning model that generates multimodal explanations, including visual pointing maps and diagnostic sentences, to improve interpretability in medical diagnosis, specifically for breast mass analysis.

## Contribution

It proposes a novel visual word constraint model to enhance the accuracy of diagnostic sentence generation in explainable AI for medical imaging.

## Key findings

- Improved explanation accuracy with visual and textual modalities
- Effective diagnosis explanation for breast mass detection
- Demonstrated superiority over existing methods

## Abstract

The ambiguity of the decision-making process has been pointed out as the main obstacle to applying the deep learning-based method in a practical way in spite of its outstanding performance. Interpretability could guarantee the confidence of deep learning system, therefore it is particularly important in the medical field. In this study, a novel deep network is proposed to explain the diagnostic decision with visual pointing map and diagnostic sentence justifying result simultaneously. For the purpose of increasing the accuracy of sentence generation, a visual word constraint model is devised in training justification generator. To verify the proposed method, comparative experiments were conducted on the problem of the diagnosis of breast masses. Experimental results demonstrated that the proposed deep network could explain diagnosis more accurately with various textual justifications.