Deep BI-RADS Network for Improved Cancer Detection from Mammograms

TL;DR

This paper presents a multi-modal deep learning model that combines textual BI-RADS lesion descriptors with mammogram images, significantly improving breast cancer detection accuracy.

Contribution

It introduces a novel multi-modal fusion approach using iterative attention to incorporate expert textual features into mammogram analysis.

Findings

Enhanced classification performance over image-only models

Significant improvements across all evaluation metrics

Demonstrates value of combining textual descriptors with imaging data

Abstract

While state-of-the-art models for breast cancer detection leverage multi-view mammograms for enhanced diagnostic accuracy, they often focus solely on visual mammography data. However, radiologists document valuable lesion descriptors that contain additional information that can enhance mammography-based breast cancer screening. A key question is whether deep learning models can benefit from these expert-derived features. To address this question, we introduce a novel multi-modal approach that combines textual BI-RADS lesion descriptors with visual mammogram content. Our method employs iterative attention layers to effectively fuse these different modalities, significantly improving classification performance over image-only models. Experiments on the CBIS-DDSM dataset demonstrate substantial improvements across all metrics, demonstrating the contribution of handcrafted features to end-to-end.