FocusMAE: Gallbladder Cancer Detection from Ultrasound Videos with Focused Masked Autoencoders

TL;DR

This paper introduces FocusMAE, a novel video-based approach using focused masked autoencoders for improved gallbladder cancer detection from ultrasound videos, achieving state-of-the-art accuracy and demonstrating generality on CT data.

Contribution

It proposes FocusMAE, a new masking strategy for autoencoders that emphasizes high-information regions in ultrasound videos for better disease representation, along with the creation of the largest US video dataset for GBC detection.

Findings

Achieved 96.4% accuracy on GBC detection, surpassing previous methods.

Demonstrated the effectiveness of FocusMAE on a public CT-based Covid dataset.

First study to utilize US video data for GBC detection.

Abstract

In recent years, automated Gallbladder Cancer (GBC) detection has gained the attention of researchers. Current state-of-the-art (SOTA) methodologies relying on ultrasound sonography (US) images exhibit limited generalization, emphasizing the need for transformative approaches. We observe that individual US frames may lack sufficient information to capture disease manifestation. This study advocates for a paradigm shift towards video-based GBC detection, leveraging the inherent advantages of spatiotemporal representations. Employing the Masked Autoencoder (MAE) for representation learning, we address shortcomings in conventional image-based methods. We propose a novel design called FocusMAE to systematically bias the selection of masking tokens from high-information regions, fostering a more refined representation of malignancy. Additionally, we contribute the most extensive US video dataset for GBC detection. We also note that, this is the first study on US video-based GBC detection. We validate the proposed methods on the curated dataset, and report a new state-of-the-art (SOTA) accuracy of 96.4% for the GBC detection problem, against an accuracy of 84% by current Image-based SOTA - GBCNet, and RadFormer, and 94.7% by Video-based SOTA - AdaMAE. We further demonstrate the generality of the proposed FocusMAE on a public CT-based Covid detection dataset, reporting an improvement in accuracy by 3.3% over current baselines. The source code and pretrained models are available at: https://gbc-iitd.github.io/focusmae