GLEAM: A Multimodal Imaging Dataset and HAMM for Glaucoma Classification

TL;DR

This paper introduces GLEAM, a comprehensive multimodal glaucoma dataset, and HAMM, a novel hierarchical attentive modeling framework for improved disease classification using diverse imaging modalities.

Contribution

The paper presents the first publicly available tri-modal glaucoma dataset and a new hierarchical attentive masked modeling approach for multimodal disease classification.

Findings

GLEAM enables effective multimodal analysis for glaucoma diagnosis.

HAMM improves cross-modal feature integration for classification accuracy.

The framework facilitates accurate glaucoma staging across multiple disease stages.

Abstract

We propose glaucoma lesion evaluation and analysis with multimodal imaging (GLEAM), the first publicly available tri-modal glaucoma dataset comprising scanning laser ophthalmoscopy fundus images, circumpapillary OCT images, and visual field pattern deviation maps, annotated with four disease stages, enabling effective exploitation of multimodal complementary information and facilitating accurate diagnosis and treatment across disease stages. To effectively integrate cross-modal information, we propose hierarchical attentive masked modeling (HAMM) for multimodal glaucoma classification. Our framework employs hierarchical attentive encoders and light decoders to focus cross-modal representation learning on the encoder.