Deep Learning for Retinal Degeneration Assessment: A Comprehensive Analysis of the MARIO Challenge

TL;DR

This paper analyzes the MARIO challenge at MICCAI 2024, which evaluates AI models for detecting and predicting age-related macular degeneration progression using OCT images, establishing benchmarks and comparing AI to physicians.

Contribution

It provides a comprehensive overview of the challenge structure, datasets, methods, and results, highlighting AI's current capabilities and limitations in AMD monitoring.

Findings

AI matches physicians in detecting AMD progression

AI currently cannot reliably predict future AMD evolution

The challenge sets a benchmark for future AMD analysis methods

Abstract

The MARIO challenge, held at MICCAI 2024, focused on advancing the automated detection and monitoring of age-related macular degeneration (AMD) through the analysis of optical coherence tomography (OCT) images. Designed to evaluate algorithmic performance in detecting neovascular activity changes within AMD, the challenge incorporated unique multi-modal datasets. The primary dataset, sourced from Brest, France, was used by participating teams to train and test their models. The final ranking was determined based on performance on this dataset. An auxiliary dataset from Algeria was used post-challenge to evaluate population and device shifts from submitted solutions. Two tasks were involved in the MARIO challenge. The first one was the classification of evolution between two consecutive 2D OCT B-scans. The second one was the prediction of future AMD evolution over three months for patients undergoing anti-vascular endothelial growth factor (VEGF) therapy. Thirty-five teams participated, with the top 12 finalists presenting their methods. This paper outlines the challenge's structure, tasks, data characteristics, and winning methodologies, setting a benchmark for AMD monitoring using OCT, infrared imaging, and clinical data (such as the number of visits, age, gender, etc.). The results of this challenge indicate that artificial intelligence (AI) performs as well as a physician in measuring AMD progression (Task 1) but is not yet able of predicting future evolution (Task 2).