Morphology-Aware KOA Classification: Integrating Graph Priors with Vision Models

TL;DR

This paper introduces a multimodal framework that combines anatomical structure graphs with vision models to improve knee osteoarthritis diagnosis from radiographs, achieving significant accuracy gains over existing methods.

Contribution

It presents a novel approach integrating morphological graph priors with vision models using mutual information maximization for enhanced KOA classification.

Findings

Achieved up to 10 ext% accuracy improvement over baselines.

Surpassed state-of-the-art methods by 8 ext% in accuracy.

Enhanced F1 score by 11 ext% through anatomical priors.

Abstract

Knee osteoarthritis (KOA) diagnosis from radiographs remains challenging due to the subtle morphological details that standard deep learning models struggle to capture effectively. We propose a novel multimodal framework that combines anatomical structure with radiographic features by integrating a morphological graph representation - derived from Segment Anything Model (SAM) segmentations - with a vision encoder. Our approach enforces alignment between geometry-informed graph embeddings and radiographic features through mutual information maximization, significantly improving KOA classification accuracy. By constructing graphs from anatomical features, we introduce explicit morphological priors that mirror clinical assessment criteria, enriching the feature space and enhancing the model's inductive bias. Experiments on the Osteoarthritis Initiative dataset demonstrate that our approach surpasses single-modality baselines by up to 10\% in accuracy (reaching nearly 80\%), while outperforming existing state-of-the-art methods by 8\% in accuracy and 11\% in F1 score. These results underscore the critical importance of incorporating anatomical structure into radiographic analysis for accurate KOA severity grading.