KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification

TL;DR

This paper introduces KD-OCT, a knowledge distillation framework that compresses a large, high-performing OCT classification model into a lightweight one, enabling real-time, accurate AMD screening suitable for clinical edge deployment.

Contribution

The study presents a novel real-time knowledge distillation method for OCT classification, achieving high accuracy with significantly reduced model size and inference time.

Findings

KD-OCT outperforms similar classifiers in efficiency and accuracy.

The lightweight model exceeds many existing frameworks in diagnostic performance.

Near-teacher performance is achieved with substantial model compression.

Abstract

Age-related macular degeneration (AMD) and choroidal neovascularization (CNV)-related conditions are leading causes of vision loss worldwide, with optical coherence tomography (OCT) serving as a cornerstone for early detection and management. However, deploying state-of-the-art deep learning models like ConvNeXtV2-Large in clinical settings is hindered by their computational demands. Therefore, it is desirable to develop efficient models that maintain high diagnostic performance while enabling real-time deployment. In this study, a novel knowledge distillation framework, termed KD-OCT, is proposed to compress a high-performance ConvNeXtV2-Large teacher model, enhanced with advanced augmentations, stochastic weight averaging, and focal loss, into a lightweight EfficientNet-B2 student for classifying normal, drusen, and CNV cases. KD-OCT employs real-time distillation with a combined loss balancing soft teacher knowledge transfer and hard ground-truth supervision. The effectiveness of the proposed method is evaluated on the Noor Eye Hospital (NEH) dataset using patient-level cross-validation. Experimental results demonstrate that KD-OCT outperforms comparable multi-scale or feature-fusion OCT classifiers in efficiency-accuracy balance, achieving near-teacher performance with substantial reductions in model size and inference time. Despite the compression, the student model exceeds most existing frameworks, facilitating edge deployment for AMD screening. Code is available at https://github.com/erfan-nourbakhsh/KD-OCT.