Patient-Aware Multimodal RGB-HSI Fusion via Incremental Heuristic Meta-Learning for Oral Lesion Classification

TL;DR

This paper introduces a novel patient-aware multimodal fusion approach using incremental heuristic meta-learning for oral lesion classification, improving robustness and accuracy in low-resource settings with limited annotated data.

Contribution

It presents a new IHML framework that effectively combines deep learning, spectral analysis, and demographic data for oral lesion classification, especially in small-sample scenarios.

Findings

Achieved macro F1 of 66.23% and accuracy of 64.56% on unseen test data.

Demonstrated improved diagnostic robustness through RGB-to-hyperspectral reconstruction and ensemble meta-learning.

Stabilized predictions in heterogeneous, small-sample datasets using IHML.

Abstract

Early detection of oral cancer and potentially malignant diseases is a major challenge in low-resource settings due to the scarcity of annotated data. We provide a unified approach for four-class oral lesion classification that incorporates deep learning, spectral analysis, and demographic data. A pathologist-verified subset of oral cavity images was curated from a publicly available dataset. Oral cavity pictures were processed using a fine-tuned ConvNeXt-v2 network for deep embeddings before being translated into the hyperspectral domain using a reconstruction algorithm. Haemoglobin-sensitive, textural, and spectral descriptors were obtained from the reconstructed hyperspectral cubes and combined with demographic data. Multiple machine-learning models were evaluated using patient-specific validation. Finally, an incremental heuristic meta-learner (IHML) was developed that merged calibrated base classifiers via probabilistic feature stacking and uncertainty-aware abstraction of multimodal representations with patient-level smoothing. By decoupling evidence extraction from decision fusion, IHML stabilizes predictions in heterogeneous, small-sample medical datasets. On an unseen test set, our proposed model achieved a macro F1 of 66.23% and an overall accuracy of 64.56%. The findings demonstrate that RGB-to-hyperspectral reconstruction and ensemble meta-learning improve diagnostic robustness in real-world oral lesion screening.