# Prediction of refractive error in adolescents using a multimodal large language model

## TL;DR

This study uses a multimodal AI model to predict refractive errors in adolescents by combining eye images and clinical data, offering a scalable tool for early vision screening.

## Contribution

The novel use of a multimodal large language model for predicting refractive error in adolescents by integrating fundus images and clinical data.

## Key findings

- The model achieved a mean absolute error of 0.647 diopters in predicting refractive error.
- Multimodal data integration significantly outperformed single-modality models in predictive performance.
- The model's predictions were found to be clinically interpretable and reliable through visualization analyses.

## Abstract

This study aims to develop and evaluate a multimodal large language model (LLM) for predicting refractive error (diopter) in adolescents by integrating fundus images with clinical and demographic data. The goal is to demonstrate how such a model can serve as a digital health tool for early screening and personalized management of refractive conditions in clinical and remote health settings.

A dataset of 16,226 annotated records from adolescents aged 2 to 18 years was used. A vision-language foundation model based on Qwen2.5-VL was fine-tuned using supervised learning, incorporating both imaging and clinical data to predict spherical equivalent (SE). The model was trained on a randomly split dataset, and performance was evaluated using mean absolute error (MAE), root mean square error (RMSE), coefficient of determination (R2) and the Pearson correlation coefficient.

The model achieved a mean absolute error of 0.647 diopters and showed strong predictive performance across most refractive subgroups. The incorporation of multimodal data significantly outperformed single-modality models. Visualization analyses confirmed that the model's predictions are clinically interpretable and reliable.

This multimodal LLM demonstrates the potential of digital health technologies to enhance refractive error prediction in adolescents. Its integration into digital health platforms can provide a non-invasive, scalable solution for early myopia screening, making it an invaluable tool for clinicians in both remote and urban settings.