Assessing large language models as assistive tools in selecting first trial lens parameters for orthokeratology

TL;DR

This study evaluates how well large language models can help choose initial lens parameters for orthokeratology, finding that they show promise but require clinician oversight.

Contribution

The study introduces the use of LLMs in orthokeratology fitting workflows and identifies key parameter biases requiring clinician verification.

Findings

GPT-o3 and GPT-4o showed the strongest overall performance in providing accurate and high-quality responses.

LLM outputs showed tendencies in key fitting parameters, particularly a smaller BC radius and a larger RZD.

Most observations for first trial lens parameters lay within the 95% limits of agreement.

Abstract

Large language models (LLMs) have the potential to be powerful tools in optometry. Orthokeratology is widely used in clinical interventions for myopia control. This study aims to evaluate the performance of LLMs as assistive tools in the CRT-related orthokeratology fitting workflow.

This retrospective analysis used four LLMs (GPT-4o, GPT-o3, GPT-4.1 and Claude 3.7 Sonnet) to analyze refractive error cases and get responses regarding the parameters of the first trial lens. Subjective evaluation includes the accuracy and overall quality of the answers provided, and objective evaluation focuses on differences in the parameters of the first trial lens.

GQS and accuracy differed across models [χ2(3) = 39.85, p < 0.001; Kendall’s W = 0.148]. GPT-o3 and GPT-4o showed the strongest overall performance on the complete response (GQS: 4.66 ± 0.48 vs. 4.47 ± 0.5, Good ratings: 83.3% vs. 76.7%), For first trial lens parameters, feasibility errors decreased across the two correction rounds, LLM outputs showed tendencies concentrated in key fitting parameters, particularly a smaller BC radius (mm) and a larger RZD, while Bland–Altman analyses indicated that most observations lay within the 95% limits of agreement.

LLMs may support routine CRT-related decision support. However, first trial-lens parameter selection required feasibility constraints and clinician verification, with systematic parameter bias mainly involving BC and RZD.

Figures