IOSVLM: A 3D Vision-Language Model for Unified Dental Diagnosis from Intraoral Scans

TL;DR

IOSVLM introduces a novel 3D vision-language model for dental diagnosis from intraoral scans, leveraging native 3D geometry and a large dataset to improve accuracy and enable visual question-answering.

Contribution

The paper presents IOSVLM, the first end-to-end 3D VLM for dental diagnosis from intraoral scans, with a new dataset and geometry-to-chromatic proxy for improved 3D perception.

Findings

Achieves at least +9.58% macro accuracy over baselines.

Demonstrates effective 3D geometry modeling for dental diagnosis.

Enables visual question-answering on intraoral scans.

Abstract

3D intraoral scans (IOS) are increasingly adopted in routine dentistry due to abundant geometric evidence, and unified multi-disease diagnosis is desirable for clinical documentation and communication. While recent works introduce dental vision-language models (VLMs) to enable unified diagnosis and report generation on 2D images or multi-view images rendered from IOS, they do not fully leverage native 3D geometry. Such work is necessary and also challenging, due to: (i) heterogeneous scan forms and the complex IOS topology, (ii) multi-disease co-occurrence with class imbalance and fine-grained morphological ambiguity, (iii) limited paired 3D IOS-text data. Thus, we present IOSVLM, an end-to-end 3D VLM that represents scans as point clouds and follows a 3D encoder-projector-LLM design for unified diagnosis and generative visual question-answering (VQA), together with IOSVQA, a large-scale multi-source IOS diagnosis VQA dataset comprising 19,002 cases and 249,055 VQA pairs over 23 oral diseases and heterogeneous scan types. To address the distribution gap between color-free IOS data and color-dependent 3D pre-training, we propose a geometry-to-chromatic proxy that stabilizes fine-grained geometric perception and cross-modal alignment. A two-stage curriculum training strategy further enhances robustness. IOSVLM consistently outperforms strong baselines, achieving gains of at least +9.58% macro accuracy and +1.46% macro F1, indicating the effectiveness of direct 3D geometry modeling for IOS-based diagnosis.