OrthoGAN:High-Precision Image Generation for Teeth Orthodontic Visualization

TL;DR

This paper introduces OrthoGAN, a high-precision generative model that visualizes orthodontic treatment outcomes on frontal facial images, aiding patient understanding and planning with improved realism and accuracy.

Contribution

The paper presents a novel multi-modal encoder-decoder model that accurately simulates teeth alignment effects in frontal images, integrating 3D teeth models and original image colors.

Findings

Effective visualization of orthodontic outcomes demonstrated

High realism and identity preservation in generated images

Validated through extensive qualitative and clinical experiments

Abstract

Patients take care of what their teeth will be like after the orthodontics. Orthodontists usually describe the expectation movement based on the original smile images, which is unconvincing. The growth of deep-learning generative models change this situation. It can visualize the outcome of orthodontic treatment and help patients foresee their future teeth and facial appearance. While previous studies mainly focus on 2D or 3D virtual treatment outcome (VTO) at a profile level, the problem of simulating treatment outcome at a frontal facial image is poorly explored. In this paper, we build an efficient and accurate system for simulating virtual teeth alignment effects in a frontal facial image. Our system takes a frontal face image of a patient with visible malpositioned teeth and the patient's 3D scanned teeth model as input, and progressively generates the visual results of the patient's teeth given the specific orthodontics planning steps from the doctor (i.e., the specification of translations and rotations of individual tooth). We design a multi-modal encoder-decoder based generative model to synthesize identity-preserving frontal facial images with aligned teeth. In addition, the original image color information is used to optimize the orthodontic outcomes, making the results more natural. We conduct extensive qualitative and clinical experiments and also a pilot study to validate our method.