3D Structure-guided Network for Tooth Alignment in 2D Photograph

TL;DR

This paper introduces a novel 3D structure-guided network that transforms 2D photographs into orthodontic comparison images with aligned teeth, aiding communication and treatment planning in dentistry.

Contribution

It presents a new method that uses 3D intra-oral scans to guide 2D tooth alignment in photographs, combining 3D modeling with diffusion techniques.

Findings

High-quality aligned teeth images generated from 2D photos.

Effective projection of 3D tooth structures onto 2D contours.

Strong applicability demonstrated in orthodontic industry.

Abstract

Orthodontics focuses on rectifying misaligned teeth (i.e., malocclusions), affecting both masticatory function and aesthetics. However, orthodontic treatment often involves complex, lengthy procedures. As such, generating a 2D photograph depicting aligned teeth prior to orthodontic treatment is crucial for effective dentist-patient communication and, more importantly, for encouraging patients to accept orthodontic intervention. In this paper, we propose a 3D structure-guided tooth alignment network that takes 2D photographs as input (e.g., photos captured by smartphones) and aligns the teeth within the 2D image space to generate an orthodontic comparison photograph featuring aesthetically pleasing, aligned teeth. Notably, while the process operates within a 2D image space, our method employs 3D intra-oral scanning models collected in clinics to learn about orthodontic treatment, i.e., projecting the pre- and post-orthodontic 3D tooth structures onto 2D tooth contours, followed by a diffusion model to learn the mapping relationship. Ultimately, the aligned tooth contours are leveraged to guide the generation of a 2D photograph with aesthetically pleasing, aligned teeth and realistic textures. We evaluate our network on various facial photographs, demonstrating its exceptional performance and strong applicability within the orthodontic industry.