Pre-Training with Diffusion models for Dental Radiography segmentation

TL;DR

This paper introduces a simple pre-training approach using Denoising Diffusion Probabilistic Models for dental radiography segmentation, improving label efficiency without changing model architecture.

Contribution

The authors propose a novel pre-training method leveraging DDPM for semantic segmentation, specifically tailored for dental radiographs, with competitive results.

Findings

Achieves high performance with limited labeled data

No architectural changes needed between pre-training and fine-tuning

Competitive with state-of-the-art pre-training methods

Abstract

Medical radiography segmentation, and specifically dental radiography, is highly limited by the cost of labeling which requires specific expertise and labor-intensive annotations. In this work, we propose a straightforward pre-training method for semantic segmentation leveraging Denoising Diffusion Probabilistic Models (DDPM), which have shown impressive results for generative modeling. Our straightforward approach achieves remarkable performance in terms of label efficiency and does not require architectural modifications between pre-training and downstream tasks. We propose to first pre-train a Unet by exploiting the DDPM training objective, and then fine-tune the resulting model on a segmentation task. Our experimental results on the segmentation of dental radiographs demonstrate that the proposed method is competitive with state-of-the-art pre-training methods.