Adaptive Texture-aware Masking for Self-Supervised Learning in 3D Dental CBCT Analysis

TL;DR

This paper introduces ATMask, an adaptive masking strategy for self-supervised learning in 3D dental CBCT analysis, improving model focus on critical regions without heavy computation.

Contribution

Proposes ATMask, a texture-based adaptive masking method, and releases the first large-scale CBCT dataset for dental AI pretraining.

Findings

ATMask outperforms random masking in downstream tasks.

The large-scale dataset enhances pretraining effectiveness.

ATMask enables more data-efficient representation learning.

Abstract

Cone Beam Computed Tomography (CBCT) is pivotal for 3D diagnostic imaging in dentistry. However, the development of robust AI models for volumetric analysis is often constrained by the scarcity of large, annotated datasets. Self-supervised learning (SSL), particularly Masked Image Modeling (MIM), offers a promising pathway to leverage unlabeled data. A limitation of standard MIM is its reliance on random masking, which fails to prioritize diagnostically critical regions in dental CBCT volumes, such as subtle pathological changes and intricate anatomical boundaries. To address this, we propose ATMask, a novel adaptive masking strategy. Instead of applying random masks or employing computationally intensive attention modules, ATMask computes an inter-slice texture variation map to identify regions with high structural or textural complexity. These high-variation areas are then selectively masked during pre-training, compelling the model to learn richer contextual representations essential for inferring complex 3D morphological transitions. Furthermore, we contribute the first large-scale CBCT dataset, curated from both public and private sources, comprising 6,314 scans, for the dental AI model pretraining. Extensive experiments on three downstream dental CBCT tasks demonstrate that our ATMask enables more data-efficient and powerful representation learning than standard random masking and other advanced SSL baselines. The dataset and code will be released.