Learning to Upscale 3D Segmentations in Neuroimaging

TL;DR

This paper introduces a scalable, boundary-aware framework for upscaling 3D neuroimaging segmentations from low to ultra-high resolution, improving detail and generalization across domains and unseen classes.

Contribution

It proposes a novel method that regresses signed distance maps for boundary-aware supervision, reducing memory use and enabling generalization to unseen classes in 3D neuroimaging.

Findings

Outperforms conventional methods in upscaling 3D neuroimages

Effective on ultra-high-resolution human brain MRI (~100 μm)

Demonstrates superior scalability and generalization in experiments

Abstract

Obtaining high-resolution (HR) segmentations from coarse annotations is a pervasive challenge in computer vision. Applications include inferring pixel-level segmentations from token-level labels in vision transformers, upsampling coarse masks to full resolution, and transferring annotations from legacy low-resolution (LR) datasets to modern HR imagery. These challenges are especially acute in 3D neuroimaging, where manual labeling is costly and resolutions continually increase. We propose a scalable framework that generalizes across resolutions and domains by regressing signed distance maps, enabling smooth, boundary-aware supervision. Crucially, our model predicts one class at a time, which substantially reduces memory usage during training and inference (critical for large 3D volumes) and naturally supports generalization to unseen classes. Generalization is further improved through training on synthetic, domain-randomized data. We validate our approach on ultra-high-resolution (UHR) human brain MRI (~100 {\mu}m), where most existing methods operate at 1 mm resolution. Our framework effectively upsamples such standard-resolution segmentations to UHR detail. Results on synthetic and real data demonstrate superior scalability and generalization compared to conventional segmentation methods. Code is available at: https://github.com/HuXiaoling/Learn2Upscale.