Fitting Segmentation Networks on Varying Image Resolutions using Splatting

TL;DR

This paper introduces a splat layer for segmentation networks that handles varying image resolutions directly, avoiding resampling artifacts and improving segmentation accuracy on multi-modal medical images.

Contribution

The novel splat layer enables neural networks to process images at native resolutions, eliminating the need for pre-processing resampling and reducing associated artifacts.

Findings

Improved segmentation accuracy over traditional resampling methods

Effective handling of multi-resolution multi-modal medical images

Validated on two public datasets with simulated and real data

Abstract

Data used in image segmentation are not always defined on the same grid. This is particularly true for medical images, where the resolution, field-of-view and orientation can differ across channels and subjects. Images and labels are therefore commonly resampled onto the same grid, as a pre-processing step. However, the resampling operation introduces partial volume effects and blurring, thereby changing the effective resolution and reducing the contrast between structures. In this paper we propose a splat layer, which automatically handles resolution mismatches in the input data. This layer pushes each image onto a mean space where the forward pass is performed. As the splat operator is the adjoint to the resampling operator, the mean-space prediction can be pulled back to the native label space, where the loss function is computed. Thus, the need for explicit resolution adjustment using interpolation is removed. We show on two publicly available datasets, with simulated and real multi-modal magnetic resonance images, that this model improves segmentation results compared to resampling as a pre-processing step.