Spatial Lifting for Dense Prediction

TL;DR

Spatial Lifting (SL) is a new method that lifts 2D images into higher-dimensional space for dense prediction, achieving competitive performance with fewer parameters and lower inference costs.

Contribution

The paper introduces Spatial Lifting, a novel approach that enhances dense prediction by operating in higher-dimensional space, reducing model complexity and inference costs.

Findings

Achieves competitive performance on 19 benchmark datasets.

Reduces model parameters by over 98% in U-Net-based models.

Lowers inference costs while maintaining accuracy.

Abstract

We present Spatial Lifting (SL), a novel methodology for dense prediction tasks. SL operates by lifting standard inputs, such as 2D images, into a higher-dimensional space and subsequently processing them using networks designed for that higher dimension, such as a 3D U-Net. Counterintuitively, this dimensionality lifting allows us to achieve good performance on benchmark tasks compared to conventional approaches, while reducing inference costs and significantly lowering the number of model parameters. The SL framework produces intrinsically structured outputs along the lifted dimension. This emergent structure facilitates dense supervision during training and enables robust, near-zero-additional-cost prediction quality assessment at test time. We validate our approach across 19 benchmark datasets (13 for semantic segmentation and 6 for depth estimation), demonstrating competitive dense prediction performance while reducing the model parameter count by over 98% (in the U-Net case) and lowering inference costs. Spatial Lifting introduces a new vision modeling paradigm that offers a promising path toward more efficient, accurate, and reliable deep networks for dense prediction tasks in vision.