PlaneCycle: Training-Free 2D-to-3D Lifting of Foundation Models Without Adapters

TL;DR

PlaneCycle is a training-free, adapter-free method that enables 2D foundation models to perform 3D tasks by cyclically distributing spatial aggregation, without retraining or architectural changes.

Contribution

It introduces a novel operator that lifts 2D models to 3D without training, applicable to any 2D network, and achieves competitive 3D performance.

Findings

Without training, models show intrinsic 3D fusion capability.

Under linear probing, outperforms slice-wise 2D baselines.

Matches standard 3D architectures after full fine-tuning.

Abstract

Large-scale 2D foundation models exhibit strong transferable representations, yet extending them to 3D volumetric data typically requires retraining, adapters, or architectural redesign. We introduce PlaneCycle, a training-free, adapter-free operator for architecture-agnostic 2D-to-3D lifting of foundation models. PlaneCycle reuses the original pretrained 2D backbone by cyclically distributing spatial aggregation across orthogonal HW, DW, and DH planes throughout network depth, enabling progressive 3D fusion while preserving pretrained inductive biases. The method introduces no additional parameters and is applicable to arbitrary 2D networks. Using pretrained DINOv3 models, we evaluate PlaneCycle on six 3D classification and three 3D segmentation benchmarks. Without any training, the lifted models exhibit intrinsic 3D fusion capability and, under linear probing, outperform slice-wise 2D baselines and strong 3D counterparts, approaching the performance of fully trained models. With full fine-tuning, PlaneCycle matches standard 3D architectures, highlighting its potential as a seamless and practical 2D-to-3D lifting operator. These results demonstrate that 3D capability can be unlocked from pretrained 2D foundation models without structural modification or retraining. Code is available at https://github.com/HINTLab/PlaneCycle.