Spatially-Adaptive Gradient Re-parameterization for 3D Large Kernel Optimization

TL;DR

This paper introduces Rep3D, a novel framework that uses spatially adaptive re-parameterization to improve large kernel 3D convolutions, enhancing stability and performance in high-resolution volumetric segmentation tasks.

Contribution

Rep3D employs a lightweight modulation network to generate spatially biased scaling masks, unifying spatial bias with optimization-aware learning without complex multi-branch designs.

Findings

Outperforms state-of-the-art transformer and fixed-prior methods on 3D segmentation benchmarks.

Ensures robust local-to-global convergence in large kernel 3D convolutions.

Demonstrates stability and effectiveness in high-resolution volumetric analysis.

Abstract

Large kernel convolutions offer a scalable alternative to vision transformers for high-resolution 3D volumetric analysis, yet naively increasing kernel size often leads to optimization instability. Motivated by the spatial bias inherent in effective receptive fields (ERFs), we theoretically demonstrate that structurally re-parameterized blocks induce spatially varying learning rates that are crucial for convergence. Leveraging this insight, we introduce Rep3D, a framework that employs a lightweight modulation network to generate receptive-biased scaling masks, adaptively re-weighting kernel updates within a plain encoder architecture. This approach unifies spatial inductive bias with optimization-aware learning, avoiding the complexity of multi-branch designs while ensuring robust local-to-global convergence. Extensive evaluations on five 3D segmentation benchmarks demonstrate that Rep3D consistently outperforms state-of-the-art transformer and fixed-prior baselines. The source code is publicly available at https://github.com/leeh43/Rep3D.