AdaSFormer: Adaptive Serialized Transformers for Monocular Semantic Scene Completion from Indoor Environments

TL;DR

AdaSFormer introduces an adaptive serialized transformer framework with novel positional encoding and normalization techniques, significantly improving indoor monocular semantic scene completion performance by effectively modeling spatial details and dependencies.

Contribution

The paper proposes AdaSFormer, a novel serialized transformer architecture with adaptive receptive fields and specialized encoding, tailored specifically for indoor MSSC tasks.

Findings

Achieves state-of-the-art results on NYUv2 and Occ-ScanNet datasets.

Demonstrates effective modeling of complex indoor spatial layouts.

Outperforms existing methods in accuracy and detail reconstruction.

Abstract

Indoor monocular semantic scene completion (MSSC) is notably more challenging than its outdoor counterpart due to complex spatial layouts and severe occlusions. While transformers are well suited for modeling global dependencies, their high memory cost and difficulty in reconstructing fine-grained details have limited their use in indoor MSSC. To address these limitations, we introduce AdaSFormer, a serialized transformer framework tailored for indoor MSSC. Our model features three key designs: (1) an Adaptive Serialized Transformer with learnable shifts that dynamically adjust receptive fields; (2) a Center-Relative Positional Encoding that captures spatial information richness; and (3) a Convolution-Modulated Layer Normalization that bridges heterogeneous representations between convolutional and transformer features. Extensive experiments on NYUv2 and Occ-ScanNet demonstrate that AdaSFormer achieves state-of-the-art performance. The code is publicly available at: https://github.com/alanWXZ/AdaSFormer.