LeMoRe: Learn More Details for Lightweight Semantic Segmentation

TL;DR

LeMoRe introduces a novel lightweight semantic segmentation approach that combines explicit and implicit modeling with nested attention to improve performance while maintaining computational efficiency across multiple datasets.

Contribution

The paper proposes a new paradigm that synergizes explicit and implicit feature modeling with nested attention, addressing the efficiency-performance trade-off in lightweight segmentation.

Findings

LeMoRe achieves competitive accuracy on ADE20K, CityScapes, Pascal Context, and COCO-Stuff.

The method balances efficiency and performance better than existing approaches.

Extensive experiments validate the effectiveness of the proposed paradigm.

Abstract

Lightweight semantic segmentation is essential for many downstream vision tasks. Unfortunately, existing methods often struggle to balance efficiency and performance due to the complexity of feature modeling. Many of these existing approaches are constrained by rigid architectures and implicit representation learning, often characterized by parameter-heavy designs and a reliance on computationally intensive Vision Transformer-based frameworks. In this work, we introduce an efficient paradigm by synergizing explicit and implicit modeling to balance computational efficiency with representational fidelity. Our method combines well-defined Cartesian directions with explicitly modeled views and implicitly inferred intermediate representations, efficiently capturing global dependencies through a nested attention mechanism. Extensive experiments on challenging datasets, including ADE20K, CityScapes, Pascal Context, and COCO-Stuff, demonstrate that LeMoRe strikes an effective balance between performance and efficiency.