FBNet: Feature Balance Network for Urban-Scene Segmentation

TL;DR

This paper introduces FBNet, a novel add-on module for urban-scene segmentation that addresses feature camouflage by balancing feature representations of foreground and background, leading to improved segmentation accuracy.

Contribution

The paper proposes FBNet, consisting of BwBCE and DFM, to mitigate feature camouflage and enhance foreground feature representation in urban-scene segmentation.

Findings

Achieves state-of-the-art results on Cityscapes and BDD100K benchmarks.

Effectively reduces feature camouflage in high-level features.

Improves segmentation of small and occluded foreground objects.

Abstract

Image segmentation in the urban scene has recently attracted much attention due to its success in autonomous driving systems. However, the poor performance of concerned foreground targets, e.g., traffic lights and poles, still limits its further practical applications. In urban scenes, foreground targets are always concealed in their surrounding stuff because of the special camera position and 3D perspective projection. What's worse, it exacerbates the unbalance between foreground and background classes in high-level features due to the continuous expansion of the reception field. We call it Feature Camouflage. In this paper, we present a novel add-on module, named Feature Balance Network (FBNet), to eliminate the feature camouflage in urban-scene segmentation. FBNet consists of two key components, i.e., Block-wise BCE(BwBCE) and Dual Feature Modulator(DFM). BwBCE serves as an auxiliary loss to ensure uniform gradients for foreground classes and their surroundings during backpropagation. At the same time, DFM intends to enhance the deep representation of foreground classes in high-level features adaptively under the supervision of BwBCE. These two modules facilitate each other as a whole to ease feature camouflage effectively. Our proposed method achieves a new state-of-the-art segmentation performance on two challenging urban-scene benchmarks, i.e., Cityscapes and BDD100K. Code will be released for reproduction.