Dynamic Feature Fusion for Semantic Edge Detection

TL;DR

This paper introduces a dynamic feature fusion method for semantic edge detection that adaptively assigns fusion weights based on input and location, leading to improved accuracy and state-of-the-art results.

Contribution

The work proposes a novel weight learner for adaptive fusion of multi-level features in semantic edge detection, surpassing fixed and naive fusion strategies.

Findings

Outperforms existing methods on Cityscapes and SBD benchmarks.

Achieves new state-of-the-art performance in semantic edge detection.

Demonstrates the effectiveness of dynamic, input-conditioned fusion weights.

Abstract

Features from multiple scales can greatly benefit the semantic edge detection task if they are well fused. However, the prevalent semantic edge detection methods apply a fixed weight fusion strategy where images with different semantics are forced to share the same weights, resulting in universal fusion weights for all images and locations regardless of their different semantics or local context. In this work, we propose a novel dynamic feature fusion strategy that assigns different fusion weights for different input images and locations adaptively. This is achieved by a proposed weight learner to infer proper fusion weights over multi-level features for each location of the feature map, conditioned on the specific input. In this way, the heterogeneity in contributions made by different locations of feature maps and input images can be better considered and thus help produce more accurate and sharper edge predictions. We show that our model with the novel dynamic feature fusion is superior to fixed weight fusion and also the na\"ive location-invariant weight fusion methods, via comprehensive experiments on benchmarks Cityscapes and SBD. In particular, our method outperforms all existing well established methods and achieves new state-of-the-art.