Progressively Diffused Networks for Semantic Image Segmentation

TL;DR

This paper proposes Progressively Diffused Networks (PDNs) that enhance semantic image segmentation by progressively broadcasting contextual information through diffusion layers with convolutional LSTMs, leading to improved performance and state-of-the-art results.

Contribution

Introduction of PDNs that integrate multi-scale context modeling with deep feature learning using diffusion layers and convolutional LSTMs for semantic segmentation.

Findings

Achieves state-of-the-art performance on ImageNet Parsing.

Significantly improves segmentation accuracy over existing models.

Demonstrates effectiveness across multiple benchmark datasets.

Abstract

This paper introduces Progressively Diffused Networks (PDNs) for unifying multi-scale context modeling with deep feature learning, by taking semantic image segmentation as an exemplar application. Prior neural networks, such as ResNet, tend to enhance representational power by increasing the depth of architectures and driving the training objective across layers. However, we argue that spatial dependencies in different layers, which generally represent the rich contexts among data elements, are also critical to building deep and discriminative representations. To this end, our PDNs enables to progressively broadcast information over the learned feature maps by inserting a stack of information diffusion layers, each of which exploits multi-dimensional convolutional LSTMs (Long-Short-Term Memory Structures). In each LSTM unit, a special type of atrous filters are designed to capture the short range and long range dependencies from various neighbors to a certain site of the feature map and pass the accumulated information to the next layer. From the extensive experiments on semantic image segmentation benchmarks (e.g., ImageNet Parsing, PASCAL VOC2012 and PASCAL-Part), our framework demonstrates the effectiveness to substantially improve the performances over the popular existing neural network models, and achieves state-of-the-art on ImageNet Parsing for large scale semantic segmentation.