Feature Reuse and Fusion for Real-time Semantic segmentation

TL;DR

This paper introduces FRFNet, a lightweight encoder-decoder network for real-time semantic segmentation that balances speed and accuracy without pre-training, using novel fusion strategies.

Contribution

The paper proposes a new encoder-decoder architecture with three fusion methods for semantic and detailed information, achieving state-of-the-art real-time segmentation performance.

Findings

Achieves 72% mIoU on Cityscapes test set

Runs at 144 FPS on a single RTX 1080Ti

Balances speed and accuracy effectively

Abstract

For real-time semantic segmentation, how to increase the speed while maintaining high resolution is a problem that has been discussed and solved. Backbone design and fusion design have always been two essential parts of real-time semantic segmentation. We hope to design a light-weight network based on previous design experience and reach the level of state-of-the-art real-time semantic segmentation without any pre-training. To achieve this goal, a encoder-decoder architectures are proposed to solve this problem by applying a decoder network onto a backbone model designed for real-time segmentation tasks and designed three different ways to fuse semantics and detailed information in the aggregation phase. We have conducted extensive experiments on two semantic segmentation benchmarks. Experiments on the Cityscapes and CamVid datasets show that the proposed FRFNet strikes a balance between speed calculation and accuracy. It achieves 72% Mean Intersection over Union (mIoU%) on the Cityscapes test dataset with the speed of 144 on a single RTX 1080Ti card. The Code is available at https://github.com/favoMJ/FRFNet.