An Empirical Study of Attention Networks for Semantic Segmentation

TL;DR

This paper empirically analyzes attention-based networks for semantic segmentation, focusing on their computational complexity, performance, and suitable application scenarios, providing insights for future research and engineering use.

Contribution

It systematically compares attention networks in semantic segmentation regarding complexity and performance, and summarizes key considerations for their construction and application.

Findings

Attention networks vary in computational complexity and accuracy.

Certain attention methods are better suited for specific application scenarios.

The paper highlights the need for consistent analysis of FLOPs and memory usage.

Abstract

Semantic segmentation is a vital problem in computer vision. Recently, a common solution to semantic segmentation is the end-to-end convolution neural network, which is much more accurate than traditional methods.Recently, the decoders based on attention achieve state-of-the-art (SOTA) performance on various datasets. But these networks always are compared with the mIoU of previous SOTA networks to prove their superiority and ignore their characteristics without considering the computation complexity and precision in various categories, which is essential for engineering applications. Besides, the methods to analyze the FLOPs and memory are not consistent between different networks, which makes the comparison hard to be utilized. What's more, various methods utilize attention in semantic segmentation, but the conclusion of these methods is lacking. This paper first conducts experiments to analyze their computation complexity and compare their performance. Then it summarizes suitable scenes for these networks and concludes key points that should be concerned when constructing an attention network. Last it points out some future directions of the attention network.