Training and Inference for Integer-Based Semantic Segmentation Network

TL;DR

This paper introduces an 8-bit integer quantization framework for semantic segmentation networks, enabling faster training and inference on fixed-point devices while maintaining accuracy.

Contribution

It is the first to fully quantize segmentation networks' parameters and operations, including data flow and batch normalization, for efficient fixed-point inference.

Findings

Achieves comparable accuracy to floating-point models on ADE20K and PASCAL VOC datasets.

Enables inference on fixed-point devices with reduced computation and storage requirements.

Maintains segmentation performance despite full quantization.

Abstract

Semantic segmentation has been a major topic in research and industry in recent years. However, due to the computation complexity of pixel-wise prediction and backpropagation algorithm, semantic segmentation has been demanding in computation resources, resulting in slow training and inference speed and large storage space to store models. Existing schemes that speed up segmentation network change the network structure and come with noticeable accuracy degradation. However, neural network quantization can be used to reduce computation load while maintaining comparable accuracy and original network structure. Semantic segmentation networks are different from traditional deep convolutional neural networks (DCNNs) in many ways, and this topic has not been thoroughly explored in existing works. In this paper, we propose a new quantization framework for training and inference of segmentation networks, where parameters and operations are constrained to 8-bit integer-based values for the first time. Full quantization of the data flow and the removal of square and root operations in batch normalization give our framework the ability to perform inference on fixed-point devices. Our proposed framework is evaluated on mainstream semantic segmentation networks like FCN-VGG16 and DeepLabv3-ResNet50, achieving comparable accuracy against floating-point framework on ADE20K dataset and PASCAL VOC 2012 dataset.