Graph-FCN for image semantic segmentation

TL;DR

This paper introduces Graph-FCN, a novel approach combining fully convolutional networks and graph convolutional networks to improve image semantic segmentation by preserving local spatial information.

Contribution

It is the first to apply graph convolutional networks to image semantic segmentation, transforming pixel classification into a graph node classification problem.

Findings

Achieved about 1.34% improvement in mIOU on VOC dataset.

Successfully integrated graph models with FCN for better segmentation.

Demonstrated competitive performance compared to traditional FCN.

Abstract

Semantic segmentation with deep learning has achieved great progress in classifying the pixels in the image. However, the local location information is usually ignored in the high-level feature extraction by the deep learning, which is important for image semantic segmentation. To avoid this problem, we propose a graph model initialized by a fully convolutional network (FCN) named Graph-FCN for image semantic segmentation. Firstly, the image grid data is extended to graph structure data by a convolutional network, which transforms the semantic segmentation problem into a graph node classification problem. Then we apply graph convolutional network to solve this graph node classification problem. As far as we know, it is the first time that we apply the graph convolutional network in image semantic segmentation. Our method achieves competitive performance in mean intersection over union (mIOU) on the VOC dataset(about 1.34% improvement), compared to the original FCN model.