GINet: Graph Interaction Network for Scene Parsing

TL;DR

GINet introduces a novel graph interaction mechanism and semantic loss to improve scene parsing by integrating linguistic knowledge, enhancing feature representations, and promoting semantic coherence in visual context reasoning.

Contribution

The paper proposes a Graph Interaction unit and Semantic Context Loss to incorporate linguistic knowledge into scene parsing, advancing context reasoning over image regions.

Findings

GINet outperforms state-of-the-art methods on Pascal-Context and COCO Stuff datasets.

The GI unit effectively enhances feature representations with high-level semantics.

SC-loss improves semantic coherence and discriminative capability.

Abstract

Recently, context reasoning using image regions beyond local convolution has shown great potential for scene parsing. In this work, we explore how to incorporate the linguistic knowledge to promote context reasoning over image regions by proposing a Graph Interaction unit (GI unit) and a Semantic Context Loss (SC-loss). The GI unit is capable of enhancing feature representations of convolution networks over high-level semantics and learning the semantic coherency adaptively to each sample. Specifically, the dataset-based linguistic knowledge is first incorporated in the GI unit to promote context reasoning over the visual graph, then the evolved representations of the visual graph are mapped to each local representation to enhance the discriminated capability for scene parsing. GI unit is further improved by the SC-loss to enhance the semantic representations over the exemplar-based semantic graph. We perform full ablation studies to demonstrate the effectiveness of each component in our approach. Particularly, the proposed GINet outperforms the state-of-the-art approaches on the popular benchmarks, including Pascal-Context and COCO Stuff.