Symbolic Graph Inference for Compound Scene Understanding

TL;DR

This paper introduces a novel symbolic graph inference method for scene understanding that reasons over scene and knowledge graphs to interpret complex scenes, demonstrating promising results on the ADE20K dataset.

Contribution

The paper presents a new graph-based reasoning approach that explicitly models scene composition and domain knowledge for improved scene understanding.

Findings

Feasibility demonstrated on ADE20K dataset

Outperforms some existing scene understanding methods

Utilizes joint graph search for reasoning

Abstract

Scene understanding is a fundamental capability needed in many domains, ranging from question-answering to robotics. Unlike recent end-to-end approaches that must explicitly learn varying compositions of the same scene, our method reasons over their constituent objects and analyzes their arrangement to infer a scene's meaning. We propose a novel approach that reasons over a scene's scene- and knowledge-graph, capturing spatial information while being able to utilize general domain knowledge in a joint graph search. Empirically, we demonstrate the feasibility of our method on the ADE20K dataset and compare it to current scene understanding approaches.