SGDraw: Scene Graph Drawing Interface Using Object-Oriented Representation

TL;DR

SGDraw is an interactive web-based interface that uses object-oriented scene graph representation to improve the accuracy and detail of scene graph annotations for computer vision applications.

Contribution

It introduces a novel object-oriented scene graph drawing interface that enhances user interaction and detail in scene graph annotation compared to traditional methods.

Findings

SGDraw produces richer scene graphs with more details.

Users find SGDraw easier and more effective for scene graph annotation.

SGDraw improves the accuracy of scene understanding in applications.

Abstract

Scene understanding is an essential and challenging task in computer vision. To provide the visually fundamental graphical structure of an image, the scene graph has received increased attention due to its powerful semantic representation. However, it is difficult to draw a proper scene graph for image retrieval, image generation, and multi-modal applications. The conventional scene graph annotation interface is not easy to use in image annotations, and the automatic scene graph generation approaches using deep neural networks are prone to generate redundant content while disregarding details. In this work, we propose SGDraw, a scene graph drawing interface using object-oriented scene graph representation to help users draw and edit scene graphs interactively. For the proposed object-oriented representation, we consider the objects, attributes, and relationships of objects as a structural unit. SGDraw provides a web-based scene graph annotation and generation tool for scene understanding applications. To verify the effectiveness of the proposed interface, we conducted a comparison study with the conventional tool and the user experience study. The results show that SGDraw can help generate scene graphs with richer details and describe the images more accurately than traditional bounding box annotations. We believe the proposed SGDraw can be useful in various vision tasks, such as image retrieval and generation.