NeSy4VRD: A Multifaceted Resource for Neurosymbolic AI Research using Knowledge Graphs in Visual Relationship Detection

TL;DR

NeSy4VRD is a comprehensive resource combining an improved VRD dataset, a domain-specific OWL ontology, and open-source tools to advance neurosymbolic AI research in visual relationship detection.

Contribution

It re-establishes access to VRD images, provides enhanced annotations, and offers an ontology and tools for integrating visual data with knowledge graphs, fostering neurosymbolic AI development.

Findings

Enhanced VRD dataset with quality annotations

Open-source tools for knowledge graph integration

Ontology supporting extensibility and research use

Abstract

NeSy4VRD is a multifaceted resource designed to support the development of neurosymbolic AI (NeSy) research. NeSy4VRD re-establishes public access to the images of the VRD dataset and couples them with an extensively revised, quality-improved version of the VRD visual relationship annotations. Crucially, NeSy4VRD provides a well-aligned, companion OWL ontology that describes the dataset domain.It comes with open source infrastructure that provides comprehensive support for extensibility of the annotations (which, in turn, facilitates extensibility of the ontology), and open source code for loading the annotations to/from a knowledge graph. We are contributing NeSy4VRD to the computer vision, NeSy and Semantic Web communities to help foster more NeSy research using OWL-based knowledge graphs.