VisionKG: Unleashing the Power of Visual Datasets via Knowledge Graph

TL;DR

VisionKG is a knowledge graph framework that unifies and manages diverse visual datasets using Semantic Web technologies, enabling easier access, querying, and semantic enrichment for computer vision research.

Contribution

It introduces a knowledge-based approach to organize visual datasets, linking 40 million entities across 30 datasets, surpassing existing metadata-based methods.

Findings

Contains 519 million RDF triples describing 40 million entities

Integrates 30 datasets and 4 CV tasks for versatile applications

Enables semantic enrichment and efficient querying of visual data

Abstract

The availability of vast amounts of visual data with heterogeneous features is a key factor for developing, testing, and benchmarking of new computer vision (CV) algorithms and architectures. Most visual datasets are created and curated for specific tasks or with limited image data distribution for very specific situations, and there is no unified approach to manage and access them across diverse sources, tasks, and taxonomies. This not only creates unnecessary overheads when building robust visual recognition systems, but also introduces biases into learning systems and limits the capabilities of data-centric AI. To address these problems, we propose the Vision Knowledge Graph (VisionKG), a novel resource that interlinks, organizes and manages visual datasets via knowledge graphs and Semantic Web technologies. It can serve as a unified framework facilitating simple access and querying of state-of-the-art visual datasets, regardless of their heterogeneous formats and taxonomies. One of the key differences between our approach and existing methods is that ours is knowledge-based rather than metadatabased. It enhances the enrichment of the semantics at both image and instance levels and offers various data retrieval and exploratory services via SPARQL. VisionKG currently contains 519 million RDF triples that describe approximately 40 million entities, and are accessible at https://vision.semkg.org and through APIs. With the integration of 30 datasets and four popular CV tasks, we demonstrate its usefulness across various scenarios when working with CV pipelines.