Hierarchical Knowledge Graph Construction from Images for Scalable E-Commerce

TL;DR

This paper introduces a novel automated method for constructing structured product knowledge graphs from images in e-commerce, leveraging vision-language and large language models, and provides a benchmark dataset for evaluation.

Contribution

It presents a new automated approach combining VLM and LLM for scalable knowledge graph construction from images, with a human-annotated dataset for benchmarking.

Findings

Outperforms baseline in all evaluated metrics.

Effectively automates knowledge graph construction from images.

Demonstrates potential for scalable e-commerce applications.

Abstract

Knowledge Graph (KG) is playing an increasingly important role in various AI systems. For e-commerce, an efficient and low-cost automated knowledge graph construction method is the foundation of enabling various successful downstream applications. In this paper, we propose a novel method for constructing structured product knowledge graphs from raw product images. The method cooperatively leverages recent advances in the vision-language model (VLM) and large language model (LLM), fully automating the process and allowing timely graph updates. We also present a human-annotated e-commerce product dataset for benchmarking product property extraction in knowledge graph construction. Our method outperforms our baseline in all metrics and evaluated properties, demonstrating its effectiveness and bright usage potential.