AspectMMKG: A Multi-modal Knowledge Graph with Aspect-aware Entities

TL;DR

This paper introduces AspectMMKG, a multi-modal knowledge graph with aspect-specific images for entities, enhancing understanding and enabling improved entity aspect linking through a novel image retrieval model.

Contribution

It constructs the first aspect-aware MMKG with aspect-related images and proposes an image retrieval model to expand and refine aspect-specific images.

Findings

AspectMMKG contains 2,380 entities and 18,139 aspects.

The proposed AIR model effectively retrieves suitable images for different entity aspects.

Using AspectMMKG improves performance on entity aspect linking tasks.

Abstract

Multi-modal knowledge graphs (MMKGs) combine different modal data (e.g., text and image) for a comprehensive understanding of entities. Despite the recent progress of large-scale MMKGs, existing MMKGs neglect the multi-aspect nature of entities, limiting the ability to comprehend entities from various perspectives. In this paper, we construct AspectMMKG, the first MMKG with aspect-related images by matching images to different entity aspects. Specifically, we collect aspect-related images from a knowledge base, and further extract aspect-related sentences from the knowledge base as queries to retrieve a large number of aspect-related images via an online image search engine. Finally, AspectMMKG contains 2,380 entities, 18,139 entity aspects, and 645,383 aspect-related images. We demonstrate the usability of AspectMMKG in entity aspect linking (EAL) downstream task and show that previous EAL models achieve a new state-of-the-art performance with the help of AspectMMKG. To facilitate the research on aspect-related MMKG, we further propose an aspect-related image retrieval (AIR) model, that aims to correct and expand aspect-related images in AspectMMKG. We train an AIR model to learn the relationship between entity image and entity aspect-related images by incorporating entity image, aspect, and aspect image information. Experimental results indicate that the AIR model could retrieve suitable images for a given entity w.r.t different aspects.