Query-Driven Knowledge Base Completion using Multimodal Path Fusion over Multimodal Knowledge Graph

TL;DR

This paper presents a query-driven system that fuses unstructured web data and structured knowledge base information through multimodal path fusion, significantly improving knowledge base completion accuracy and response speed.

Contribution

It introduces a novel multimodal path fusion algorithm and query-driven techniques for efficient knowledge base completion using multimodal knowledge graphs.

Findings

Outperforms baseline fusion algorithms in accuracy

Achieves faster response times for user queries

Demonstrates effectiveness through extensive experiments

Abstract

Over the past few years, large knowledge bases have been constructed to store massive amounts of knowledge. However, these knowledge bases are highly incomplete, for example, over 70% of people in Freebase have no known place of birth. To solve this problem, we propose a query-driven knowledge base completion system with multimodal fusion of unstructured and structured information. To effectively fuse unstructured information from the Web and structured information in knowledge bases to achieve good performance, our system builds multimodal knowledge graphs based on question answering and rule inference. We propose a multimodal path fusion algorithm to rank candidate answers based on different paths in the multimodal knowledge graphs, achieving much better performance than question answering, rule inference and a baseline fusion algorithm. To improve system efficiency, query-driven techniques are utilized to reduce the runtime of our system, providing fast responses to user queries. Extensive experiments have been conducted to demonstrate the effectiveness and efficiency of our system.