A Framework for Leveraging Human Computation Gaming to Enhance Knowledge Graphs for Accuracy Critical Generative AI Applications

TL;DR

This paper introduces GAME-KG, a federated framework that uses human feedback via video games to improve knowledge graphs, especially for critical AI applications requiring explainability, demonstrated through real-world and simulated scenarios.

Contribution

The paper presents a novel framework that leverages crowdsourced gaming feedback to enhance both explicit and implicit connections in knowledge graphs for improved AI explainability.

Findings

GAME-KG effectively enhances knowledge graph accuracy.

Crowdsourced feedback improves implicit connection detection.

Framework supports explainability in sensitive domains.

Abstract

External knowledge graphs (KGs) can be used to augment large language models (LLMs), while simultaneously providing an explainable knowledge base of facts that can be inspected by a human. This approach may be particularly valuable in domains where explainability is critical, like human trafficking data analysis. However, creating KGs can pose challenges. KGs parsed from documents may comprise explicit connections (those directly stated by a document) but miss implicit connections (those obvious to a human although not directly stated). To address these challenges, this preliminary research introduces the GAME-KG framework, standing for "Gaming for Augmenting Metadata and Enhancing Knowledge Graphs." GAME-KG is a federated approach to modifying explicit as well as implicit connections in KGs by using crowdsourced feedback collected through video games. GAME-KG is shown through two demonstrations: a Unity test scenario from Dark Shadows, a video game that collects feedback on KGs parsed from US Department of Justice (DOJ) Press Releases on human trafficking, and a following experiment where OpenAI's GPT-4 is prompted to answer questions based on a modified and unmodified KG. Initial results suggest that GAME-KG can be an effective framework for enhancing KGs, while simultaneously providing an explainable set of structured facts verified by humans.