Darth Vecdor: An Open-Source System for Generating Knowledge Graphs Through Large Language Model Queries

TL;DR

Darth Vecdor is an open-source system that extracts knowledge from large language models into a structured, queryable database to improve efficiency, safety, and confidence in high-volume applications like healthcare.

Contribution

The paper introduces Darth Vecdor, a novel open-source system that addresses issues in extracting and structuring knowledge from LLMs into a SQL database with features for error mitigation and user-friendly interface.

Findings

Successfully extracts knowledge into a structured database

Mitigates errors and inconsistencies in LLM responses

Provides a user-friendly, browser-based interface

Abstract

Many large language models (LLMs) are trained on a massive body of knowledge present on the Internet. Darth Vecdor (DV) was designed to extract this knowledge into a structured, terminology-mapped, SQL database ("knowledge base" or "knowledge graph"). Knowledge graphs may be useful in many domains, including healthcare. Although one might query an LLM directly rather than a SQL-based knowledge graph, concerns such as cost, speed, safety, and confidence may arise, especially in high-volume operations. These may be mitigated when the information is pre-extracted from the LLM and becomes query-able through a standard database. However, the author found the need to address several issues. These included erroneous, off-topic, free-text, overly general, and inconsistent LLM responses, as well as allowing for multi-element responses. DV was built with features intended to mitigate these issues. To facilitate ease of use, and to allow for prompt engineering by those with domain expertise but little technical background, DV provides a simple, browser-based graphical user interface. DV has been released as free, open-source, extensible software, on an "as is" basis, without warranties or conditions of any kind, either express or implied. Users need to be cognizant of the potential risks and benefits of using DV and its outputs, and users are responsible for ensuring any use is safe and effective. DV should be assumed to have bugs, potentially very serious ones. However, the author hopes that appropriate use of current and future versions of DV and its outputs can help improve healthcare.