Improving Biomedical Knowledge Graph Quality: A Community Approach

TL;DR

This paper proposes evaluation criteria based on data standards to improve transparency, consistency, and reusability of biomedical knowledge graphs, which currently vary widely in quality and documentation.

Contribution

It introduces a set of community-driven evaluation criteria for biomedical KGs and demonstrates their application across multiple resources to promote standardization.

Findings

Many biomedical KGs lack essential information for reuse.

Significant variation exists in models and terminology among KGs.

Adoption of shared standards can enhance transparency and comparability.

Abstract

Biomedical knowledge graphs (KGs) are widely used across research and translational settings, yet their design decisions and implementation are often opaque. Unlike ontologies that more frequently adhere to established creation principles, biomedical KGs lack consistent practices for construction, documentation, and dissemination. To address this gap, we introduce a set of evaluation criteria grounded in widely accepted data standards and principles from related fields. We apply these criteria to 16 biomedical KGs, revealing that even those that appear to align with best practices often obscure essential information required for external reuse. Moreover, biomedical KGs, despite pursuing similar goals and ingesting the same sources in some cases, display substantial variation in models, source integration, and terminology for node types. Reaping the potential benefits of knowledge graphs for biomedical research while reducing wasted effort requires community-wide adoption of shared criteria and maturation of standards such as BioLink and KGX. Such improvements in transparency and standardization are essential for creating long-term reusability, improving comparability across resources, and enhancing the overall utility of KGs within biomedicine.