Machine Learning-Friendly Biomedical Datasets for Equivalence and Subsumption Ontology Matching

TL;DR

This paper introduces five new biomedical ontology matching tasks with high-quality reference mappings, supporting the evaluation of ML-based and traditional systems, and provides a comprehensive framework for assessing OM performance.

Contribution

It presents new biomedical OM tasks, improved reference mappings, and a comprehensive evaluation framework addressing limitations of previous benchmarks.

Findings

Evaluation results demonstrate the utility of new resources.

Support for ML-based OM system assessment is enhanced.

Benchmark resources are publicly available for community use.

Abstract

Ontology Matching (OM) plays an important role in many domains such as bioinformatics and the Semantic Web, and its research is becoming increasingly popular, especially with the application of machine learning (ML) techniques. Although the Ontology Alignment Evaluation Initiative (OAEI) represents an impressive effort for the systematic evaluation of OM systems, it still suffers from several limitations including limited evaluation of subsumption mappings, suboptimal reference mappings, and limited support for the evaluation of ML-based systems. To tackle these limitations, we introduce five new biomedical OM tasks involving ontologies extracted from Mondo and UMLS. Each task includes both equivalence and subsumption matching; the quality of reference mappings is ensured by human curation, ontology pruning, etc.; and a comprehensive evaluation framework is proposed to measure OM performance from various perspectives for both ML-based and non-ML-based OM systems. We report evaluation results for OM systems of different types to demonstrate the usage of these resources, all of which are publicly available as part of the new BioML track at OAEI 2022.