Informetric Analyses of Knowledge Organization Systems (KOSs)

TL;DR

This paper reviews how quantitative informetric measures can evaluate the structure and quality of various types of knowledge organization systems, including ontologies and folksonomies.

Contribution

It introduces specific measures and indicators for analyzing the effectiveness and quality of different KOS approaches, filling a gap in systematic evaluation methods.

Findings

Most evaluation studies focus on ontologies.

Measures include groundedness, tangledness, fan-out, and granularity.

Indicators cover completeness, consistency, overlap, and use.

Abstract

A knowledge organization system (KOS) is made up of concepts and semantic relations between the concepts which represent a knowledge domain terminologically. We distinguish between five approaches to KOSs: nomenclatures, classification systems, thesauri, ontologies and, as a borderline case of KOSs, folksonomies. The research question of this paper is: How can we informetrically analyze the effectiveness of KOSs? Quantitative informetric measures and indicators allow for the description, for comparative analyses as well as for evaluation of KOSs and their quality. We describe the state of the art of KOS evaluation. Most of the evaluation studies found in the literature are about ontologies. We introduce measures of the structure of KOSs (e.g., groundedness, tangledness, fan-out factor, or granularity) and indicators of KOS quality (completeness, consistency, overlap, and use).