Quality Assessment of Linked Datasets using Probabilistic Approximation

TL;DR

This paper introduces a probabilistic approach to efficiently approximate data quality metrics for large, evolving Linked Open Datasets, enabling scalable quality assessment within the Luzzu framework.

Contribution

It presents a novel integration of probabilistic techniques like Reservoir Sampling and Bloom Filters for approximate quality metric computation in linked data.

Findings

High accuracy of approximate metrics demonstrated

Significant reduction in computation time achieved

Effective handling of large, dynamic datasets

Abstract

With the increasing application of Linked Open Data, assessing the quality of datasets by computing quality metrics becomes an issue of crucial importance. For large and evolving datasets, an exact, deterministic computation of the quality metrics is too time consuming or expensive. We employ probabilistic techniques such as Reservoir Sampling, Bloom Filters and Clustering Coefficient estimation for implementing a broad set of data quality metrics in an approximate but sufficiently accurate way. Our implementation is integrated in the comprehensive data quality assessment framework Luzzu. We evaluated its performance and accuracy on Linked Open Datasets of broad relevance.