Graph integration of structured, semistructured and unstructured data for data journalism

TL;DR

This paper presents a comprehensive system for integrating diverse data types—structured, semi-structured, and unstructured—into a unified graph framework to support data journalism, addressing scalability and usability challenges.

Contribution

It introduces a novel approach and system, ConnectionLens, for scalable integration of heterogeneous datasets into graphs tailored for non-IT experts in journalism.

Findings

Successful implementation of ConnectionLens system

Effective handling of dynamic heterogeneous data sources

Validated scalability and usability through experiments

Abstract

Digital data is a gold mine for modern journalism. However, datasets which interest journalists are extremely heterogeneous, ranging from highly structured (relational databases), semi-structured (JSON, XML, HTML), graphs (e.g., RDF), and text. Journalists (and other classes of users lacking advanced IT expertise, such as most non-governmental-organizations, or small public administrations) need to be able to make sense of such heterogeneous corpora, even if they lack the ability to define and deploy custom extract-transform-load workflows, especially for dynamically varying sets of data sources. We describe a complete approach for integrating dynamic sets of heterogeneous datasets along the lines described above: the challenges we faced to make such graphs useful, allow their integration to scale, and the solutions we proposed for these problems. Our approach is implemented within the ConnectionLens system; we validate it through a set of experiments.