Implications of construction decisions in keyword-based networks: an empirical assessment

TL;DR

This paper examines how early data preprocessing choices significantly influence the analysis outcomes of keyword-based networks derived from semi-structured online data, emphasizing the importance of transparency.

Contribution

It provides an empirical assessment of the impact of construction decisions on network analysis results, highlighting the need for transparent preprocessing reporting.

Findings

High sensitivity of network metrics to preprocessing choices

Preprocessing decisions significantly affect analysis outcomes

Highlights importance of transparent data lineage reporting

Abstract

The large amounts of data continuously generated online offer opportunities to identify and analyse trends in various aspects of society. For instance, data from online social media are frequently used as a means of analysing informal interactions, opinions, and feelings of groups of people. Additionally, bibliometric data can be used to investigate more formal trends that occur in scientific research. A popular approach to analysing such complex semi-structured data is the construction of complex networks based on keywords or concept extraction. However, such keyword-based complex network data are often shared in a preprocessed form, with little information about the underlying process used to construct it. Indeed, key decisions are normally made at an early stage in the construction of complex networks from raw data, and can have a significant impact on subsequent analysis and interpretation. In this paper, we highlight the sensitivity of results to data preprocessing decisions by looking at two different case studies which employ networks constructed from underlying semi-structured data. The experiments conducted show high sensitivity to data preprocessing for many commonly adopted metrics. These results demonstrate the need for transparent reporting of data lineage and preprocessing decisions.