How Far are we from Data Mining Democratisation? A Systematic Review

TL;DR

This systematic review assesses the progress towards democratizing data mining, revealing that current approaches still require data scientists and face challenges in automation and accuracy, indicating that full democratization remains distant.

Contribution

The paper provides a comprehensive evaluation of existing data mining democratization approaches, identifying gaps and challenges that hinder full accessibility for non-experts.

Findings

Only two of four categories offer effective solutions.

One approach requires minimal data scientist intervention.

Automated methods face accuracy and completeness issues.

Abstract

Context: Data mining techniques have demonstrated to be a powerful technique for discovering insights hidden in data from a domain. However, these techniques demand very specialised skills. People willing to analyse data often lack these skills, so they must rely on data scientists, which hinders data mining democratisation. Different approaches have appeared in the last years to address this issue. Objective: Analyse the state of the art to know how far are we from an effective data mining democratisation, what has already been accomplished, and what should be done in the upcoming years. Method: We performed a state-of-the-art review following a systematic and objective procedure, which included works both from the academia and the industry. The reviewed works were grouped in four categories. Each category was then evaluated in detail using a well-defined evaluation criteria to identify its strengths and weaknesses. Results: Around 700 works were initially considered, from which 43 were finally selected for a more in-depth analysis. Only two out of the four identified categories provide effective solutions to data mining democratisation. From these two categories, one always requires a minimum intervention of a data scientist, whereas the other one does not provide support for all the stages of the data mining process, and might exhibit accuracy problems in some contexts. Conclusion: In all analysed approaches, a data scientist is still required to perform some steps of the analysis process. Moreover, automated approaches that do not require data scientists for some steps expose some problems in other quality attributes, such as accuracy. Therefore, although existent work shows some promising initial steps, we are still far from data mining democratisation.