A Pipeline for Business Intelligence and Data-Driven Root Cause Analysis on Categorical Data

TL;DR

This paper introduces a new data mining pipeline combining clustering and association rule mining to extract business insights and facilitate root cause analysis from categorical data, aiding decision-making and model development.

Contribution

It presents a novel pipeline that integrates clustering with association rule mining to improve business intelligence and root cause analysis from categorical datasets.

Findings

Generated association rules with metrics for business insights

Enabled data-driven root cause analysis

Supported decision-making and model updates

Abstract

Business intelligence (BI) is any knowledge derived from existing data that may be strategically applied within a business. Data mining is a technique or method for extracting BI from data using statistical data modeling. Finding relationships or correlations between the various data items that have been collected can be used to boost business performance or at the very least better comprehend what is going on. Root cause analysis (RCA) is discovering the root causes of problems or events to identify appropriate solutions. RCA can show why an event occurred and this can help in avoiding occurrences of an issue in the future. This paper proposes a new clustering + association rule mining pipeline for getting business insights from data. The results of this pipeline are in the form of association rules having consequents, antecedents, and various metrics to evaluate these rules. The results of this pipeline can help in anchoring important business decisions and can also be used by data scientists for updating existing models or while developing new ones. The occurrence of any event is explained by its antecedents in the generated rules. Hence this output can also help in data-driven root cause analysis.