A novel approach for fast mining frequent itemsets use N-list structure based on MapReduce

TL;DR

This paper introduces HPrepost, a MapReduce-based algorithm that efficiently mines frequent itemsets with reduced runtime and memory, especially effective on dense, large datasets.

Contribution

It presents an improved Prepost algorithm utilizing Hadoop and MapReduce for faster, more memory-efficient frequent itemset mining on large datasets.

Findings

HPrepost outperforms existing algorithms in runtime.

HPrepost uses less memory on dense datasets.

Effective with large datasets and small support thresholds.

Abstract

Frequent Pattern Mining is a one field of the most significant topics in data mining. In recent years, many algorithms have been proposed for mining frequent itemsets. A new algorithm has been presented for mining frequent itemsets based on N-list data structure called Prepost algorithm. The Prepost algorithm is enhanced by implementing compact PPC-tree with the general tree. Prepost algorithm can only find a frequent itemsets with required (pre-order and post-order) for each node. In this chapter, we improved prepost algorithm based on Hadoop platform (HPrepost), proposed using the Mapreduce programming model. The main goals of proposed method are efficient mining frequent itemsets requiring less running time and memory usage. We have conduct experiments for the proposed scheme to compare with another algorithms. With dense datasets, which have a large average length of transactions, HPrepost is more effective than frequent itemsets algorithms in terms of execution time and memory usage for all min-sup. Generally, our algorithm outperforms algorithms in terms of runtime and memory usage with small thresholds and large datasets.