Toward a Distributed Knowledge Discovery system for Grid systems

TL;DR

This paper proposes a distributed knowledge discovery system for Grid environments that combines data-driven and architecture-driven strategies to efficiently process large, heterogeneous datasets using Grid middleware tools.

Contribution

It introduces a novel distributed data mining system integrating dataset-driven and architecture-driven approaches within a Grid middleware framework.

Findings

Enhances data processing efficiency for large, heterogeneous datasets.

Supports dynamic and autonomous data mining operations.

Integrates large data manipulation operations in a Grid environment.

Abstract

During the last decade or so, we have had a deluge of data from not only science fields but also industry and commerce fields. Although the amount of data available to us is constantly increasing, our ability to process it becomes more and more difficult. Efficient discovery of useful knowledge from these datasets is therefore becoming a challenge and a massive economic need. This led to the need of developing large-scale data mining (DM) techniques to deal with these huge datasets either from science or economic applications. In this chapter, we present a new DDM system combining dataset-driven and architecture-driven strategies. Data-driven strategies will consider the size and heterogeneity of the data, while architecture driven will focus on the distribution of the datasets. This system is based on a Grid middleware tools that integrate appropriate large data manipulation operations. Therefore, this allows more dynamicity and autonomicity during the mining, integrating and processing phases