Evolving a Stigmergic Self-Organized Data-Mining

TL;DR

This paper introduces a novel stigmergy-based data-mining approach that combines swarm intelligence and evolutionary computation to create adaptive, self-organized systems, demonstrated through real-time web usage data analysis.

Contribution

It proposes a new stigmergic data-mining paradigm integrating bio-inspired algorithms for adaptive, distributed analysis, applied to large-scale web data.

Findings

System shows promising results compared to recent methods.

Effective real-time web usage data analysis achieved.

Demonstrates the potential of stigmergic self-organization in data-mining.

Abstract

Self-organizing complex systems typically are comprised of a large number of frequently similar components or events. Through their process, a pattern at the global-level of a system emerges solely from numerous interactions among the lower-level components of the system. Moreover, the rules specifying interactions among the system's components are executed using only local information, without reference to the global pattern, which, as in many real-world problems is not easily accessible or possible to be found. Stigmergy, a kind of indirect communication and learning by the environment found in social insects is a well know example of self-organization, providing not only vital clues in order to understand how the components can interact to produce a complex pattern, as can pinpoint simple biological non-linear rules and methods to achieve improved artificial intelligent adaptive categorization systems, critical for Data-Mining. On the present work it is our intention to show that a new type of Data-Mining can be designed based on Stigmergic paradigms, taking profit of several natural features of this phenomenon. By hybridizing bio-inspired Swarm Intelligence with Evolutionary Computation we seek for an entire distributed, adaptive, collective and cooperative self-organized Data-Mining. As a real-world, real-time test bed for our proposal, World-Wide-Web Mining will be used. Having that purpose in mind, Web usage Data was collected from the Monash University's Web site (Australia), with over 7 million hits every week. Results are compared to other recent systems, showing that the system presented is by far promising.