Quantitative Evaluation of Performance and Validity Indices for Clustering the Web Navigational Sessions

TL;DR

This paper evaluates various performance and validity indices for clustering web navigational sessions, comparing multiple clustering algorithms to determine their effectiveness in capturing user behavior.

Contribution

It provides a comprehensive comparison of clustering techniques and validity measures specifically applied to web usage data, highlighting their relative performance.

Findings

Different clustering algorithms yield varying cluster quality.

Validity indices like Dunn's and Silhouette effectively evaluate cluster quality.

Performance results help select suitable clustering methods for web usage mining.

Abstract

Clustering techniques are widely used in Web Usage Mining to capture similar interests and trends among users accessing a Web site. For this purpose, web access logs generated at a particular web site are preprocessed to discover the user navigational sessions. Clustering techniques are then applied to group the user session data into user session clusters, where intercluster similarities are minimized while the intra cluster similarities are maximized. Since the application of different clustering algorithms generally results in different sets of cluster formation, it is important to evaluate the performance of these methods in terms of accuracy and validity of the clusters, and also the time required to generate them, using appropriate performance measures. This paper describes various validity and accuracy measures including Dunn's Index, Davies Bouldin Index, C Index, Rand Index, Jaccard Index, Silhouette Index, Fowlkes Mallows and Sum of the Squared Error (SSE). We conducted the performance evaluation of the following clustering techniques: k-Means, k-Medoids, Leader, Single Link Agglomerative Hierarchical and DBSCAN. These techniques are implemented and tested against the Web user navigational data. Finally their performance results are presented and compared.