An innovative data collection method to eliminate the preprocessing phase in web usage mining

TL;DR

This paper introduces a novel data collection method for web usage mining that eliminates the need for preprocessing server logs, enabling more efficient and reliable web analytics and user session management.

Contribution

It proposes an innovative approach and API for direct data collection from enterprise web applications, improving data quality and usability for web analytics.

Findings

Collected data is more structured and easier to process than server logs.

The method enables real-time web analytics and recommendation systems.

Data stored in relational databases enhances mining performance.

Abstract

The underlying data source for web usage mining (WUM) is commonly thought to be server logs. However, access log files ensure quite limited data about the clients. Identifying sessions from this messy data takes a considerable effort, and operations performed for this purpose do not always yield excellent results. Also, this data cannot be used for web analytics efficiently. This study proposes an innovative method for user tracking, session management, and collecting web usage data. The method is mainly based on a new approach for using collected data for web analytics extraction as the data source in web usage mining. An application-based API has been developed with a different strategy from conventional client-side methods to obtain and process log data. The log data has been successfully gathered by integrating the technique into an enterprise web application. The results reveal that the homogeneous structured data collected and stored with this method is more convenient to browse, filter, and process than web server logs. This data stored on a relational database can be used effortlessly as a reliable data source for high-performance web usage mining activity, real-time web analytics, or a functional recommendation system.