Similarity based Dynamic Web Data Extraction and Integration System from Search Engine Result Pages for Web Content Mining

TL;DR

This paper introduces two similarity-based mechanisms, WDES and WDICS, for extracting and integrating web content from search engine result pages to improve web content mining efficiency.

Contribution

The paper presents novel similarity-based methods for extracting and integrating SERP data, outperforming existing techniques like DEPTA in precision and recall.

Findings

WDES and WDICS outperform DEPTA in precision and recall

Effective extraction and integration of SERP data for web content mining

Enhanced offline browsing and analysis capabilities

Abstract

There is an explosive growth of information in the World Wide Web thus posing a challenge to Web users to extract essential knowledge from the Web. Search engines help us to narrow down the search in the form of Search Engine Result Pages (SERP). Web Content Mining is one of the techniques that help users to extract useful information from these SERPs. In this paper, we propose two similarity based mechanisms; WDES, to extract desired SERPs and store them in the local depository for offline browsing and WDICS, to integrate the requested contents and enable the user to perform the intended analysis and extract the desired information. Our experimental results show that WDES and WDICS outperform DEPTA [1] in terms of Precision and Recall.