Web Data Extraction, Applications and Techniques: A Survey

TL;DR

This survey provides a comprehensive overview of Web Data Extraction techniques, classifying applications into enterprise and social web domains, highlighting their uses, challenges, and potential for cross-domain reuse.

Contribution

It offers a structured classification framework and discusses the application of Web Data Extraction across different domains, emphasizing cross-fertilization opportunities.

Findings

Web Data Extraction is crucial for enterprise and social web applications.

Techniques are often domain-specific but can be reused across domains.

Extraction enables large-scale analysis of human behavior and business intelligence.

Abstract

Web Data Extraction is an important problem that has been studied by means of different scientific tools and in a broad range of applications. Many approaches to extracting data from the Web have been designed to solve specific problems and operate in ad-hoc domains. Other approaches, instead, heavily reuse techniques and algorithms developed in the field of Information Extraction. This survey aims at providing a structured and comprehensive overview of the literature in the field of Web Data Extraction. We provided a simple classification framework in which existing Web Data Extraction applications are grouped into two main classes, namely applications at the Enterprise level and at the Social Web level. At the Enterprise level, Web Data Extraction techniques emerge as a key tool to perform data analysis in Business and Competitive Intelligence systems as well as for business process re-engineering. At the Social Web level, Web Data Extraction techniques allow to gather a large amount of structured data continuously generated and disseminated by Web 2.0, Social Media and Online Social Network users and this offers unprecedented opportunities to analyze human behavior at a very large scale. We discuss also the potential of cross-fertilization, i.e., on the possibility of re-using Web Data Extraction techniques originally designed to work in a given domain, in other domains.