Using Fuzzy Logic to Leverage HTML Markup for Web Page Representation

TL;DR

This paper introduces AFCC, a fuzzy logic-based method that leverages HTML structure to improve web page clustering by identifying the most representative words, outperforming traditional TF-IDF in various datasets.

Contribution

The paper presents a novel fuzzy term weighing approach that exploits HTML markup to enhance document representation for clustering, adapting to different dataset feature distributions.

Findings

AFCC outperforms TF-IDF in web page clustering tasks.

The fuzzy combination approach adapts well to datasets with varying feature distributions.

Exploiting HTML structure improves the identification of representative words.

Abstract

The selection of a suitable document representation approach plays a crucial role in the performance of a document clustering task. Being able to pick out representative words within a document can lead to substantial improvements in document clustering. In the case of web documents, the HTML markup that defines the layout of the content provides additional structural information that can be further exploited to identify representative words. In this paper we introduce a fuzzy term weighing approach that makes the most of the HTML structure for document clustering. We set forth and build on the hypothesis that a good representation can take advantage of how humans skim through documents to extract the most representative words. The authors of web pages make use of HTML tags to convey the most important message of a web page through page elements that attract the readers' attention, such as page titles or emphasized elements. We define a set of criteria to exploit the information provided by these page elements, and introduce a fuzzy combination of these criteria that we evaluate within the context of a web page clustering task. Our proposed approach, called Abstract Fuzzy Combination of Criteria (AFCC), can adapt to datasets whose features are distributed differently, achieving good results compared to other similar fuzzy logic based approaches and TF-IDF across different datasets.