Extraction of Relevant Images for Boilerplate Removal in Web Browsers

TL;DR

This paper presents a method to extract relevant images from webpages for boilerplate removal by using a headless browser to obtain layout information, improving content focus in reader modes.

Contribution

It introduces a novel framework and classifier leveraging headless browser rendering to accurately identify relevant images for boilerplate removal.

Findings

Effective extraction of relevant images demonstrated

Improved accuracy over heuristic methods

Framework suitable for dynamic webpage content

Abstract

Boilerplate refers to unwanted and repeated parts of a webpage (such as ads or table of contents) that distracts the user from reading the core content of the webpage, such as a news article. Accurate detection and removal of boilerplate content from a webpage can enable the users to have a clutter free view of the webpage or news article. This can be useful in features like reader mode in web browsers. Current implementations of reader mode in web browsers such as Firefox, Chrome and Edge perform reasonably well for textual content in webpages. However, they are mostly heuristic based and not flexible when the webpage content is dynamic. Also they often do not perform well for removing boilerplate content in the form of images and multimedia in webpages. For detection of boilerplate images, one needs to have knowledge of the actual layout of the images in the webpage, which is only possible when the webpage is rendered. In this paper we discuss some of the issues in relevant image extraction. We also present the design of a testing framework to measure accuracy and a classifier to extract relevant images by leveraging a headless browser solution that gives the rendering information for images.