Cross-Domain Web Information Extraction at Pinterest

TL;DR

This paper presents Pinterest's scalable, cost-effective system for extracting structured product data from e-commerce websites, leveraging a novel multi-modal webpage representation that enables simple models to outperform complex LLMs.

Contribution

Introduction of a novel webpage representation combining structural, visual, and text data, enabling small models to achieve high accuracy in attribute extraction.

Findings

Achieves over 1,000 URLs per second processing speed.

Cost is 1000 times lower than GPT-based methods.

Outperforms complex LLMs like GPT in attribute extraction accuracy.

Abstract

The internet offers a massive repository of unstructured information, but it's a significant challenge to convert this into a structured format. At Pinterest, the ability to accurately extract structured product data from e-commerce websites is essential to enhance user experiences and improve content distribution. In this paper, we present Pinterest's system for attribute extraction, which achieves remarkable accuracy and scalability at a manageable cost. Our approach leverages a novel webpage representation that combines structural, visual, and text modalities into a compact form, optimizing it for small model learning. This representation captures each visible HTML node with its text, style and layout information. We show how this allows simple models such as eXtreme Gradient Boosting (XGBoost) to extract attributes more accurately than much more complex Large Language Models (LLMs) such as Generative Pre-trained Transformer (GPT). Our results demonstrate a system that is highly scalable, processing over 1,000 URLs per second, while being 1000 times more cost-effective than the cheapest GPT alternatives.