Synthesizing Products for Online Catalogs

TL;DR

This paper presents an end-to-end system for automated product synthesis in online catalogs, addressing challenges in data extraction, schema reconciliation, and data fusion, with a novel scalable schema matching technique trained without manual labels.

Contribution

The paper introduces a scalable, knowledge-based schema matching method for product synthesis that outperforms existing techniques and requires no manual labeling.

Findings

Successfully processed over 800K offers from 1000 merchants

Achieved higher precision and recall than state-of-the-art schema matching methods

Generated accurate product specifications for 400 categories

Abstract

A high-quality, comprehensive product catalog is essential to the success of Product Search engines and shopping sites such as Yahoo! Shopping, Google Product Search or Bing Shopping. But keeping catalogs up-to-date becomes a challenging task, calling for the need of automated techniques. In this paper, we introduce the problem of product synthesis, a key component of catalog creation and maintenance. Given a set of offers advertised by merchants, the goal is to identify new products and add them to the catalog together with their (structured) attributes. A fundamental challenge is the scale of the problem: a Product Search engine receives data from thousands of merchants and millions of products; the product taxonomy contains thousands of categories, where each category comes in a different schema; and merchants use representations for products that are different from the ones used in the catalog of the Product Search engine. We propose a system that provides an end-to-end solution to the product synthesis problem, and includes components for extraction, and addresses issues involved in data extraction from offers, schema reconciliation, and data fusion. We developed a novel and scalable technique for schema matching which leverages knowledge about previously-known instance-level associations between offers and products; and it is trained using automatically created training sets (no manually-labeled data is needed). We present an experimental evaluation of our system using data from Bing Shopping for more than 800K offers, a thousand merchants, and 400 categories. The evaluation confirms that our approach is able to automatically generate a large number of accurate product specifications, and that our schema reconciliation component outperforms state-of-the-art schema matching techniques in terms of precision and recall.