ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine

TL;DR

This paper introduces ProphetNet-Ads, a novel strategy for generative retrieval in sponsored search engines that enhances retrieval accuracy by optimizing Trie-constrained search space, addressing limitations of previous models.

Contribution

It proposes a looking ahead strategy for generative retrieval models that directly optimizes Trie-constrained search space, improving recall in sponsored search applications.

Findings

Recall improved by 15.58% with single model

Recall improved by 18.8% with integrated model

Effective alleviation of Trie search problems

Abstract

In a sponsored search engine, generative retrieval models are recently proposed to mine relevant advertisement keywords for users' input queries. Generative retrieval models generate outputs token by token on a path of the target library prefix tree (Trie), which guarantees all of the generated outputs are legal and covered by the target library. In actual use, we found several typical problems caused by Trie-constrained searching length. In this paper, we analyze these problems and propose a looking ahead strategy for generative retrieval models named ProphetNet-Ads. ProphetNet-Ads improves the retrieval ability by directly optimizing the Trie-constrained searching space. We build a dataset from a real-word sponsored search engine and carry out experiments to analyze different generative retrieval models. Compared with Trie-based LSTM generative retrieval model proposed recently, our single model result and integrated result improve the recall by 15.58\% and 18.8\% respectively with beam size 5. Case studies further demonstrate how these problems are alleviated by ProphetNet-Ads clearly.