Predicting Query-Item Relationship using Adversarial Training and Robust Modeling Techniques

TL;DR

This paper introduces a robust deep learning approach combining transformers, LSTMs, and adversarial training techniques to predict query-item relationships, especially for unseen queries, achieving top performance in a competitive benchmark.

Contribution

It presents a novel combination of pre-trained transformers, LSTMs, and multiple robustness techniques for query-item relationship prediction, applicable across various deep learning tasks.

Findings

Achieved 10th place in KDD Cup 2022 Product Substitution Classification

Demonstrated effectiveness of adversarial training and ensemble methods in robustness

Validated strategies applicable to other deep learning applications

Abstract

We present an effective way to predict search query-item relationship. We combine pre-trained transformer and LSTM models, and increase model robustness using adversarial training, exponential moving average, multi-sampled dropout, and diversity based ensemble, to tackle an extremely difficult problem of predicting against queries not seen before. All of our strategies focus on increasing robustness of deep learning models and are applicable in any task where deep learning models are used. Applying our strategies, we achieved 10th place in KDD Cup 2022 Product Substitution Classification task.