Sequence-to-Set Generative Models

TL;DR

This paper introduces a sequence-to-set framework that transforms sequence generative models into set models, enabling evaluation of set utility and improving learning from order data with a size-bias technique.

Contribution

The paper proposes a novel sequence-to-set method, including GRU2Set and SetNN, with an importance sampling algorithm and size-bias trick for better set distribution learning.

Findings

Models outperform baselines in order data learning.

Size-bias trick consistently improves set distribution quality.

Effective on e-commerce datasets TMALL and HKTVMALL.

Abstract

In this paper, we propose a sequence-to-set method that can transform any sequence generative model based on maximum likelihood to a set generative model where we can evaluate the utility/probability of any set. An efficient importance sampling algorithm is devised to tackle the computational challenge of learning our sequence-to-set model. We present GRU2Set, which is an instance of our sequence-to-set method and employs the famous GRU model as the sequence generative model. To further obtain permutation invariant representation of sets, we devise the SetNN model which is also an instance of the sequence-to-set model. A direct application of our models is to learn an order/set distribution from a collection of e-commerce orders, which is an essential step in many important operational decisions such as inventory arrangement for fast delivery. Based on the intuition that small-sized sets are usually easier to learn than large sets, we propose a size-bias trick that can help learn better set distributions with respect to the $\ell_1$-distance evaluation metric. Two e-commerce order datasets, TMALL and HKTVMALL, are used to conduct extensive experiments to show the effectiveness of our models. The experimental results demonstrate that our models can learn better set/order distributions from order data than the baselines. Moreover, no matter what model we use, applying the size-bias trick can always improve the quality of the set distribution learned from data.