Mixed Membership Recurrent Neural Networks

TL;DR

This paper introduces a mixed membership RNN model that captures group-level effects and varying time intervals in sequential data, enabling dynamic topic modeling and improved analysis of large-scale online shopping sequences.

Contribution

It presents a novel RNN-based model incorporating group-level parameters and mixed membership concepts for better handling of grouped, irregularly timed sequential data.

Findings

Effective modeling of group effects in sequential data.

Application to large-scale online shopping data.

Demonstrated dynamic topic evolution over time.

Abstract

Models for sequential data such as the recurrent neural network (RNN) often implicitly model a sequence as having a fixed time interval between observations and do not account for group-level effects when multiple sequences are observed. We propose a model for grouped sequential data based on the RNN that accounts for varying time intervals between observations in a sequence by learning a group-level base parameter to which each sequence can revert. Our approach is motivated by the mixed membership framework, and we show how it can be used for dynamic topic modeling in which the distribution on topics (not the topics themselves) are evolving in time. We demonstrate our approach on a dataset of 3.4 million online grocery shopping orders made by 206K customers.