Event Representation with Sequential, Semi-Supervised Discrete Variables

TL;DR

This paper introduces a semi-supervised neural variational autoencoder for event modeling that incorporates external discrete knowledge, improving narrative script induction performance and training efficiency.

Contribution

It presents a novel semi-supervised approach using Gumbel-Softmax in a neural variational autoencoder for event sequence modeling.

Findings

Outperforms existing baselines and state-of-the-art in narrative script induction.

Converges faster than previous methods.

Effectively incorporates external discrete knowledge into sequence modeling.

Abstract

Within the context of event modeling and understanding, we propose a new method for neural sequence modeling that takes partially-observed sequences of discrete, external knowledge into account. We construct a sequential neural variational autoencoder, which uses Gumbel-Softmax reparametrization within a carefully defined encoder, to allow for successful backpropagation during training. The core idea is to allow semi-supervised external discrete knowledge to guide, but not restrict, the variational latent parameters during training. Our experiments indicate that our approach not only outperforms multiple baselines and the state-of-the-art in narrative script induction, but also converges more quickly.