Deconvolutional Latent-Variable Model for Text Sequence Matching

TL;DR

This paper introduces a deconvolutional latent-variable model for text matching that improves semantic representation, training efficiency, and semi-supervised performance over traditional LSTM-based models.

Contribution

It presents a novel deconvolutional network-based latent-variable model that enhances text sequence matching by better capturing semantic information and enabling faster, more effective training.

Findings

Outperforms LSTM decoders in predictive accuracy

Requires fewer parameters and trains faster

Achieves superior results in semi-supervised settings

Abstract

A latent-variable model is introduced for text matching, inferring sentence representations by jointly optimizing generative and discriminative objectives. To alleviate typical optimization challenges in latent-variable models for text, we employ deconvolutional networks as the sequence decoder (generator), providing learned latent codes with more semantic information and better generalization. Our model, trained in an unsupervised manner, yields stronger empirical predictive performance than a decoder based on Long Short-Term Memory (LSTM), with less parameters and considerably faster training. Further, we apply it to text sequence-matching problems. The proposed model significantly outperforms several strong sentence-encoding baselines, especially in the semi-supervised setting.