Energy-based Self-attentive Learning of Abstractive Communities for Spoken Language Understanding

TL;DR

This paper introduces an energy-based self-attentive neural model for abstractive community detection in spoken language understanding, effectively grouping conversation utterances by their shared summaries.

Contribution

It proposes a novel neural encoder with self-attention mechanisms trained via energy-based architectures for improved community detection.

Findings

Outperforms state-of-the-art baselines on AMI corpus

Demonstrates effectiveness of energy-based training for utterance grouping

Provides publicly available code and data

Abstract

Abstractive community detection is an important spoken language understanding task, whose goal is to group utterances in a conversation according to whether they can be jointly summarized by a common abstractive sentence. This paper provides a novel approach to this task. We first introduce a neural contextual utterance encoder featuring three types of self-attention mechanisms. We then train it using the siamese and triplet energy-based meta-architectures. Experiments on the AMI corpus show that our system outperforms multiple energy-based and non-energy based baselines from the state-of-the-art. Code and data are publicly available.