Leveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling

TL;DR

This paper improves LSTM-based sequence labeling for semantic slot filling by modeling label dependencies and incorporating global sequence information, achieving state-of-the-art performance on ATIS.

Contribution

It introduces an encoder-labeler LSTM architecture that encodes entire input sequences and uses this encoding to enhance slot filling accuracy.

Findings

Achieved a state-of-the-art F1-score of 95.66% on ATIS.

Enhanced LSTM with label dependency modeling improves sequence labeling.

Incorporating global sequence information boosts slot filling performance.

Abstract

Recurrent Neural Network (RNN) and one of its specific architectures, Long Short-Term Memory (LSTM), have been widely used for sequence labeling. In this paper, we first enhance LSTM-based sequence labeling to explicitly model label dependencies. Then we propose another enhancement to incorporate the global information spanning over the whole input sequence. The latter proposed method, encoder-labeler LSTM, first encodes the whole input sequence into a fixed length vector with the encoder LSTM, and then uses this encoded vector as the initial state of another LSTM for sequence labeling. Combining these methods, we can predict the label sequence with considering label dependencies and information of whole input sequence. In the experiments of a slot filling task, which is an essential component of natural language understanding, with using the standard ATIS corpus, we achieved the state-of-the-art F1-score of 95.66%.