Incorporating Loose-Structured Knowledge into Conversation Modeling via Recall-Gate LSTM

TL;DR

This paper presents a novel Recall-Gate LSTM model that integrates loose-structured domain knowledge into conversation modeling, improving chat-bot response relevance by enhancing semantic understanding.

Contribution

The paper introduces a Recall-Gate mechanism that incorporates easily constructed domain knowledge into LSTM for better conversation modeling.

Findings

Improved response selection accuracy on two datasets.

Effective integration of loose-structured knowledge enhances semantic relevance.

Model outperforms baseline methods in conversation modeling tasks.

Abstract

Modeling human conversations is the essence for building satisfying chat-bots with multi-turn dialog ability. Conversation modeling will notably benefit from domain knowledge since the relationships between sentences can be clarified due to semantic hints introduced by knowledge. In this paper, a deep neural network is proposed to incorporate background knowledge for conversation modeling. Through a specially designed Recall gate, domain knowledge can be transformed into the extra global memory of Long Short-Term Memory (LSTM), so as to enhance LSTM by cooperating with its local memory to capture the implicit semantic relevance between sentences within conversations. In addition, this paper introduces the loose structured domain knowledge base, which can be built with slight amount of manual work and easily adopted by the Recall gate. Our model is evaluated on the context-oriented response selecting task, and experimental results on both two datasets have shown that our approach is promising for modeling human conversations and building key components of automatic chatting systems.