History-Aware Hierarchical Transformer for Multi-session Open-domain Dialogue System
Tong Zhang, Yong Liu, Boyang Li, Zhiwei Zeng, Pengwei Wang, Yuan You,, Chunyan Miao, Lizhen Cui

TL;DR
This paper introduces HAHT, a hierarchical transformer model that effectively incorporates long-term conversation history to improve multi-session open-domain dialogue systems, outperforming baseline models in relevance and human-likeness.
Contribution
The paper presents a novel hierarchical transformer architecture that encodes and utilizes long-term conversation history for multi-session dialogue, a less explored area.
Findings
HAHT outperforms baseline models on large-scale MSC dataset.
Human evaluations show HAHT generates more human-like responses.
HAHT effectively leverages historical information for context understanding.
Abstract
With the evolution of pre-trained language models, current open-domain dialogue systems have achieved great progress in conducting one-session conversations. In contrast, Multi-Session Conversation (MSC), which consists of multiple sessions over a long term with the same user, is under-investigated. In this paper, we propose History-Aware Hierarchical Transformer (HAHT) for multi-session open-domain dialogue. HAHT maintains a long-term memory of history conversations and utilizes history information to understand current conversation context and generate well-informed and context-relevant responses. Specifically, HAHT first encodes history conversation sessions hierarchically into a history memory. Then, HAHT leverages historical information to facilitate the understanding of the current conversation context by encoding the history memory together with the current context with…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Speech and dialogue systems · Natural Language Processing Techniques
MethodsAttention Is All You Need · Linear Layer · Residual Connection · Dense Connections · Multi-Head Attention · Position-Wise Feed-Forward Layer · Dropout · Softmax · Absolute Position Encodings · Byte Pair Encoding
