EmpTransfo: A Multi-head Transformer Architecture for Creating Empathetic Dialog Systems
Rohola Zandie, Mohammad H. Mahoor

TL;DR
EmpTransfo introduces a multi-head Transformer architecture leveraging emotion history and metadata to enhance empathetic responses in dialog systems, outperforming existing models on key metrics.
Contribution
The paper proposes EmpTransfo, a novel Transformer-based model that incorporates emotion history and metadata for more empathetic and context-aware dialogue generation.
Findings
Outperforms other models in Hit@1 metric
Achieves lower perplexity (PPL) indicating better language modeling
Utilizes pre-trained models like OpenAI-GPT effectively
Abstract
Understanding emotions and responding accordingly is one of the biggest challenges of dialog systems. This paper presents EmpTransfo, a multi-head Transformer architecture for creating an empathetic dialog system. EmpTransfo utilizes state-of-the-art pre-trained models (e.g., OpenAI-GPT) for language generation, though models with different sizes can be used. We show that utilizing the history of emotions and other metadata can improve the quality of generated conversations by the dialog system. Our experimental results using a challenging language corpus show that the proposed approach outperforms other models in terms of Hit@1 and PPL (Perplexity).
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Speech and dialogue systems
MethodsLinear Layer · Absolute Position Encodings · Position-Wise Feed-Forward Layer · Residual Connection · Byte Pair Encoding · Dense Connections · Label Smoothing · *Communicated@Fast*How Do I Communicate to Expedia? · Adam · Softmax
