Towards More Accurate Prediction of Human Empathy and Emotion in Text and Multi-turn Conversations by Combining Advanced NLP, Transformers-based Networks, and Linguistic Methodologies

TL;DR

This paper develops an advanced NLP system combining transformers, linguistic methods, and ensemble modeling to improve the prediction of empathy and emotion in text and multi-turn conversations, based on shared tasks.

Contribution

It introduces a multi-stage approach with model enhancements, class imbalance handling, lexical resource integration, and ensemble techniques for empathy and emotion detection.

Findings

Improved accuracy in empathy and emotion prediction.

Effective handling of class imbalance.

Successful adaptation to multi-turn conversations.

Abstract

Based on the WASSA 2022 Shared Task on Empathy Detection and Emotion Classification, we predict the level of empathic concern and personal distress displayed in essays. For the first stage of this project we implemented a Feed-Forward Neural Network using sentence-level embeddings as features. We experimented with four different embedding models for generating the inputs to the neural network. The subsequent stage builds upon the previous work and we have implemented three types of revisions. The first revision focuses on the enhancements to the model architecture and the training approach. The second revision focuses on handling class imbalance using stratified data sampling. The third revision focuses on leveraging lexical resources, where we apply four different resources to enrich the features associated with the dataset. During the final stage of this project, we have created the final end-to-end system for the primary task using an ensemble of models to revise primary task performance. Additionally, as part of the final stage, these approaches have been adapted to the WASSA 2023 Shared Task on Empathy Emotion and Personality Detection in Interactions, in which the empathic concern, emotion polarity, and emotion intensity in dyadic text conversations are predicted.