A comprehensive empirical analysis on cross-domain semantic enrichment for detection of depressive language

TL;DR

This paper presents a method for enhancing word embeddings with domain-specific information to improve depression detection in Tweets, demonstrating significant performance gains over traditional models.

Contribution

It introduces a novel augmentation approach combining general and domain-specific embeddings using various mapping techniques for depression detection.

Findings

Augmented embeddings outperform baseline models in F1 score.

Auto-encoder based mapping improves semantic relevance.

Data ablation confirms effectiveness of augmentation methods.

Abstract

We analyze the process of creating word embedding feature representations designed for a learning task when annotated data is scarce, for example, in depressive language detection from Tweets. We start with a rich word embedding pre-trained from a large general dataset, which is then augmented with embeddings learned from a much smaller and more specific domain dataset through a simple non-linear mapping mechanism. We also experimented with several other more sophisticated methods of such mapping including, several auto-encoder based and custom loss-function based methods that learn embedding representations through gradually learning to be close to the words of similar semantics and distant to dissimilar semantics. Our strengthened representations better capture the semantics of the depression domain, as it combines the semantics learned from the specific domain coupled with word coverage from the general language. We also present a comparative performance analyses of our word embedding representations with a simple bag-of-words model, well known sentiment and psycholinguistic lexicons, and a general pre-trained word embedding. When used as feature representations for several different machine learning methods, including deep learning models in a depressive Tweets identification task, we show that our augmented word embedding representations achieve a significantly better F1 score than the others, specially when applied to a high quality dataset. Also, we present several data ablation tests which confirm the efficacy of our augmentation techniques.