Happiness Entailment: Automating Suggestions for Well-Being

TL;DR

This paper proposes a neural network-based system to analyze journaled happy moments and suggest personalized actions to improve well-being, focusing on the development of the Happiness Entailment Recognition module.

Contribution

It introduces a novel neural network model for happiness entailment recognition, enabling automated, personalized suggestions for well-being based on journal entries.

Findings

The HER module can determine if a suggestion is likely beneficial based on a user's happy moment.

The system captures psychologically significant features in texts for better suggestion relevance.

Prototype demonstrates potential for automated well-being enhancement tools.

Abstract

Understanding what makes people happy is a central topic in psychology. Prior work has mostly focused on developing self-reporting assessment tools for individuals and relies on experts to analyze the periodic reported assessments. One of the goals of the analysis is to understand what actions are necessary to encourage modifications in the behaviors of the individuals to improve their overall well-being. In this paper, we outline a complementary approach; on the assumption that the user journals her happy moments as short texts, a system can analyze these texts and propose sustainable suggestions for the user that may lead to an overall improvement in her well-being. We prototype one necessary component of such a system, the Happiness Entailment Recognition (HER) module, which takes as input a short text describing an event, a candidate suggestion, and outputs a determination about whether the suggestion is more likely to be good for this user based on the event described. This component is implemented as a neural network model with two encoders, one for the user input and one for the candidate actionable suggestion, with additional layers to capture psychologically significant features in the happy moment and suggestion.

Tables

Table 1. TABLE I : Statistics of the training dataset.

# of samples (# of pos / # of neg)	1364 (682 / 682)
# of distinct happy moments (HM)	709
# of distinct sustainable suggestions (S)	30
% agency (HM, S)	78.3, 100
% sociality (HM, S)	45.2, 9.5

Table 2. TABLE II : Most Common Concepts in HappyDB.

Concept	HM (%)	Concept	HM (%)	Concept	HM (%)
Family	23.04	Romance	5.71	Technology	3.25
Food	13.44	Conversation	4.89	Weather	2.61
Entertainment	12.08	Exercise	4.57	Party	2.54
Career	10.06	Education	4.32	Vacation	2.44
Shopping	6.29	Animals	4.17	Religion	1.01
				All	78.15

Table 3. TABLE III : Experimental results for the HER task.

Method	AU-ROC	Method	AU-ROC
Concept Model	0.561	Dual-encoder	0.770
AllenNLP Entailment	0.548	+ concept (C)	0.748
		+ agency (A)	0.748
		+ sociality (S)	0.819
		+ A + S	0.806
		+ C + A + S	0831

Table 4. TABLE IV : Dataset statistics for suggestibility prediction.

# of samples (# of pos / # of neg)	999 (440 / 559)
# of filtered samples (# of pos / # of neg)	290 (150 / 140)

Table 5. TABLE V : Experimental results for the suggestibility prediction task.

Method	AU-ROC
Guessing all positive	0.500
Logistic Regression	0.867
Bi-LSTM	0.900

Full text

*In this paper, we outline a complementary approach; on the assumption that the user journals her happy moments as short texts, a system can analyze these texts and propose sustainable suggestions for the user that may lead to an overall improvement in her well-being. We prototype one necessary component of such a system, the Happiness Entailment Recognition (HER) module, which takes as input a short text describing an event, a candidate suggestion, and outputs a determination about whether the suggestion is more likely to be good for this user based on the event described. This component is implemented as a neural network model with two encoders, one for the user input and one for the candidate actionable suggestion, with additional layers to capture psychologically significant features in the happy moment and suggestion. Our model achieves an AU-ROC of 0.831 and outperforms our baseline as well as the current state-of-the-art Textual Entailment model from AllenNLP by more than 48% of improvements, confirming the uniqueness and complexity of the HER task. *

happiness, positive psychology, natural language processing, textual entailment, happiness entailment *

I Introduction

The field of positive psychology has made great strides in applying empirical methods to understand happiness **[1, 2, 3, 4]** and find interventions that demonstrably improve it **[5, 6]**.

To understand one’s well-being and particularly, what actions lead to (un)happiness, a wide variety of self-reporting assessment tools have been used, such as the Day Reconstruction Method (DRM) **[7]**, End-of-Day Diaries (EDD), the Experience Sampling Method (ESM) **[8]** and the Ecological Momentary Assessment (EMA) **[9]**. These tools are designed to capture experiences and behavior of a person as accurately a possible, and have become standard tools for measuring happiness **[10]**. However, these surveys are often tedious to fill out and do not always capture the nuances of one’s life.

A more recent approach to investigating happiness looks at short textual descriptions of experiences that a person had and journaled (often referred to as moments) **[11, 12, 13]**. This is driven by the development of new Natural Language Understanding techniques that learn from moments that a person reports, thereby freeing the user from having to answer lengthy surveys. Some of the initial work on this approach has focused on distinguishing happy moments from unhappy ones, and finding the activities, people, and patterns involved in the reported happy moments.

One of the main goals of positive psychology is to discover behaviors that contribute to an individual’s happiness in a sustainable fashion. A main barrier to this approach is the tendency for people to mis-estimate how happy things will make them **[14]**. Most people spend significant time and resources on activities with the goal of improving their well-being, only to find that the affect boost they gain is smaller or shorter lasting than they had hoped. Hindsight, especially from an outside observer, is a more reliable indicator of which activities actually foster happiness.

Making individually-tailored suggestions has long been in the domain of psychology professionals, but we propose a design for a system that can supplement professional help by generating suggestions for well-being, especially in one of the numerous places where there is a shortage of mental health professionals **[15]**. This system would (1) discover sustainable suggestions for activities, and (2) identify which suggestions make sense for individual users. As an input to this system, users create short journal entry “happy moments,” an intervention proven to improve subjective well-being **[16]**. The relationship between these two tasks is shown in Fig. 1.

For the first task, we define suggestions as sustainable in the sense they improve short-term well-being without being detrimental in the long term, even if the suggestion is repeated. For example, a user may have reported that adopting a puppy or buying a new car has made them happy. However, we would not want to recommend doing them again because the repetition is not sustainable. In contrast, activities like going for a swim or a walk in the park have the potential to improve the well-being of the user and are repeatable, so we consider them sustainable suggestions. Such suggestions become part of our SuggestionDB. Ideally, domain experts (e.g., psychologists) would create those sustainable suggestions, but we also develop a classifier that identifies happy moments with sustainable activities, and confirm that the classifier achieved an AU-ROC of 0.900.

For the second of these tasks, we introduce the happiness entailment recognition problem (HER), inspired by the problem of recognizing textual entailment (RTE) **[17]** in NLP: given a set of happy moments reported by a user, and a set of possible suggested activities, which of the activities is likely to make the user happy? We develop a neural network (NN) model enhanced with information about concepts, agency, and sociality to predict with an AU-ROC of 0.831 whether a suggestion fits a happy moment. “Concepts” represents frequently occurring topics in HappyDB as a one-hot vector. “Agency” and “sociality” are binary indicators of whether the author had control over their happy moment, and whether there were other people involved, respectively. We claim that these classifications of the texts capture psychological perspectives that are important for the task.

*Experimental results show that for this task, the proposed method outperforms the state-of-the-art textual entailment recognition model, as well as our baseline. The results support that the HER task is significantly different from conventional textual entailment task, and that adding psychological perspectives to the model improves the performance. *

*Our contributions are as follows: *

• •

We describe a system for making sustainable suggestions to a user based on her description of a positive event that happened to her.

• •

We develop a sustainable suggestion classifier that filters “suggestible” happy moments from a corpus, and show that it performs robustly.

• •

*We describe our approach to obtaining sustainable suggestions and our solution for the HER problem: a neural network-based method that incorporates psychological features. Our HER model outperforms both our baseline and the state-of-the-art textual entailment recognition model. *

II The Happiness Entailment Recognition Task

Our goal is to identify which activity suggestions help improve the user’s well-being and are sustainable. Such suggestions are essential for building a system to help the user improve her well-being such as Jo **[11]**.

Ideally suggestions would be based on a collection of happy moments from the user, knowledge about the user, and knowledge about the user’s surroundings such as location and weather. In this paper, we explore the simplest formulation of this problem where we examine only one happy moment to determine a potential suggestion that would make a user happy.

*For example, “paint your nails” may be a good sustainable suggestion to a person who describes a happy moment about a haircut, indicating that personal grooming boosts their affect. The suggestion is less relevant if the happy moment describes “painting the wall.” Therefore, it requires common sense and inference to understand the relationship between a happy moment and a sustainable suggestion and to evaluate if the suggestion would make the author happy. *

We formulate this as a binary classification task, where the system takes two input sentences, namely a happy moment and a suggestion, and classifies the pair into two classes: (1) “entailment” or (2) “non-entailment,” following the original problem formulation of the RTE task **[17]**. Compared to RTE, which has mostly focused on logical reasoning, HER considers a more subjective version of entailment based on implicit cues in a happy moment.

Because the ground truth labels for HER are given by human annotators, our model can learn common sense about the relationship between event(s) that made the author happy and a sustainable suggestion. Averaging the results of non-expert annotations has very high agreement with expert labels for NLP tasks **[18]**, so we use crowdsourcing for human annotations. We hope that this task can lead to insights into the causes of happiness, since predicting entailed suggestions requires an understanding of the (possibly implicit) reasons that a moment made the user happy.

III Obtaining Training Data for HER

In this section, we describe the dataset creation methodology for the HER task. As we described in Section II, each sample consists of (1) a happy moment, (2) a sustainable suggestion, and (3) a label for the pair. We filtered happy moments from HappyDB and manually crafted sustainable suggestions before generating pairs based on them for crowdsourced annotations.

*Sustainable Suggestions *** We manually crafted 36 sustainable suggestions based on the happy moments in HappyDB. We filtered the happy moments written in a single sentence, and then created clusters based on the verbs. This helped us come up with different types of activities that should have reasonable coverage with respect to the happy moments in HappyDB. The curated sustainable suggestions are shown in Fig. 2. Note that our system can take other sustainable suggestions as input. In Section VI, we will describe how to expand this set by filtering sustainable suggestions from happy moments.

**Label Collection *** We used Amazon Mechanical Turk (MTurk) for crowdsourced annotations. We recruited only MTurk Master Workers to ensure high-quality annotations. Each task consists of $N$ (= 3 or 5) happy moments reported by different people and 1 sustainable suggestion randomly chosen from 30 in our manually curated set. An example task is shown in Fig. 3. For each task, 5 workers were asked to choose the happy moments for which they considered the given sustainable suggestion would be appropriate111With limited information about the user, we cannot be certain that the suggestion is a good fit, but we can leverage the common sense of annotators to infer that conditional on the happy moment, it is more likely to be a good fit than previously expected.. Lastly, we left an optional free-response text box with the question about the reason and evidence of the annotation. This served two purposes: including such a field for explanations has been shown to improve the quality of annotations, and it helped us understand cases where annotators disagreed. 121 unique workers completed these tasks. *

We were concerned that the neighboring happy moments that a happy moment is presented with (the “context”) might bias the annotations assigned. To avoid this effect, we made sure that happy moments were shuffled and presented with different contexts for different workers. We also experimented with showing workers either $N=3$ or $5$ happy moments at a time but found no significant difference.

*Confidence Filtering *** We obtained 5,387 labels (entailment or non-entailment) that were computed by taking the majority vote of the 5 annotators for the happy moment and sustainable suggestion pairs. To ensure quality, we applied confident filtering to the annotations. For the positive (entailment) class, we filtered only rows where at least four of the five annotators agreed. For the negative (non-entailment) class, we applied more rigorous agreement standards to minimize the effects of default bias, using only those which had a unanimous agreement between annotators. Table  5 shows examples of pairs of happy moments and sustainable suggestions, including their original annotations. The negative examples still outweighed the positive examples, so we balanced the dataset by randomly sampling a subset of the negative examples. This filtering left 1,364 examples, which were split into 1,068 training samples, and 148 samples each for validation and test. Basic statistics of the final dataset after the filtering are shown in Table I.

IV HER model

In this section, we describe our proposed model for the HER task. Our basic HER module is implemented as a dual-encoder (DE) model **[19*]**, which is a standard NN model for NLI tasks. As shown in Fig. 4, it has a shared encoder for both inputs (i.e., a happy moment and a sustainable suggestion.) Note that a model for the HER task has to be able to take arbitrary two textual inputs to judge the entailment, and thus cannot define the classes (i.e., multi-class classification) in advance. *

*Psychological Features *** As variants of the DE model, we enhanced the input happy moments and sustainable suggestions with additional psychological features: (1) concept, (2) agency, and (3) sociality. Fig. 4 shows the architecture where the “Feature” box represents the additional features (i.e. concept, agency, or sociality).

We hypothesized that evidence for a sustainable suggestion often comes from overlapping concepts in the happy moment and the sustainable suggestion. For example, if the happy moment and sustainable suggestion both mention exercise, that may be evidence for happiness entailment. This hypothesis was motivated by the concept of learning generalization in psychology—a person responds similarly to similar stimuli **[20]**. We approximate similarity by using 15 concepts which were developed for use in the Cl-Aff Happiness Shared task. The concepts were chosen from a series of experimental MTurk tasks that asked workers to write a noun describing the event in a happy moment. The 15 most frequently occurring responses were chosen as concepts. Table II shows these 15 concepts and their coverage in HappyDB. We use Logistic Regression classifiers using bag-of-words features. The combined classifiers output a 15 dimensional “concept vector,” where each index represents the presence or absence of a concept (with a 0 or a 1). The concept vectors for happy moments and sustainable suggestions share a fully-connected layer as an encoder, whose output is concatenated to the embedding of the happy moment or suggestion.

Similarly, we use two Logistic Regression classifiers on bag-of-words features to predict agency and sociality labels for happy moments, which are concatenated into embedding vectors in the same manner as concept vectors. The sociality label captures whether a text involves people other than the author. Because humans are highly motivated to develop and maintain strong relationships **[21]**, and cultivation of social relationships is linked with happiness **[14]**, we hypothesized that sociality labels could encourage the model to learn when social interaction was critical to a happy moment, thus should be suggested in an activity. The agency label describes whether the author is in control or not. Exercising agency in the form of pursuing autonomously chosen personal goals is shown to improve well-being **[22]**, so this feature may similarly help the model learn how agency plays a role in what should be suggested. By nature, all the suggestions are agentic since the user must have the ability to engage in this activity.

By providing these psychological features, we allow the model to learn which features generalize between happy moments and sustainable suggestions, and possibly about psychological concepts that generalize to each other in the context of HER.

V Evaluation

In this section, we describe our baseline models and evaluation method. We selected two baseline methods: a concept-based classifier and a textual entailment classifier. We compared these to our HER models, described in section IV.

*Concepts Model *** Our baseline model uses the concept classifiers to detect concepts for the happy moment and the sustainable suggestion, and outputs a positive label for happiness entailment if they share any in common. We trained these classifiers based on HappyDB, which achieved F-scores between 0.67 and 0.90.

*Textual Entailment Model *** As we mentioned, the RTE task can be formulated as the same classification problem as the HER task. The task takes two different sentences and outputs a single output from the entailment, neutral, or contradiction classes. To verify the overlap between the RTE and HER tasks, we evaluated a pre-trained textual entailment recognition model. Specifically, we used the AllenNLP textual entailment (TE) model [23] that is trained on the Stanford NLI (SNLI) dataset [24] since it is considered one of the state-of-the-art methods and a de-facto standard baseline method for the RTE task. We found that changing the suggestions to a first-person conjugation (e.g., “Go for a walk” becomes “I should go for a walk”) improved the performance of the TE model, so we applied this as a pre-processing step to all the suggestions for this model only.

*DE Model *** The DE model was implemented using PyTorch. The word embeddings were initialized with 300-dimension pre-trained GloVe embeddings, using a vocabulary size of 1,643. The model used a shared encoder that is a 3-layer bidirectional LSTM with 500 hidden states. We used Adam optimization with an initial learning rate of $8.28643\times 10^{-4}$. We used a dropout rate of $5.87755\times 10^{-1}$, a batch size of 32 and trained for 30 epochs. These parameters were chosen on the validation set by training 9 versions of the model with different hyper-parameters using randomized search [25].

*DE + Feature(s) model *** We used the same concepts model as the baseline model. The agency and sociality models were trained on the dataset of 10k labeled happy moments presented at the CL-Aff Happiness Shared Task [13], and they achieved accuracy values of 0.898 and 0.950 for the agency and sociality classification tasks respectively222These models were not chosen as baselines because they achieved accuracy the same as for random guessing (0.51–0.53 AU-ROC).. In addition to testing these three features separately, we trained and evaluated a DE model enhanced with all three features.

**Experimental Settings *** For fair comparisons, we used the same dataset split into training, validation, and test data as described in Section III. The dual-encoder methods were trained on the training set and tuned using the validation set, and all the methods were evaluated on the test set.

We used AU-ROC as an evaluation metric as it is considered the standard evaluation metric for binary classification. AU-ROC measures the probability that a randomly chosen positive test sample will be scored higher than a randomly chosen negative test sample, and the AU-ROC value for random guessing will be 0.5.*

V-A Results and Discussion

Table III shows the AU-ROC values on our test set of the 7 models described above. The TE model, although state-of-the-art for the RTE task, performs about the same as random guessing for the HER task. This shows that the HER task is significantly different from the conventional RTE task, and we need a HER corpus, not an RTE corpus, to train a model. Table III also shows that the concept model performs only slightly better than random guessing, mostly due to false negatives. For example, the happy moment “Coffee has made me incredibly happy” and the sustainable suggestion “meditate” was labeled as entailment, but they share no concept from our list in Table II, although they both reflect mindfulness. Our dual-encoder model is able to predict this relation, both with and without the added features.

Based on the performance, sociality is the most informative feature when combined with the dual-encoder model, for this task. This indicates that the sociality label in many cases influences what type of sustainable suggestion is well-suited. This is especially interesting because there is no correlation between the sociality label (of either the happy moment or the suggestion) and the entailment label, so the labels alone do not predict whether the sustainable suggestion matches the happy moment. Rather, the model seems to learn for which happy moments social interaction is essential to the cause of happiness, and thus should be part of a sustainable suggestion.

*The best performing model includes all three additional features, but it is not clear that the difference in performance is statistically significant. Future work includes further investigation to determine why the features interact this way, testing the importance of social interaction in suggestions, and discovering other informative psychological features of texts. *

VI Sustainable Suggestion Classification

The sustainable suggestions we used in our HER task were manually curated based on ideas from happy moments and positive psychology **[14]**. We attempt to reduce this bottleneck by building a classifier that automatically filters candidate suggestions from a corpus of happy moments such as HappyDB.

We hypothesize that happy moments contain many sustainable activities that we can suggest if we learn how to extract them. Therefore, we build a model that takes a happy moment as input and determines if the activity is suggestible. We define two criteria for suggestibility: (1) the activity can be repeated with a reasonable amount of effort (repeatability), and (2) it would likely make someone happy to repeat this (sustainability).

Examples of happy moments that are repeatable but not sustainable are “I adopted a puppy” and “I bought a new air conditioner.” There is a limit to how many puppies a user should adopt, and buying a new air-conditioner is probably will only make one happy if the air-conditioner is needed.

At first we attempted to crowdsource sustainable suggestions by giving MTurk workers the prompt “Imagine a friend of yours is bored, or in a bad mood. What are some things you might suggest they do to cheer up?”, followed by three boxes for free text entries. Most of the results were either not repeatable or not sustainable, so we explore an automated way of extracting sustainable suggestions.

VI-A Methods

*Label collection *** Because the notion of suggestibility only applies to single activities, not entire happy moments, we selected only happy moments with a single activity to label333The number of activities was defined as the number of consecutive sequences of conjugated or modal verbs, as detected by the constituency parsing model of AllenNLP [26].. For each selected happy moment, 5 workers on MTurk answered the questions “Is it in the author’s control to repeat this activity?” and “Do you think it would make the author happy to repeat this activity (assuming they could)?” to elicit labels for our two criteria (i.e., repeatability and sustainability.) Happy moments with 4:1 agreement for both of these labels were added to the training data. Although both of our criteria are subjective, we found that filtering by agreement this way yielded reasonable results.

For each happy moment, we labeled it as suggestible if it was rated as repeatable and sustainable by at least 4 of 5 annotators. It was labeled as not suggestible if it was rated “no” for repeatably or for sustainability by at least 4 out of 5 annotators. Examples are shown in Fig. 6, and Table IV shows more details about the dataset.

**Experimental Setting *** For the suggestibility prediction task, we first trained a simple Logistic Regression model trained on Bag-of-Words features. We also implemented a Bi-LSTM RNN model with self-attention (Bi-LSTM) [27], which is a standard technique for text classification tasks in NLP. We split the data into 70% training, 15% validation, and 15% test sets, and trained and evaluated both models on the training and test sets respectively. *

VI-B Results and Discussion

The results are shown in Table V. Both the Logistic Regression model and the Bi-LSTM model performed well, with the Bi-LSTM performing better. This indicates that our sustainable suggestion classification framework, regardless of machine learning algorithms, accurately filters sustainable suggestion candidates from a corpus of happy moments.

*The bag-of-words features allow us to see which words carry the most weight in the classification decision of the Logistic Regression model. The model has highest weights for words that relate to spending or winning, as this is a strong indicator that an event is not suggestible (winning is usually out of the author’s control, and spending money is usually not labeled as sustainable.) Words relating to family members or friends also had high weights, as most social activities are sustainable and repeatable. This simple model works quite well for our task because words like this carry a lot of information about whether the activity is suggestible. *

*We consider two major limitations of this method as follows. First, many happy moments are similar, so we need post-processing to detect redundant suggestions. Second, the happy moments are still in the first person and use descriptive or potentially de-anonymizing language. For example, the activity “I played ball with our Golden Retriever Rosie” is labeled and predicted as suggestible, but it should be rephrased as “play ball with your dog” before it is presented to a user. *

VII Related Work

Textual analysis for understanding happiness is an emerging topic in the intersection between NLP and psychology fields. Asai et al. **[12]** collected 100k happy moments called HappyDB using MTurk. CL-Aff Shared Task 2019 **[13]** collected another set corpus based on HappyDB, contributing two additional labels on each happy moment: agency and sociality, which we utilized in this work. Several studies have applied NLP techniques to the corpora to understand the cause of happiness. Rajendran et al. **[28]** have tested various machine learning techniques including NN models for the sociality and the agency classification tasks, and they confirm that NN methods robustly performed better for these tasks. Gupta et al. **[29]** have used intensity scores of five emotions (valence, joy, anger, feature, and sadness) in addition to concept vectors for the sociality and agency classification tasks. Their experimental results show that concept vectors significantly contribute to the classification accuracy compared to the emotional intensity scores.

The Recognizing Textual Entailment (RTE) task **[17]** is defined as recognizing whether the meaning of one text can be inferred (i.e., entailed) from the other based on given two text inputs. The task has been intensively studied in NLP **[24, 19]**. Although the problem formulation of the HER task is similar to the RTE task, we empirically show that the state-of-the-art pre-trained RTE model does not perform well on the HER task, and thus there is little overlap between these two tasks.

VIII Conclusion and Future Work

Inspired by our initial work on the Jo application **[11*]**, this paper took a first step towards a module that automatically suggests activities that will increase the well-being of its user. We developed two essential components of generating suggestions. The first is the Happiness Entailment Recognition (HER) component that enables one to determine, based on textual analysis, whether a sustainable suggestion is suitable given a happy moment. Our results show that our dual-encoder HER model with the concepts, agency and sociality labels achieve the highest accuracy compared to the baseline methods, supporting the idea that we need to train a dedicated model for the HER task. For the second component, we developed a sustainable suggestion classifier using machine learning and we have shown that the model can detect sustainable activities in happy moments. *

*Future work includes incorporating more sophisticated information about the user in proposing activities, including the user’s history of happy moments, their responses to suggestions, and possibly other contextual information such as location, weather, mood, etc. Collection of more diverse suggestions, whether curated or automatically extracted by expanding on our techniques, will be critical to the success of this system.

Further investigation is also required to understand the interactions between agency, sociality, and concept labels with respect to HER, and to explore other psychological features that may be important. *