Lemotif: An Affective Visual Journal Using Deep Neural Networks

TL;DR

Lemotif is a system that combines NLP and image generation to create visual representations of journal entries, helping users reflect on their emotions and daily experiences more effectively.

Contribution

It introduces a novel integrated system that automatically visualizes emotions and themes from journal text, encouraging emotional awareness and regular journaling.

Findings

Users prefer Lemotif-generated motifs over baselines.

Motifs are perceived as representative of journal entries.

Users are more likely to journal regularly with Lemotif.

Abstract

We present Lemotif, an integrated natural language processing and image generation system that uses machine learning to (1) parse a text-based input journal entry describing the user's day for salient themes and emotions and (2) visualize the detected themes and emotions in creative and appealing image motifs. Synthesizing approaches from artificial intelligence and psychology, Lemotif acts as an affective visual journal, encouraging users to regularly write and reflect on their daily experiences through visual reinforcement. By making patterns in emotions and their sources more apparent, Lemotif aims to help users better understand their emotional lives, identify opportunities for action, and track the effectiveness of behavioral changes over time. We verify via human studies that prospective users prefer motifs generated by Lemotif over corresponding baselines, find the motifs representative of their journal entries, and think they would be more likely to journal regularly using a Lemotif-based app.