ESSYS* Sharing #UC: An Emotion-driven Audiovisual Installation

TL;DR

ESSYS* Sharing #UC is an audiovisual installation that visualizes and sonifies Twitter data related to Coimbra's university and city, using emotion recognition and autonomous creative processes to foster online and physical space integration.

Contribution

The paper introduces a novel emotion-driven audiovisual installation that combines neural classification of tweets with autonomous creative generation for urban art.

Findings

The installation effectively visualizes emotional data from Twitter.

Participants found the posters expressive and engaging.

The approach demonstrates potential for social and commercial applications.

Abstract

We present ESSYS* Sharing #UC, an audiovisual installation artwork that reflects upon the emotional context related to the university and the city of Coimbra, based on the data shared about them on Twitter. The installation was presented in an urban art gallery of C\'irculo de Artes Pl\'asticas de Coimbra during the summer and autumn of 2021. In the installation space, one may see a collection of typographic posters displaying the tweets and listening to an ever-changing ambient sound. The present audiovisuals are created by an autonomous computational creative approach, which employs a neural classifier to recognize the emotional context of a tweet and uses this resulting data as feedstock for the audiovisual generation. The installation's space is designed to promote an approach and blend between the online and physical perceptions of the same location. We applied multiple experiments with the proposed approach to evaluate the capability and performance. Also, we conduct interview-based evaluation sessions to understand how the installation elements, especially poster designs, are experienced by people regarding diversity, expressiveness and possible employment in other commercial and social scenarios.