Humane Visual AI: Telling the Stories Behind a Medical Condition

TL;DR

This paper introduces a comprehensive framework combining deep learning, probabilistic analysis, and storytelling visualization to better understand and communicate the biological, psychological, and social aspects of medical conditions.

Contribution

It presents a novel interdisciplinary approach integrating social media mining, prescription data analysis, and narrative visualization for medical condition storytelling.

Findings

Participants valued psychological and social insights more after interaction.

10% of users prioritized psychological aspects post-visualization.

27% of users became more favorable towards social media data in healthcare.

Abstract

A biological understanding is key for managing medical conditions, yet psychological and social aspects matter too. The main problem is that these two aspects are hard to quantify and inherently difficult to communicate. To quantify psychological aspects, this work mined around half a million Reddit posts in the sub-communities specialised in 14 medical conditions, and it did so with a new deep-learning framework. In so doing, it was able to associate mentions of medical conditions with those of emotions. To then quantify social aspects, this work designed a probabilistic approach that mines open prescription data from the National Health Service in England to compute the prevalence of drug prescriptions, and to relate such a prevalence to census data. To finally visually communicate each medical condition's biological, psychological, and social aspects through storytelling, we designed a narrative-style layered Martini Glass visualization. In a user study involving 52 participants, after interacting with our visualization, a considerable number of them changed their mind on previously held opinions: 10% gave more importance to the psychological aspects of medical conditions, and 27% were more favourable to the use of social media data in healthcare, suggesting the importance of persuasive elements in interactive visualizations.