A Grounded Well-being Conversational Agent with Multiple Interaction Modes: Preliminary Results

TL;DR

This paper introduces Marie Bot Wellbeing, a multi-modal conversational agent with a human avatar designed to improve engagement and adherence in health-related interactions by mimicking doctor-patient communication and supporting diverse conversations.

Contribution

The paper presents a novel grounded, multi-modal conversational agent with a human avatar for medical question answering and general engagement, addressing low adoption of health tech.

Findings

Prototype implementation completed

Preliminary user engagement results positive

Multiple interaction modes enhance user experience

Abstract

Technologies for enhancing well-being, healthcare vigilance and monitoring are on the rise. However, despite patient interest, such technologies suffer from low adoption. One hypothesis for this limited adoption is loss of human interaction that is central to doctor-patient encounters. In this paper we seek to address this limitation via a conversational agent that adopts one aspect of in-person doctor-patient interactions: A human avatar to facilitate medical grounded question answering. This is akin to the in-person scenario where the doctor may point to the human body or the patient may point to their own body to express their conditions. Additionally, our agent has multiple interaction modes, that may give more options for the patient to use the agent, not just for medical question answering, but also to engage in conversations about general topics and current events. Both the avatar, and the multiple interaction modes could help improve adherence. We present a high level overview of the design of our agent, Marie Bot Wellbeing. We also report implementation details of our early prototype , and present preliminary results.