AInsight: Augmenting Expert Decision-Making with On-the-Fly Insights Grounded in Historical Data

TL;DR

This paper presents AInsight, a system that provides real-time, data-driven insights during expert decision-making conversations by leveraging historical data and a retrieval-based LLM, demonstrated in a healthcare context.

Contribution

It introduces a novel conversational interface that integrates real-time data retrieval and insight generation for decision support in complex, dynamic scenarios.

Findings

Effective retrieval of relevant data during conversations

Generation of concise, contextually relevant insights

Identification of challenges and future directions

Abstract

In decision-making conversations, experts must navigate complex choices and make on-the-spot decisions while engaged in conversation. Although extensive historical data often exists, the real-time nature of these scenarios makes it infeasible for decision-makers to review and leverage relevant information. This raises an interesting question: What if experts could utilize relevant past data in real-time decision-making through insights derived from past data? To explore this, we implemented a conversational user interface, taking doctor-patient interactions as an example use case. Our system continuously listens to the conversation, identifies patient problems and doctor-suggested solutions, and retrieves related data from an embedded dataset, generating concise insights using a pipeline built around a retrieval-based Large Language Model (LLM) agent. We evaluated the prototype by embedding Health Canada datasets into a vector database and conducting simulated studies using sample doctor-patient dialogues, showing effectiveness but also challenges, setting directions for the next steps of our work.