ProactiveVA: Proactive Visual Analytics with LLM-Based UI Agent

TL;DR

ProactiveVA introduces an LLM-powered UI agent that monitors user interactions in visual analytics systems to provide proactive, context-aware assistance, enhancing user support beyond explicit help requests.

Contribution

This work presents a novel proactive assistance framework in visual analytics, utilizing an LLM-based UI agent that anticipates user needs through interaction monitoring and context analysis.

Findings

Agent effectively predicts user needs from interaction logs

Proactive assistance improves user experience and efficiency

Framework demonstrates generalizability across VA systems

Abstract

Visual analytics (VA) is typically applied to complex data, thus requiring complex tools. While visual analytics empowers analysts in data analysis, analysts may get lost in the complexity occasionally. This highlights the need for intelligent assistance mechanisms. However, even the latest LLM-assisted VA systems only provide help when explicitly requested by the user, making them insufficiently intelligent to offer suggestions when analysts need them the most. We propose a ProactiveVA framework in which LLM-powered UI agent monitors user interactions and delivers context-aware assistance proactively. To design effective proactive assistance, we first conducted a formative study analyzing help-seeking behaviors in user interaction logs, identifying when users need proactive help, what assistance they require, and how the agent should intervene. Based on this analysis, we distilled key design requirements in terms of intent recognition, solution generation, interpretability and controllability. Guided by these requirements, we develop a three-stage UI agent pipeline including perception, reasoning, and acting. The agent autonomously perceives users' needs from VA interaction logs, providing tailored suggestions and intuitive guidance through interactive exploration of the system. We implemented the framework in two representative types of VA systems, demonstrating its generalizability, and evaluated the effectiveness through an algorithm evaluation, case and expert study and a user study. We also discuss current design trade-offs of proactive VA and areas for further exploration.