InvestChat: Exploring Multimodal Interaction via Natural Language, Touch, and Pen in an Investment Dashboard

TL;DR

InvestChat is a multimodal investment dashboard integrating natural language, touch, and pen inputs, enhancing user engagement and interaction for novice investors through an LLM-powered chat system.

Contribution

This work introduces a novel multimodal tablet application for stock exploration that combines multiple input modalities with an LLM chat, evaluated with novice users.

Findings

Combining modalities increases user engagement.

Natural language is most effective among the modalities.

Participants enjoyed the freedom of modality choice.

Abstract

We designed and implemented InvestChat, a multimodal tablet-based application that supports stock market exploration with multiple coordinated views and an LLM-powered chat. We evaluated the application with 12 novice investors. Our findings suggest that combining natural language, touch, and pen input during stock market exploration facilitates user engagement. Participants leveraged the modalities in complementary ways, enjoying the freedom of choice and finding natural language most effective.