AwareLLM: A Proactive Multimodal Ecosystem for Personalized Human-AI Collaboration to Enhance Productivity
Amog Rao, Utkarsh Agarwal, Amol Harsh, and Siddharth Siddharth

TL;DR
AwareLLM is a proactive, multimodal framework that enhances human-AI collaboration by adapting to users' psychophysiological states, leading to improved productivity and reduced fatigue.
Contribution
It introduces a novel multimodal system integrating physiological data with LLMs for real-time, personalized interventions in human-AI collaboration.
Findings
Significant improvements in task performance with AwareLLM.
Reductions in cognitive fatigue and mental demand.
Participants found interventions timely and relevant.
Abstract
Information workers' productivity is significantly influenced by their cognitive states and physiological responses. AI assistants such as ChatGPT, Copilot, and others have become integral components of knowledge-intensive workplaces. These AI assistants utilize pre-defined user preferences and chat interaction histories, thus confining themselves to reactive exchanges, lacking sufficient adaptability. Consequently, they fail to cater to individual user preferences and are unable to adapt to their psychophysiological states, diminishing potential productivity gains. To bridge this gap, we introduce AwareLLM, a novel multimodal framework that integrates egocentric vision, pupillometry, eye-gaze tracking, posture detection, heart activity, and the inferencing capabilities of large language models (LLMs) to create a proactive and context-aware ecosystem. AwareLLM dynamically adapts to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
