Towards Real Smart Apps: Investigating Human-AI Interactions in Smartphone On-Device AI Apps
Jason Ching Yuen Siu, Jieshan Chen, Yujin Huang, Zhenchang Xing,, Chunyang Chen

TL;DR
This paper investigates human-AI interactions in smartphone on-device AI apps, analyzing 176 apps to identify interaction patterns and inform better design, supported by a user study validating the findings.
Contribution
First empirical study exploring user-AI interaction in mobile apps, identifying key interaction patterns and implementing a search-enabled gallery for improved design guidance.
Findings
Identified 255 AI features across 176 apps
Summarized 759 implementations into three primary interaction patterns
User study confirms the usefulness of the identified patterns
Abstract
With the emergence of deep learning techniques, smartphone apps are now embedded on-device AI features for enabling advanced tasks like speech translation, to attract users and increase market competitiveness. A good interaction design is important to make an AI feature usable and understandable. However, AI features have their unique challenges like sensitiveness to the input, dynamic behaviours and output uncertainty. Existing guidelines and tools either do not cover AI features or consider mobile apps which are confirmed by our informal interview with professional designers. To address these issues, we conducted the first empirical study to explore user-AI-interaction in mobile apps. We aim to understand the status of on-device AI usage by investigating 176 AI apps from 62,822 apps. We identified 255 AI features and summarised 759 implementations into three primary interaction…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsInnovative Human-Technology Interaction · Green IT and Sustainability · Mobile Crowdsensing and Crowdsourcing
