IDA: Breaking Barriers in No-code UI Automation Through Large Language Models and Human-Centric Design

TL;DR

This paper introduces IDA, a no-code web UI automation tool leveraging large language models and human-centric design to empower non-technical business users, demonstrated through a prototype and user study showing its effectiveness and user-friendliness.

Contribution

The paper presents IDA, a novel no-code UI automation system that integrates LLMs and human-centric principles, enabling business users to automate tasks without technical expertise.

Findings

Users effectively created automations with IDA

IDA was perceived as user-friendly and trustworthy

Prototype demonstrated successful real-world application

Abstract

Business users dedicate significant amounts of time to repetitive tasks within enterprise digital platforms, highlighting a critical need for automation. Despite advancements in low-code tools for UI automation, their complexity remains a significant barrier to adoption among non-technical business users. However, recent advancements in large language models (LLMs) have created new opportunities to overcome this barrier by offering more powerful, yet simpler and more human-centric programming environments. This paper presents IDA (Intelligent Digital Apprentice), a novel no-code Web UI automation tool designed specifically to empower business users with no technical background. IDA incorporates human-centric design principles, including guided programming by demonstration, semantic programming model, and teacher-student learning metaphor which is tailored to the skill set of business users. By leveraging LLMs, IDA overcomes some of the key technical barriers that have traditionally limited the possibility of no-code solutions. We have developed a prototype of IDA and conducted a user study involving real world business users and enterprise applications. The promising results indicate that users could effectively utilize IDA to create automation. The qualitative feedback indicates that IDA is perceived as user-friendly and trustworthy. This study contributes to unlocking the potential of AI assistants to enhance the productivity of business users through no-code user interface automation.