AutoRPA: Efficient GUI Automation through LLM-Driven Code Synthesis from Interactions

TL;DR

AutoRPA automatically converts LLM-driven GUI interaction strategies into efficient, reusable RPA functions, significantly reducing token usage and improving runtime performance in repetitive tasks.

Contribution

AutoRPA introduces a novel framework that distills ReAct-style GUI agents into robust RPA functions using a translator-builder pipeline and hybrid repair strategies.

Findings

RPA functions reduce token usage by 82% to 96%.

AutoRPA improves runtime efficiency in GUI tasks.

Generated RPA functions successfully solve similar tasks.

Abstract

Large Language Model (LLM) based agents have demonstrated proficiency in multi-step interactions with graphical user interfaces (GUIs). While most research focuses on improving single-task performance, practical scenarios often involve repetitive GUI tasks for which invoking LLM reasoning repeatedly, i.e., the ReAct paradigm, is inefficient. Prior to LLMs, traditional Robotic Process Automation (RPA) offers runtime efficiency but demands significant manual effort to develop and maintain. To bridge this gap, we propose AutoRPA, a framework that automatically distills the decision logic of ReAct-style agents into robust RPA functions. AutoRPA introduces two core innovations: (1) A translator-builder pipeline, where a translator agent converts hard-coded ReAct actions into soft-coded procedures, and a builder agent synthesizes robust RPA functions via retrieval-augmented generation over multiple trajectories; (2) A hybrid repair strategy during code verification, combining RPA execution with ReAct-based fallback for iterative refinement. Experiments across multiple GUI environments demonstrate that RPA functions generated by AutoRPA successfully solve similar tasks while reducing token usage by 82% to 96%, significantly improving runtime efficiency and reusability.