EmbeWebAgent: Embedding Web Agents into Any Customized UI
Chenyang Ma, Clyde Fare, Matthew Wilson, Dave Braines
TL;DR
EmbeWebAgent introduces a framework for embedding web agents directly into existing enterprise UIs, enabling more robust and expressive interactions through lightweight frontend hooks and a reusable backend workflow.
Contribution
It presents a novel, stack-agnostic method for integrating agents into web UIs with minimal retrofitting, supporting diverse actions and domain-specific analytics.
Findings
Supports mixed-granularity actions from GUI primitives to high-level tasks
Enables robust multi-step behaviors in live UI settings
Requires minimal retrofitting effort
Abstract
Most web agents operate at the human interface level, observing screenshots or raw DOM trees without application-level access, which limits robustness and action expressiveness. In enterprise settings, however, explicit control of both the frontend and backend is available. We present EmbeWebAgent, a framework for embedding agents directly into existing UIs using lightweight frontend hooks (curated ARIA and URL-based observations, and a per-page function registry exposed via a WebSocket) and a reusable backend workflow that performs reasoning and takes actions. EmbeWebAgent is stack-agnostic (e.g., React or Angular), supports mixed-granularity actions ranging from GUI primitives to higher-level composites, and orchestrates navigation, manipulation, and domain-specific analytics via MCP tools. Our demo shows minimal retrofitting effort and robust multi-step behaviors grounded in a live…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsWeb Data Mining and Analysis · Advanced Software Engineering Methodologies · Logic, programming, and type systems