DiLogics: Creating Web Automation Programs With Diverse Logics
Kevin Pu, Jim Yang, Angel Yuan, Minyi Ma, Rui Dong, Xinyu Wang, Yan, Chen, Tovi Grossman

TL;DR
DiLogics is a programming-by-demonstration system that leverages NLP to help users create web automation programs capable of handling diverse input conditions and specifications.
Contribution
It introduces a novel approach combining semantic segmentation and demonstration recording to generalize web automation tasks for varied requirements.
Findings
Non-experts can effectively use DiLogics for diverse web automation tasks.
DiLogics enables creation of flexible automation programs with minimal user effort.
The system outperforms existing tools in handling varied input conditions.
Abstract
Knowledge workers frequently encounter repetitive web data entry tasks, like updating records or placing orders. Web automation increases productivity, but translating tasks to web actions accurately and extending to new specifications is challenging. Existing tools can automate tasks that perform the same logical trace of UI actions (e.g., input text in each field in order), but do not support tasks requiring different executions based on varied input conditions. We present DiLogics, a programming-by-demonstration system that utilizes NLP to assist users in creating web automation programs that handle diverse specifications. DiLogics first semantically segments input data to structured task steps. By recording user demonstrations for each step, DiLogics generalizes the web macros to novel but semantically similar task requirements. Our evaluation showed that non-experts can effectively…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
