Turbocharging Web Automation: The Impact of Compressed History States

TL;DR

This paper introduces a web history compressor that distills relevant information from verbose web states, improving web automation accuracy by 1.2-5.4% across multiple datasets.

Contribution

It presents a novel history compression method that enhances web automation by effectively utilizing history states, addressing the verbosity challenge.

Findings

Achieved 1.2-5.4% accuracy improvements on Mind2Web and WebLINX datasets.

Effectively reduces input sequence length and improves information relevance.

Demonstrates the benefit of history state compression in web automation tasks.

Abstract

Language models have led to a leap forward in web automation. The current web automation approaches take the current web state, history actions, and language instruction as inputs to predict the next action, overlooking the importance of history states. However, the highly verbose nature of web page states can result in long input sequences and sparse information, hampering the effective utilization of history states. In this paper, we propose a novel web history compressor approach to turbocharge web automation using history states. Our approach employs a history compressor module that distills the most task-relevant information from each history state into a fixed-length short representation, mitigating the challenges posed by the highly verbose history states. Experiments are conducted on the Mind2Web and WebLINX datasets to evaluate the effectiveness of our approach. Results show that our approach obtains 1.2-5.4% absolute accuracy improvements compared to the baseline approach without history inputs.