VISCA: Inferring Component Abstractions for Automated End-to-End Testing

TL;DR

VISCA introduces a hierarchical, semantic abstraction of webpages into UI components using multimodal LLMs, significantly improving the accuracy and coverage of automated end-to-end test case generation.

Contribution

The paper presents VISCA, a novel method for transforming webpages into semantic component abstractions, enhancing E2E testing accuracy over existing LLM-based approaches.

Findings

Achieved 92% feature coverage in generated test cases.

Outperformed state-of-the-art LLM-based methods by 16%.

Demonstrated effective hierarchical webpage abstraction.

Abstract

Providing optimal contextual input presents a significant challenge for automated end-to-end (E2E) test generation using large language models (LLMs), a limitation that current approaches inadequately address. This paper introduces Visual-Semantic Component Abstractor (VISCA), a novel method that transforms webpages into a hierarchical, semantically rich component abstraction. VISCA starts by partitioning webpages into candidate segments utilizing a novel heuristic-based segmentation method. These candidate segments subsequently undergo classification and contextual information extraction via multimodal LLM-driven analysis, facilitating their abstraction into a predefined vocabulary of user interface (UI) components. This component-centric abstraction offers a more effective contextual basis than prior approaches, enabling more accurate feature inference and robust E2E test case generation. Our evaluations demonstrate that the test cases generated by VISCA achieve an average feature coverage of 92%, exceeding the performance of the state-of-the-art LLM-based E2E test generation method by 16%.