Semantic Constraint Inference for Web Form Test Generation

TL;DR

This paper presents FormNexus, a novel method that uses semantic inference and large language models to generate comprehensive web form tests, achieving high coverage and surpassing baseline performance.

Contribution

Introducing FormNexus, a semantic inference framework utilizing LLMs and a new graph model to automate and improve web form test generation.

Findings

Achieves 89% form submission state coverage.

Outperforms baseline models by 25%.

Effectively uses semantic insights for testing.

Abstract

Automated test generation for web forms has been a longstanding challenge, exacerbated by the intrinsic human-centric design of forms and their complex, device-agnostic structures. We introduce an innovative approach, called FormNexus, for automated web form test generation, which emphasizes deriving semantic insights from individual form elements and relations among them, utilizing textual content, DOM tree structures, and visual proximity. The insights gathered are transformed into a new conceptual graph, the Form Entity Relation Graph (FERG), which offers machine-friendly semantic information extraction. Leveraging LLMs, FormNexus adopts a feedback-driven mechanism for generating and refining input constraints based on real-time form submission responses. The culmination of this approach is a robust set of test cases, each produced by methodically invalidating constraints, ensuring comprehensive testing scenarios for web forms. This work bridges the existing gap in automated web form testing by intertwining the capabilities of LLMs with advanced semantic inference methods. Our evaluation demonstrates that FormNexus combined with GPT-4 achieves 89% coverage in form submission states. This outcome significantly outstrips the performance of the best baseline model by a margin of 25%.