Similarity-based web element localization for robust test automation

TL;DR

This paper introduces Similo, a similarity-based web element localization method that improves robustness in test automation by effectively identifying web elements across website updates, outperforming existing approaches.

Contribution

The paper proposes and empirically evaluates Similo, a novel similarity-based approach for web element localization that enhances robustness in GUI test automation.

Findings

Similo outperforms the baseline in locating web elements across website updates.

Similo failed in 72 out of 598 cases, compared to 146 for the baseline.

Quantifying attribute similarity improves web element localization robustness.

Abstract

Non-robust (fragile) test execution is a commonly reported challenge in GUI-based test automation, despite much research and several proposed solutions. A test script needs to be resilient to (minor) changes in the tested application but, at the same time, fail when detecting potential issues that require investigation. Test script fragility is a multi-faceted problem, but one crucial challenge is reliably identifying and locating the correct target web elements when the website evolves between releases or otherwise fails and reports an issue. This paper proposes and evaluates a novel approach called similarity-based web element localization (Similo), which leverages information from multiple web element locator parameters to identify a target element using a weighted similarity score. The experimental study compares Similo to a baseline approach for web element localization. To get an extensive empirical basis, we target 40 of the most popular websites on the Internet in our evaluation. Robustness is considered by counting the number of web elements found in a recent website version compared to how many of these existed in an older version. Results of the experiment show that Similo outperforms the baseline representing the current state-of-the-art; it failed to locate the correct target web element in 72 out of 598 considered cases compared to 146 failed cases for the baseline approach. This study presents evidence that quantifying the similarity between multiple attributes of web elements when trying to locate them, as in our proposed Similo approach, is beneficial. With acceptable efficiency, Similo gives significantly higher effectiveness (i.e., robustness) than the baseline web element localization approach.