Framework Matters: Energy Efficiency of UI Automation Testing Frameworks

TL;DR

This study compares the energy consumption of four web UI automation frameworks for various actions, revealing significant differences and emphasizing the importance of energy-aware testing decisions.

Contribution

It provides empirical data on energy efficiency across frameworks and actions, highlighting the need for transparency to enable energy-aware testing practices.

Findings

Puppeteer is most efficient for several actions

Selenium excels in refresh and scroll actions

Energy costs can differ by up to six times between frameworks

Abstract

We examine per action energy consumption across four web user interface (UI) automation testing frameworks to determine whether consistent tendencies can guide energy-aware test design. Using a controlled client-server setup with external power metering, we repeat each UI action (refresh, click variants, checkbox, drag&drop, input-text, scroll) 35 times. Across each of the actions, energy costs vary by both framework and action. Puppeteer is the most efficient for left-click, right-click, double-click, checkbox, and input-text; Selenium is the most efficient for refresh and scroll; Nightwatch is generally the least energy efficient. The energy cost of performing the same action varied by up to a factor of six depending on the framework. This indicates that providing transparency of energy consumption for UI automation testing frameworks allows developers to make informed, energy-aware decisions when testing a specific UI action.