Perceived Performance of Webpages In the Wild: Insights from Large-scale Crowdsourcing of Above-the-Fold QoE

TL;DR

This study introduces 'SpeedPerception', a large-scale crowdsourcing framework that analyzes how humans perceive webpage loading speed, revealing limitations of traditional metrics and proposing a machine learning model for better prediction of perceived performance.

Contribution

The paper presents a novel crowdsourcing framework and dataset for understanding perceived webpage speed, and develops a machine learning model that better predicts human judgments than traditional metrics.

Findings

Traditional metrics like onLoad and TTFB poorly match human perception.

A simple 3-variable machine learning model achieves 87% accuracy in predicting perceived speed.

End-users evaluate perceived webpage speed faster than visualComplete event timing.

Abstract

Clearly, no one likes webpages with poor quality of experience (QoE). Being perceived as slow or fast is a key element in the overall perceived QoE of web applications. While extensive effort has been put into optimizing web applications (both in industry and academia), not a lot of work exists in characterizing what aspects of webpage loading process truly influence human end-user's perception of the "Speed" of a page. In this paper we present "SpeedPerception", a large-scale web performance crowdsourcing framework focused on understanding the perceived loading performance of above-the-fold (ATF) webpage content. Our end goal is to create free open-source benchmarking datasets to advance the systematic analysis of how humans perceive webpage loading process. In Phase-1 of our "SpeedPerception" study using Internet Retailer Top 500 (IR 500) websites (https://github.com/pahammad/speedperception), we found that commonly used navigation metrics such as "onLoad" and "Time To First Byte (TTFB)" fail (less than 60% match) to represent majority human perception when comparing the speed of two webpages. We present a simple 3-variable-based machine learning model that explains the majority end-user choices better (with $87 \pm 2\%$ accuracy). In addition, our results suggest that the time needed by end-users to evaluate relative perceived speed of webpage is far less than the time of its "visualComplete" event.