"Way back then": A Data-driven View of 25+ years of Web Evolution

TL;DR

This paper provides a comprehensive quantitative analysis of over 25 years of web evolution by examining top websites' changes in popularity, content types, and categories using Internet Archive data.

Contribution

It offers the first detailed, data-driven documentation of web evolution over 25+ years, including shifts in website popularity, content types, and categories.

Findings

Decline in popularity of news and education websites.

Rise of streaming media and social networking sites.

Increasing prevalence of multimedia content types.

Abstract

Since the inception of the first web page three decades back, the Web has evolved considerably, from static HTML pages in the beginning to the dynamic web pages of today, from mainly the text-based pages of the 1990s to today's multimedia rich pages, etc. Although much of this is known anecdotally, to our knowledge, there is no quantitative documentation of the extent and timing of these changes. This paper attempts to address this gap in the literature by looking at the top 100 Alexa websites for over 25 years from the Internet Archive or the "Wayback Machine", archive.org. We study the changes in popularity, from Geocities and Yahoo! in the mid-to-late 1990s to the likes of Google, Facebook, and Tiktok of today. We also look at different categories of websites and their popularity over the years and find evidence for the decline in popularity of news and education-related websites, which have been replaced by streaming media and social networking sites. We explore the emergence and relative prevalence of different MIME-types (text vs. image vs. video vs. javascript and json) and study whether the use of text on the Internet is declining.