# Pythia: a Framework for the Automated Analysis of Web Hosting Environments

## TL;DR

Pythia is a lightweight framework that accurately identifies whether websites are self-hosted or use third-party hosting services, aiding research in web infrastructure analysis.

## Contribution

It introduces Pythia, a novel tool for automated, accurate detection of web hosting environments, addressing limitations of previous ad hoc methods.

## Key findings

- Achieves 90% accuracy in identifying hosting types.
- Detects less than 11% of popular domains are self-hosted.
- Provides a publicly available tool for research use.

## Abstract

A common approach when setting up a website is to utilize third party Web hosting and content delivery networks. Without taking this trend into account, any measurement study inspecting the deployment and operation of websites can be heavily skewed. Unfortunately, the research community lacks generalizable tools that can be used to identify how and where a given website is hosted. Instead, a number of ad hoc techniques have emerged, e.g., using Autonomous System databases, domain prefixes for CNAME records. In this work we propose Pythia, a novel lightweight approach for identifying Web content hosted on third-party infrastructures, including both traditional Web hosts and content delivery networks. Our framework identifies the organization to which a given Web page belongs, and it detects which Web servers are self-hosted and which ones leverage third-party services to provide contents. To test our framework we run it on 40,000 URLs and evaluate its accuracy, both by comparing the results with similar services and with a manually validated groundtruth. Our tool achieves an accuracy of 90% and detects that under 11% of popular domains are self-hosted. We publicly release our tool to allow other researchers to reproduce our findings, and to apply it to their own studies.