Rusty Clusters? Dusting an IPv6 Research Foundation

TL;DR

This paper analyzes the IPv6 Hitlist service, revealing biases caused by the Great Firewall of China, and proposes methods to improve address collection and coverage, including the inclusion of CDN addresses and new candidate sources.

Contribution

It identifies biases in the IPv6 Hitlist, demonstrates the importance of including CDN addresses, and evaluates new address generation methods to enhance hitlist coverage.

Findings

The hitlist is heavily biased by the Great Firewall of China.

Removing aliased prefixes excludes major CDNs like Fastly and Cloudflare.

Combining multiple address generation methods increases responsive address coverage by 174%.

Abstract

The long-running IPv6 Hitlist service is an important foundation for IPv6 measurement studies. It helps to overcome infeasible, complete address space scans by collecting valuable, unbiased IPv6 address candidates and regularly testing their responsiveness. However, the Internet itself is a quickly changing ecosystem that can affect longrunning services, potentially inducing biases and obscurities into ongoing data collection means. Frequent analyses but also updates are necessary to enable a valuable service to the community. In this paper, we show that the existing hitlist is highly impacted by the Great Firewall of China, and we offer a cleaned view on the development of responsive addresses. While the accumulated input shows an increasing bias towards some networks, the cleaned set of responsive addresses is well distributed and shows a steady increase. Although it is a best practice to remove aliased prefixes from IPv6 hitlists, we show that this also removes major content delivery networks. More than 98% of all IPv6 addresses announced by Fastly were labeled as aliased and Cloudflare prefixes hosting more than 10M domains were excluded. Depending on the hitlist usage, e.g., higher layer protocol scans, inclusion of addresses from these providers can be valuable. Lastly, we evaluate different new address candidate sources, including target generation algorithms to improve the coverage of the current IPv6 Hitlist. We show that a combination of different methodologies is able to identify 5.6M new, responsive addresses. This accounts for an increase by 174% and combined with the current IPv6 Hitlist, we identify 8.8M responsive addresses.