Adaptive Edge Content Delivery Networks for Web-Scale File Systems

TL;DR

This paper presents Startrail, an adaptive caching mechanism for IPFS that enhances scalability, resilience, and performance of decentralized content delivery during high-demand scenarios.

Contribution

It introduces a novel adaptive network caching mechanism that improves IPFS scalability and resilience, enabling a decentralized CDN with better handling of flash-crowd conditions.

Findings

Improved scalability and resilience of IPFS-based CDN.

Enhanced performance during sudden demand spikes.

Effective decentralized content distribution with adaptive caching.

Abstract

The InterPlanetary File System (IPFS) is an hyper-media distribution protocol, addressed by content and identities. It aims to make the web faster, safer, and more open. The JavaScript implementation of IPFS runs on the browser, benefiting from the mass adoption potential that it yields. Startrail takes advantage of the IPFS ecosystem and strives to further evolve it, making it more scalable and performant through the implementation of an adaptive network caching mechanism. Our solution aims to add resilience to IPFS and improve its overall scalability, by avoiding overloading the nodes providing highly popular content, particularly during flash-crowd-like conditions where popularity and demand grow suddenly. We add a novel crucial key component to enable an IPFS-based decentralized Content Distribution Network (CDN). Following a peer-to-peer architecture, it runs on a scalable, highly available network of untrusted nodes that distribute immutable and authenticated objects which are cached progressively towards the sources of requests.