Compressing IP Forwarding Tables: Towards Entropy Bounds and Beyond

TL;DR

This paper demonstrates how to compress IP forwarding tables to approach entropy bounds, significantly reducing memory usage while maintaining or improving lookup speed and update efficiency in network routers.

Contribution

It introduces a novel entropy-compressed representation of the IP FIB and re-designs the prefix tree for optimal lookup and near-optimal updates, advancing data structure efficiency in networking.

Findings

Compressed a 440K prefix FIB to 100-400 KB of memory.

Achieved a threefold increase in lookup throughput.

Maintained efficient FIB updates without penalty.

Abstract

Lately, there has been an upsurge of interest in compressed data structures, aiming to pack ever larger quantities of information into constrained memory without sacrificing the efficiency of standard operations, like random access, search, or update. The main goal of this paper is to demonstrate how data compression can benefit the networking community, by showing how to squeeze the IP Forwarding Information Base (FIB), the giant table consulted by IP routers to make forwarding decisions, into information-theoretical entropy bounds, with essentially zero cost on longest prefix match and FIB update. First, we adopt the state-of-the-art in compressed data structures, yielding a static entropy-compressed FIB representation with asymptotically optimal lookup. Then, we re-design the venerable prefix tree, used commonly for IP lookup for at least 20 years in IP routers, to also admit entropy bounds and support lookup in optimal time and update in nearly optimal time. Evaluations on a Linux kernel prototype indicate that our compressors encode a FIB comprising more than 440K prefixes to just about 100--400 KBytes of memory, with a threefold increase in lookup throughput and no penalty on FIB updates. This technical report contains a number of important corrections and revisions to the original manuscript.