Pegasus: A Universal Framework for Scalable Deep Learning Inference on the Dataplane

TL;DR

Pegasus introduces a novel framework translating deep learning operations into dataplane-friendly primitives, enabling scalable, accurate, and general DL inference directly on network hardware, significantly outperforming existing methods.

Contribution

It proposes a universal set of primitives and fusion techniques to enable scalable, accurate DL inference on the dataplane, addressing limitations of current match-action table abstractions.

Findings

Supports various DL models including MLP, RNN, CNN, AutoEncoder

Achieves up to 22.8% accuracy improvement over state-of-the-art

Enables model sizes up to 248x larger and input scales 212x larger

Abstract

The paradigm of Intelligent DataPlane (IDP) embeds deep learning (DL) models on the network dataplane to enable intelligent traffic analysis at line-speed. However, the current use of the match-action table (MAT) abstraction on the dataplane is misaligned with DL inference, leading to several key limitations, including accuracy degradation, limited scale, and lack of generality. This paper proposes Pegasus to address these limitations. Pegasus translates DL operations into three dataplane-oriented primitives to achieve generality: Partition, Map, and SumReduce. Specifically, Partition "divides" high-dimensional features into multiple low-dimensional vectors, making them more suitable for the dataplane; Map "conquers" computations on the low-dimensional vectors in parallel with the technique of fuzzy matching, while SumReduce "combines" the computation results. Additionally, Pegasus employs Primitive Fusion to merge computations, improving scalability. Finally, Pegasus adopts full precision weights with fixed-point activations to improve accuracy. Our implementation on a P4 switch demonstrates that Pegasus can effectively support various types of DL models, including Multi-Layer Perceptron (MLP), Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), and AutoEncoder models on the dataplane. Meanwhile, Pegasus outperforms state-of-the-art approaches with an average accuracy improvement of up to 22.8%, along with up to 248x larger model size and 212x larger input scale.