FitNets: An Adaptive Framework to Learn Accurate Traffic Distributions

TL;DR

FitNets is an adaptive network monitoring system that uses feedback between data and control planes, employing Kernel Density Estimators to accurately learn traffic feature distributions in real-time.

Contribution

It introduces a novel feedback-based framework combining control and data planes with Kernel Density Estimators for precise traffic distribution learning.

Findings

Capable of estimating hundreds of distributions at 60 million samples/sec

Provides accurate error estimates for learned distributions

Adapts dynamically to complex traffic patterns

Abstract

Learning precise distributions of traffic features (e.g., burst sizes, packet inter-arrival time) is still a largely unsolved problem despite being critical for management tasks such as capacity planning or anomaly detection. A key limitation nowadays is the lack of feedback between the control plane and the data plane. Programmable data planes offer the opportunity to create systems that let data- and control plane to work together, compensating their respective shortcomings. We present FitNets, an adaptive network monitoring system leveraging feedback between the data- and the control plane to learn accurate traffic distributions. In the control plane, FitNets relies on Kernel Density Estimators which allow to provably learn distributions of any shape. In the data plane, FitNets tests the accuracy of the learned distributions while dynamically adapting data collection to the observed distribution fitness, prioritizing under-fitted features. We have implemented FitNets in Python and P4 (including on commercially available programmable switches) and tested it on real and synthetic traffic traces. FitNets is practical: it is able to estimate hundreds of distributions from up to 60 millions samples per second, while providing accurate error estimates and adapting to complex traffic patterns.