IIsy: Practical In-Network Classification

TL;DR

IIsy enables in-network machine learning classification using existing network devices, reducing server load and latency by hybridizing models on switches and backends, addressing key implementation challenges.

Contribution

The paper introduces IIsy, a practical system for deploying diverse machine learning models directly in network devices, supporting hybrid classification to improve scalability and efficiency.

Findings

Supports various ML models with scalable architecture

Achieves near-optimal classification with reduced latency

Significantly decreases server load

Abstract

The rat race between user-generated data and data-processing systems is currently won by data. The increased use of machine learning leads to further increase in processing requirements, while data volume keeps growing. To win the race, machine learning needs to be applied to the data as it goes through the network. In-network classification of data can reduce the load on servers, reduce response time and increase scalability. In this paper, we introduce IIsy, implementing machine learning classification models in a hybrid fashion using off-the-shelf network devices. IIsy targets three main challenges of in-network classification: (i) mapping classification models to network devices (ii) extracting the required features and (iii) addressing resource and functionality constraints. IIsy supports a range of traditional and ensemble machine learning models, scaling independently of the number of stages in a switch pipeline. Moreover, we demonstrate the use of IIsy for hybrid classification, where a small model is implemented on a switch and a large model at the backend, achieving near optimal classification results, while significantly reducing latency and load on the servers.