SpliDT: Partitioned Decision Trees for Scalable Stateful Inference at Line Rate

TL;DR

SPLIDT introduces a partitioned decision tree system for programmable data planes that enables scalable, real-time inference by efficiently reusing resources and optimizing feature allocation, significantly improving accuracy and scalability.

Contribution

It proposes a novel partitioned decision tree architecture with in-band control and a custom training framework, enabling scalable in-network inference with higher accuracy.

Findings

Supports up to 5x more features than prior systems

Maintains low time-to-detection comparable to existing methods

Scales to millions of flows with minimal recirculation overhead

Abstract

Machine learning (ML) is increasingly being deployed in programmable data planes (switches and SmartNICs) to enable real-time traffic analysis, security monitoring, and in-network decision-making. Decision trees (DTs) are particularly well-suited for these tasks due to their interpretability and compatibility with data-plane architectures, i.e., match-action tables (MATs). However, existing in-network DT implementations are constrained by the need to compute all input features upfront, forcing models to rely on a small, fixed set of features per flow. This significantly limits model accuracy and scalability under stringent hardware resource constraints. We present SPLIDT, a system that rethinks DT deployment in the data plane by enabling partitioned inference over sliding windows of packets. SPLIDT introduces two key innovations: (1) it assigns distinct, variable feature sets to individual sub-trees of a DT, grouped into partitions, and (2) it leverages an in-band control channel (via recirculation) to reuse data-plane resources (both stateful registers and match keys) across partitions at line rate. These insights allow SPLIDT to scale the number of stateful features a model can use without exceeding hardware limits. To support this architecture, SPLIDT incorporates a custom training and design-space exploration (DSE) framework that jointly optimizes feature allocation, tree partitioning, and DT model depth. Evaluation across multiple real-world datasets shows that SPLIDT achieves higher accuracy while supporting up to 5x more stateful features than prior approaches (e.g., NetBeacon and Leo). It maintains the same low time-to-detection (TTD) as these systems, while scaling to millions of flows with minimal recirculation overhead (<0.05%).

Tables

Table 1. Table 1 : Feature density (%) across partitions and subtrees in a DT, and max. recirculation bandwidth (Mbps) when processing datasets ( D1–3 ) for two datacenter environments, Webserver (WS) and Hadoop (HD), § 5 .

Data	Feature Density (%)		Recirc. Bandwidth (Mbps)
	/ Partition	/ Subtree	WS	HD
D1	47.15 $\pm$ 38.44	6.15 $\pm$ 2.95	2.93 $\pm$ 2.44	5.99 $\pm$ 3.51
D2	53.49 $\pm$ 44.19	7.28 $\pm$ 2.72	6.01 $\pm$ 4.01	12.32 $\pm$ 5.76
D3	53.95 $\pm$ 43.42	6.08 $\pm$ 3.37	3.58 $\pm$ 3.21	7.33 $\pm$ 4.62

Table 2. Table 2 : Real-world network traffic datasets used for evaluating SpliDT across diverse security scenarios.

Dataset	Description	Classes
D1: CIC-IoMT2024	A cybersecurity dataset [15] with Internet of Medical Things (IoMT) traffic for intrusion detection in healthcare.	19
D2: CIC-IoT2023-a	A simplified version of the CIC-IoT-2023 dataset [16], categorized into four primary classes of IoT traffic.	4
D3: ISCX-VPN2016	A dataset containing VPN and non-VPN traffic [17] for evaluating VPN detection and privacy-related analyses.	13
D4: CampusTraffic	UCSB campus dataset [35] containing various application types, including web, cloud, social, and, streaming traffic.	11
D5: CIC-IoT2023-b	A comprehensive IoT dataset [16] containing multi-class network traffic data for evaluating IoT security threats.	32
D6: CIC-IDS2017	A network intrusion detection dataset [13] for various attack scenarios, including DoS, DDoS, and brute force.	10
D7: CIC-IDS2018	An anomaly detection dataset [14] capturing network traffic for diverse attacks and benign activities.	10

Table 3. Table 3 : Model performance vs. resource usage tested against Tofino1 switch ( 6.4   Mbits 6.4\text{\,}\mathrm{M}\mathrm{b}\mathrm{i}\mathrm{t}\mathrm{s} TCAM budget, 12 stages)  [ 41 ] .

Data	#Flows	F1 Score			Depth / #Partitions			#Features			#TCAM Entries			Register Size (bits)
		NB	Leo	SpliDT	NB	Leo	SpliDT	NB	Leo	SpliDT	NB	Leo	SpliDT	NB	Leo	SpliDT
D1	100K	0.51	0.50	0.60	13	11	24 / 5	6	6	20	6,596	16,384	4,583	192	192	160
	500K	0.49	0.43	0.55	13	6	21 / 5	4	4	21	6,154	2,048	4,509	128	128	96
	1M	0.41	0.18	0.48	12	3	13 / 1	2	2	2	1,534	2,048	8,238	64	64	64
D2	100K	0.79	0.79	0.86	12	11	45 / 3	6	6	30	8,479	16,384	17,083	192	192	160
	500K	0.78	0.74	0.80	12	6	27 / 3	2	4	30	11,996	2,048	12,002	64	128	64
	1M	0.75	0.63	0.75	18	3	12 / 1	1	2	2	2,540	2,048	465	32	64	64
D3	100K	0.78	0.73	0.85	13	11	21 / 2	6	5	23	5,056	16,384	2,562	192	160	160
	500K	0.56	0.57	0.77	12	6	16 / 5	4	4	28	1,566	2,048	2,931	128	128	64
	1M	0.31	0.05	0.59	4	3	18 / 4	2	1	15	170	2,048	928	64	32	32
D4	100K	0.67	0.66	0.71	13	10	28 / 2	6	7	27	11,361	8,192	14,507	192	224	160
	500K	0.49	0.52	0.64	10	11	20 / 2	4	2	26	4,355	16,384	12,934	128	64	64
	1M	0.39	0.20	0.51	9	3	27 / 2	2	1	1	4,221	2,048	393	64	32	32
D5	100K	0.36	0.40	0.43	10	10	49 / 5	6	7	20	7,755	8,192	27,302	192	224	96
	500K	0.34	0.30	0.36	12	6	33 / 4	2	4	21	30,891	2,048	21,433	64	128	64
	1M	0.31	0.10	0.31	7	3	11 / 1	2	2	2	2,063	2,048	639	64	64	64
D6	100K	0.59	0.85	0.96	5	10	15 / 5	4	4	28	308	8,192	587	128	128	160
	500K	0.59	0.64	0.94	5	7	23 / 3	3	3	16	354	2,048	578	96	96	96
	1M	0.51	0.34	0.73	8	3	10 / 4	2	2	9	403	2,048	174	64	64	32
D7	100K	0.87	0.86	0.99	5	10	26 / 5	6	6	17	251	8,192	191	192	192	160
	500K	0.82	0.76	0.96	8	7	10 / 6	4	3	10	495	2,048	106	128	96	64
	1M	0.47	0.41	0.76	5	3	10 / 6	2	2	7	71	2,048	76	64	64	32

Table 4. Table 4 : Average time per iteration across different stages of the SpliDT framework.

Stages	D1	D2	D3	D4	D5	D6	D7
Fetch	0.9s	0.32s	0.01s	0.07s	0.91s	0.24s	0.18s
Training	556s	228s	10s	84s	725s	163s	111s
Optimizer	33s	45s	43s	43s	30s	32s	37s
Rulegen	0.8s	0.99s	0.91s	1.08s	0.71s	0.71s	0.91s
Backend	42$\mu$s	45$\mu$s	43$\mu$s	42$\mu$s	46$\mu$s	44$\mu$s	47$\mu$s
Time	589s	273s	54s	128s	756s	196s	148s

Table 5. Table 5 : Maximum recirculation bandwidth (Mbps) of SpliDT partitioned trees when processing datasets ( D1–7 ) for the two datacenter environments, Webserver (WS) and Hadoop (HD), with varying flow sizes.

Environment	Data	Recirc. Bandwidth (Mbps)
		100K	500K	1M
Webserver (WS)	D1	2.4$\pm$1.4	12.2$\pm$6.9	0.0$\pm$0.0
	D2	1.5$\pm$0.8	7.3$\pm$4.2	0.0$\pm$0.0
	D3	1.0$\pm$0.6	12.2$\pm$6.9	19.5$\pm$11.1
	D4	1.0$\pm$0.6	4.9$\pm$2.8	9.8$\pm$5.5
	D5	2.4$\pm$1.4	9.8$\pm$5.5	0.0$\pm$0.0
	D6	2.4$\pm$1.4	7.3$\pm$4.2	19.5$\pm$11.1
	D7	2.4$\pm$1.4	14.6$\pm$8.3	29.3$\pm$16.6
Hadoop (HD)	D1	5.0$\pm$2.0	25.0$\pm$9.9	0.0$\pm$0.0
	D2	3.0$\pm$1.2	15.0$\pm$6.0	0.0$\pm$0.0
	D3	2.0$\pm$0.8	25.0$\pm$9.9	40.0$\pm$15.9
	D4	2.0$\pm$0.8	10.0$\pm$4.0	20.0$\pm$8.0
	D5	5.0$\pm$2.0	20.0$\pm$8.0	0.0$\pm$0.0
	D6	5.0$\pm$2.0	15.0$\pm$6.0	40.0$\pm$15.9
	D7	5.0$\pm$2.0	30.0$\pm$11.9	60.0$\pm$23.9

Full text

We present SpliDT, a system that rethinks DT deployment in the data plane by enabling partitioned inference over sliding windows of packets. SpliDT introduces two key innovations: (1) it assigns distinct, variable feature sets to individual subtrees of a DT, grouped into partitions, and (2) it leverages an in-band control channel (via recirculation) to reuse data-plane resources (both stateful registers and match keys) across partitions at line rate. These insights allow SpliDT to scale the number of stateful features a model can use without exceeding hardware limits. To support this architecture, SpliDT incorporates a custom training and design-space exploration (DSE) framework that jointly optimizes feature allocation, tree partitioning, and DT model depth. Evaluation across multiple real-world datasets shows that SpliDT achieves higher accuracy while supporting up to 5$\times$ more stateful features than prior approaches (e.g., NetBeacon and Leo). It maintains the same low time-to-detection (TTD) as these systems, while scaling to millions of flows with minimal recirculation overhead ($\leq$0.05%).

1 Introduction

11footnotetext: Both authors contributed equally to this work.

Machine Learning (ML) is rapidly becoming a cornerstone of modern networking, driving increasingly sophisticated applications such as DDoS detection (LUCID [25], Flowlens [5]), intrusion detection [13, 14, 71], encrypted traffic analysis [70, 76, 4], malware classification [29, 2], IoT botnet detection [24] as well as congestion control [36, 73, 82, 48, 23], and variable bitrate (VBR) video streaming [51, 81]. These use cases demand real-time, high-throughput inference [67] to keep up with the ever-growing scale and complexity of network traffic [61, 6, 13, 14, 17, 15, 16].

Programmable data planes—including modern switches (e.g., Broadcom Trident [11], Xsight X2 [80], Intel Tofino [41, 42], and NVIDIA Spectrum-X [57]) and emerging SmartNICs (e.g., Intel IPU [46], AMD Pensando DPU [1], and NVIDIA BlueField-3 DPU [56])—with their ability to process packets at line rate, have emerged as a powerful platform for deploying ML models directly in the network [43, 85, 67, 68, 79, 84, 75]. By offloading inference tasks to network hardware, these data planes eliminate the need for control-plane intervention, enabling low-latency and high-performance decision-making [43, 85, 67, 62].

A major focus of recent work has been on mapping decision tree (DT) models to programmable data planes [79, 75, 43, 85, 12], mainly because of these models’ interpretability and natural alignment with the (reconfigurable) match-action table (MAT) architecture [41, 42, 9, 11, 80, 77, 78, 10, 56]. While systems such as IIsy [79], NetBeacon [85], and Leo [43] have demonstrated the feasibility of deploying DTs in the data plane—to be able to operate within the stringent resource constraints of these programmable switches and SmartNICs—they have mainly addressed the challenge of rule explosion and concentrated on optimizing model representations (e.g., pruning DTs [43] and compressing MAT rules [79, 85]).

However, despite these advancements, the critical aspect of feature collection and engineering, i.e., selecting and computing complex (stateful) input features, remains largely unexplored in this context, primarily due to the resource limitations of underlying network hardware. Existing approaches either constrain the number of stateful features to a small, fixed set (e.g., the top-$k$ most important features, as in NetBeacon [85] and Leo [43]) or avoid using the stateful features altogether (as in IIsy [79] and Mousika [75]). As we show in §2, these strategies result in poor model performance (e.g., reduced F1 scores) and prevent deployed DT models from capturing complex, real-world traffic patterns effectively [43, 85, 79].

This limited focus on feature engineering in prior work has its roots in two commonly made assumptions. The first assumption is that all selected features must be computed upfront before DT traversal can begin Figure˜1 (top). Second, programmable data planes are assumed to be limited to executing DT models in a single, one-shot manner, prohibiting resource reuse across different portions of the DT. As a result, existing approaches view reducing the number of stateful features as the only viable solution to satisfy the given resource constraints, thus making it necessary to sacrifice model performance for scalability (i.e., supporting more concurrent flows) or vice versa [43, 85].

In this paper, we challenge these assumptions and present SpliDT, a system that enables scalable and resource-efficient deployment of DTs in the data plane by rethinking how features are computed and reused. SpliDT is built on two key insights. First, feature computation can be deferred: DTs do not require all features to be computed upfront. Instead, SpliDT divides the DT into partitions, where each partition—a group of consecutive layers containing one or more subtrees—computes only the features relevant to its specific subtree. This allows features to be computed incrementally as the DT traversal progresses, Figure˜1 (bottom). Second, resources can be reused across subtrees: by leveraging packet recirculation as an in-band control channel, SpliDT reuses data-plane resources (i.e., registers and match keys) between subtrees, enabling more efficient use of constrained hardware without sacrificing line rate.

SpliDT leverages these insights in an intuitive way to significantly scale the total number of stateful features that a DT can utilize. Instead of applying the same top-$k$ features across the entire DT, SpliDT assigns each subtree—resulting from tree partitioning—its own set of relevant features, allowing feature selection to vary across subtrees. These subtrees are then triggered sequentially, via recirculated control packets in the data plane, reusing the stateful registers and match keys at each stage of DT traversal for the currently active subtree, Figure˜1 (bottom). We demonstrate in §5 that SpliDT supports five times more stateful features (i.e., the total number of unique features across all subtrees) than state-of-the-art approaches [43, 85], all while achieving higher model accuracy and scaling to millions of concurrent flows at line rate.

In enabling these benefits, SpliDT must overcome two key challenges: (1) determining how to select and compute the appropriate features at runtime for each subtree during inference, and (2) designing an effective partitioning strategy at training time that balances model accuracy and hardware resource efficiency to maximize the number of supported concurrent flows in the data plane.

To address the first challenge, SpliDT processes each flow in windows of packets, specific to each subtree of a partition. Ideally, every subtree should have access to the entire flow during inference; however, since the data plane operates (and monitors traffic) at line rate without buffering, this is not feasible in practice [41, 42, 9, 11, 80, 77, 78]. Instead, SpliDT splits each flow into uniform windows, allowing each subtree to observe a portion of the flow during inference, Figure˜1 (bottom). Modern datacenter transport protocols (e.g., Homa [52] and NDP [37]) embed flow size information in packet headers, which can be parsed in the hardware to determine window boundaries. This information allows the data plane to halt feature collection at the designated boundary, trigger the selection of active subtrees, and transition to the next partition via recirculation.

To tackle the second challenge, SpliDT employs an iterative design search methodology integrated with a custom training framework to fine-tune DT configurations, including partitioning strategies and resource allocation policies, for specific use cases (and datasets). The goal is to optimize the trade-off between model accuracy and the number of flows that can be supported, identifying configurations that lie on the Pareto frontier. Leveraging Bayesian Optimization (BO), such as HyperMapper [53], SpliDT systematically explores the design space to identify the most effective hyperparameters, including the maximum number of features ($k$) per subtree, the number and size of partitions, and the overall tree depth. Intuitively, smaller values of $k$ enable support for more flows,111In Tofino1 switch [41], $k=4$ supports up to 100,000 flows, which decreases to 65,000 with $k=6$, and so on [85, 43]. SmartNICs (e.g., AMD Pensando DPU) exhibit similar behavior, with flow capacity dropping from about 64,000 ($k=4$) to 40,000 ($k=6$) [1, 56]. while deeper subtrees generally improve model accuracy. Similarly, increasing the number of subtrees expands the total set of unique features across the DT model, but reduces the number of packets each subtree can observe, limiting the temporal window for feature computation. SpliDT navigates this trade-off space to derive Pareto-optimal models that balance inference accuracy and flow scalability within the constraints of the underlying hardware.

Our results (§5) demonstrate that SpliDT sets a new state-of-the-art for deploying DT models in the data plane. It consistently outperforms NetBeacon [85] and Leo [43], achieving a superior accuracy-to-flow count Pareto frontier across all levels of supported flows (Table˜2). SpliDT supports 5$\times$ more stateful features, enabling richer in-network inference, all while maintaining a low recirculation overhead—just $50\text{\,}\mathrm{M}\mathrm{b}\mathrm{p}\mathrm{s}$ (0.05%) in the worst case—and matching the low time-to-detection (TTD) performance of current systems. To facilitate reproducibility and further research in this area, we will publicly release the complete artifact, including training scripts, models, and evaluation datasets [22].

2 Background & Motivation

We first review prior work on in-network classification systems that use decision tree (DT) models and highlight the need for more stateful features for DT-based inference (§2.1). We then revisit the anatomy of DTs to derive domain-specific insights (§2.2) that guide us in addressing the challenges of scaling DTs on modern programmable switches (§2.3).

2.1 The Need for More Stateful Features

As network traffic scales to multi-Tbps rates, existing approaches, such as IIsy [79] and Planter [84], aim to support real-time inference by mapping DTs onto match-action tables (MATs) while relying solely on stateless, per-packet features. These methods optimize DT representation to fit within the constraints of available switch resources (i.e., MATs) but lack flow-level context, limiting their classification accuracy and adaptability [79, 84, 85, 75]. While they efficiently handle large flow volumes in the data plane, their reliance on per-packet features significantly reduces accuracy, yielding F1 scores nearly 2$\times$ lower than models with full features access (Figure˜2).

More recent work, such as NetBeacon [85] and Leo [43], improves classification accuracy by incorporating stateful features (i.e., top-$k$), allowing DT models to leverage flow-level context, which provides richer insights than per-packet features alone. While this approach enhances accuracy, it places substantial pressure on the limited memory resources of programmable data planes [9, 41, 42, 11, 80, 77, 78], ultimately limiting the number of flows that can be supported concurrently.

First, stateful features must be stored in registers, which share limited space with match-action tables (MATs) within each stage of the data-plane pipeline. This creates a trade-off between feature storage and model complexity. For example, in Tofino1 [41], allocating just four registers per flow exhausts an entire switch stage at 65K flows (or about 40K flows on a Pensando DPU-based SmartNIC [1]), preventing that stage from being used for model execution. Increasing the number of registers or supporting more flows further reduces available MAT stages, limiting DT depth and restricting feature selection. Second, adding more stateful features increases match key sizes, which inflates the size of table entries and exacerbates TCAM memory usage—making it harder to map DTs efficiently onto MATs in the data plane [43, 85, 41].

As a result, prior work has been limited to a maximum of 200K flow rules and a handful of stateful features (top-$k\leq 6$)—bounds imposed by hardware memory constraints and the need to balance register usage with model depth—yielding only moderate accuracy gains [43, 85]. Beyond this threshold, performance degrades: DT depth becomes restricted, lowering classification accuracy, while larger match key sizes increase TCAM overhead, reducing scalability (§5). These trade-offs highlight the core challenge of integrating stateful features into DT-based inference without sacrificing scalability on resource-constrained programmable data planes.

Observation: The constraints of prior DT-based systems are often perceived as intrinsic to programmable data-plane architectures, but are they truly fundamental? We argue that these limitations do not stem from hardware constraints but from ingrained assumptions about how DTs should be processed. Conventional approaches assume that all stateful features must be collected before inference begins, necessitating upfront register allocation. Furthermore, these approaches treat DT execution as a single-pass, feed-forward process, confining computation to the spatially available pipeline resources.

This paper challenges the prevailing belief that feature richness and scalability must always be in conflict. We demonstrate that by decoupling stateful feature selection from DT execution, both can scale independently. Unlike prior work (e.g., NetBeacon [85] and Leo [43]), which sacrifices feature expressiveness for flow scalability, SpliDT dynamically selects and reuses stateful features across inference steps, efficiently managing available hardware resources without restricting model complexity. Compared to traditional top-$k$ systems, SpliDT achieves a significantly improved Pareto frontier, simultaneously enhancing F1 scores and flow rule capacity (Figure˜2). These results challenge the notion that hardware-imposed constraints inherently limit DT scalability, showing instead that the primary bottleneck arises from rigid execution models that preallocate features and enforce single-pass processing.

2.2 Domain-Specific Properties of DTs

DT inference begins at the root node, where a decision is made based on selected features to determine the next node to visit. This process continues at each level, using different features at each step until a leaf node is reached. Instead of processing the tree level by level, we can group consecutive levels into partitions (Figure˜3) and focus on the subtrees in each partition. With this approach, inference progresses one partition at a time, where the decisions resulting from traversing the active subtree in one partition determine which subtree to traverse in the next partition. This allows for more efficient traversal, selecting only the next subtree based on relevant features and their conditions rather than evaluating entire levels sequentially.

This subtree-by-subtree execution enables features to be collected incrementally and on-demand for the active subtree in each partition (Figure˜1, bottom). Unlike traditional approaches that require gathering all features upfront, this method loads only the features needed for the current subtree. With just $k$ available feature slots, we can dynamically allocate and process only the relevant features at each step, avoiding the restrictive top-$k$ selection enforced by existing systems [43, 85]. This approach maximizes feature utilization without discarding valuable contextual information across the entire DT.

However, two key conditions must hold for this approach to be effective: (a) feature density across subtrees: each subtree within a partition must use at most $k$ features out of the total $N$ available features. Unlike traditional top-$k$ approaches that select a fixed set of $k$ features for the entire tree, here $k$ represents the number of feature slots available at any given step (subtree), which can be dynamically reassigned to different features as the inference progresses. If even a single subtree requires more than $k$ features, the benefits of incremental feature computation are lost, as more than $k$ features would need to be allocated upfront; (b) incremental computation architecture: the system must support subtree-by-subtree traversal, dynamically collecting and computing features while reusing the same $k$ feature slots at each step. This ensures efficient execution without requiring all features to be preloaded simultaneously, overcoming the limitations of prior top-$k$ selection methods that discard all but the most globally important features across the entire DT.

To examine the feasibility of the first condition, we studied feature usage across subtrees in multiple datasets (Table˜1). In the considered datasets (D1–3),222This trend extends beyond D1–3; similar feature sparsity holds for D4–7 (see §5) and is commonly observed in real-world DT classification tasks [2, 35]. we found that at most only 10% of features were required in any given subtree. For instance, in dataset D1, where $N=41$, subtrees required on average only 3.73 features. Note that this 11$\times$ reduction in storage requirement makes it possible to execute the entire DT using only $k$ registers (e.g., 4 for D1) and avoids the need to impose a strict limit of $k$ on the overall features.

For the second condition, we demonstrate in the next section (§2.3) how modern programmable architectures, with support for packet recirculation [41, 42, 11, 80, 77, 78], can be reimagined as time-shared machines, enabling efficient reuse of resources (e.g., registers and match keys) across partitions within the data plane.

2.3 Switch as a Time-Shared Machine

Programmable data planes are traditionally viewed as spatial architectures with fixed resource constraints, where exceeding available resources leads to failures during program compilation [8, 9, 41]. However, we argue that because this perspective focuses mainly on the static aspects of both programmable data planes and spatial architectures, it is overly restrictive and in need of being revisited.

For example, modern data planes (switches [41, 42] and SmartNICs [77, 78]) support packet recirculation, with bandwidths reaching $100\text{\,}\mathrm{G}\mathrm{b}\mathrm{p}\mathrm{s}$ (e.g., Tofino [41], X2 [80], and Trident [11]), without impacting line rate. This capability adds an important dynamic element to the traditional view. In effect, it enables temporal execution, allowing different stages of a program (e.g., in P4 [8] or NPL [30]) to activate across multiple recirculations in a time-shared manner. By leveraging this mechanism, the state can be distributed over time, facilitating the reuse of limited resources (e.g., registers and match keys in MATs).333Recirculation in SpliDT is used as a fast, in-band control channel—not for forwarding data traffic—avoiding the overhead of the software control plane.

By carefully restructuring DTs (e.g., expressed in P4 [8] or NPL [30]), we can exploit this dynamic capability to scale DT-based inference beyond the physical limits of available resources—akin to how CPUs abstract resource constraints by reusing registers over time. Table˜1 shows that for the evaluated datasets (D1–3),00footnotemark: 0 the maximum recirculation path usage is within $20\text{\,}\mathrm{M}\mathrm{b}\mathrm{p}\mathrm{s}$ for the two datacenter environments E1–2, significantly lower than the total available bandwidth of $100\text{\,}\mathrm{G}\mathrm{b}\mathrm{p}\mathrm{s}$.

In the next section, we show how SpliDT leverages the domain-specific properties of DTs and reuse of switch resources (via recirculation) to optimize the Pareto frontier (F1 score vs. number of flow rules) while supporting all stateful features at line rate.

3 Design of SpliDT

We now present the SpliDT design; its partitioned inference architecture for efficient DT execution in the data plane (§3.1) and the custom training framework, for effective model configuration through design search (§3.2). We then bring these components together to show how SpliDT enables scalable and resource-efficient inference in practice (§3.3).

3.1 Partitioned Inference Architecture

As illustrated in Figure˜4, SpliDT’s partitioned inference architecture operates in two phases: (1) Feature Collection and Engineering (§3.1.1) and (2) Subtree Model Prediction (§3.1.2). For each flow, it iteratively processes windows of packets using the active subtree within each DT partition, leveraging resource reuse (i.e., registers and match keys) via recirculation (§3.1.3).

3.1.1 Feature Collection and Engineering.

In SpliDT, we maintain three distinct register-array sets to manage stateful feature collection and subtree selection for prediction, as shown in Figure˜4. These registers include: (1) reserved state registers for tracking metadata such as the subtree ID (SID) and per flow packet counters, (2) registers for computing intermediate and dependent states (e.g., timestamps for inter-arrival time (IAT) calculations), and (3) $k$ registers for storing stateful features specific to the active subtree within the current DT partition.

INFO: Reserved and dependency chain registers can significantly limit the number of features per subtree ($k$) as they must scale alongside the $k$ features to support the same number of flows. This contention for register space creates a tradeoff that must be carefully managed to balance feature capacity and flow scalability in SpliDT, §3.2.

Upon packet arrival, SpliDT hashes its 5-tuple using CRC32 [38] to determine the register index corresponding to the flow. First, it retrieves the subtree ID (SID) from the reserved register array and updates the packet count in the second register array. Next, depending on the use case (and dataset), some DT models require intermediate values to compute stateful features before prediction. For instance, IAT computation requires storing the previous packet’s timestamp to calculate the inter-packet gap.

To support such computations, SpliDT implements a dependency chain, a sequence of register arrays distributed across multiple pipeline stages to enable hierarchical computation. Since programmable data planes [41, 42, 9, 11, 10, 56] cannot process dependent data within a single stage, computation must be spread across multiple stages. In our evaluations, the deepest observed dependency chain was 3 stages—a depth well within the capabilities of modern data-plane architectures.

Operator Selection. Since each subtree may require a different operation to compute its stateful features, SpliDT dynamically updates the operation applied to each feature in every flow window. To achieve this, SpliDT utilizes match-action tables (MATs) for each stateful feature, acting as selectors to apply the appropriate operation on demand (Figure˜4).

At compile time (§3.2.1), SpliDT populates these tables with rules that define which operator to apply for each subtree. The MATs match on the subtree ID (SID) and select the corresponding action to perform the necessary computation. Modern data-plane architectures support the parallel execution of multiple MATs within a stage. For instance, Tofino1 supports up to 16 MATs with 750 entries each, which is well within the requirements of our design—SpliDT utilizes only six MATs to support $k=6$ stateful features in our evaluations (§5), with each table containing at most 200 entries.

Lastly, to select the appropriate features to populate in the feature registers, prevent continuous feature updates on every arriving packet, and identify window boundaries, SpliDT incorporates additional match fields in the MATs. For instance, to update a stateful feature only on SYN packets (such as SYN packet count), the MATs can include TCP flags as a match condition, ensuring the feature update is triggered only when a SYN is received.

3.1.2 Subtree Model Prediction.

The prediction phase (Figure˜4) executes the decision tree (DT) model using the Range Marking Algorithm [85], which maps the partitioned DT into a sequence of MATs. The first group of MATs, the $k$ match key generators, constructs the match keys required for the DT model. As with the operator selection tables, SpliDT maintains a separate MAT for each feature, using precomputed rules to determine which stateful feature (register) serves as the key. Current feature values—accumulated over the most recent flow window—are stored in dedicated metadata headers. These metadata values are then used as match keys in the generator tables. Each table’s action produces a range mark, a unique bit string written to a corresponding metadata field. These per-feature range marks, combined with the subtree ID (SID), form the match keys for a final MAT that implements the Range Marking Algorithm, encoding the DT model rules and performing classification.

If the current subtree is in an intermediate partition, classification yields the next SID; if it is in the final partition or an early-exit node, it outputs the flow’s final class label. In the latter case, the label is sent to the controller as a digest. Otherwise, the next SID is recirculated via the resubmission channel (§3.1.3), updating the SID register and resetting the dependency chain and $k$ stateful feature registers in preparation for the next flow window.

3.1.3 Resource Reuse via In-Band Control Channel (Recirculation).

At the end of inference for each subtree, the next subtree ID (SID) is propagated to the stateful registers in the feature collection and engineering stages via resubmission, which serves as an in-band control channel [41]. A single packet—triggered after processing the corresponding flow window—is resubmitted with the updated SID encoded in a metadata header field. This resubmission imposes minimal bandwidth overhead (see Table˜1), as only one control packet is required per flow window. This mechanism allows SpliDT to perform partitioned DT inference incrementally, reusing the same pipeline resources across subtrees without introducing resource conflicts or execution overhead.

3.2 Custom Search/Training Framework

We now describe SpliDT’s design search framework for generating optimal DTs (§3.2.1) and partitioning them into subtrees using a custom training algorithm (§3.2.2).

3.2.1 SpliDT Design Search.

Figure˜5 illustrates the overall workflow of the SpliDT framework. The goal is to generate a Pareto frontier of partitioned DT configurations. Given a set of optimization objectives, input parameter ranges, and hardware/performance constraints, a Bayesian Optimization (BO) search phase begins, iteratively proposing model configurations for evaluation. For each configuration, we query a corresponding window-based training/test dataset based on the suggested number of partitions,444These datasets are preprocessed offline and can be efficiently stored and queried using commercial databases, e.g., PostgreSQL [33] or MongoDB [39]. and use SpliDT’s custom partitioned DT training algorithm (§3.2.2) to train the model and evaluate its F1 score. We then generate the corresponding TCAM entries, compute hardware resource usage, determine flow scalability, and assess whether the model can be deployed at line rate on the target switch. The results—F1 score, supported flow count, and feasibility metrics—are fed back into the BO loop to guide the next iteration; this process continues for a predefined number of iterations.

In the following sections, we detail each stage of this workflow, beginning with the inputs to the BO search.

Parameter Space, Objectives, and Constraints. To initiate the design search, the user specifies three key components that define the configuration space and guide the optimization:

• •

Model Hyperparameters: These define the structure and complexity of the partitioned DT. The search space includes: the maximum tree depth ($D$), the number of features per subtree ($k$), and a list of partition sizes $[i_{1},i_{2},\dots,i_{p}]$, where $p$ is the number of partitions and the sum of the partition sizes equals the total tree depth, i.e., $D=\sum[i_{1},i_{2},\dots,i_{p}]$.

• •

Hardware and Performance Constraints: These reflect the capabilities of the target platform (switch or SmartNIC), including available TCAM blocks, register space, pipeline stages, and recirculation bandwidth. They ensure that candidate models are not only accurate but also deployable within the hardware’s resource envelope, at line rate.

• •

Optimization Objectives: The design search seeks to jointly maximize model accuracy (e.g., F1 score) and flow scalability (e.g., number of concurrent flows), yielding a Pareto frontier of configurations that effectively trade off between these two metrics.

Together, these inputs, along with the dataset specification, are fed into the Bayesian Optimization (BO) loop that drives the search for feasible and high-performing DT configurations for SpliDT.

Bayesian Search. Given the input search space and objectives, the framework performs a design-space exploration to generate a Pareto frontier of SpliDT DT configurations for a target dataset. Bayesian Optimization (BO) is a black-box optimization method [72] designed for optimizing expensive-to-evaluate functions [53, 49, 27]. It constructs a probabilistic surrogate model, typically a Gaussian Process (GP) or Random Forest, to approximate the objective function, and employs an acquisition function to guide the selection of promising candidates. By balancing exploration (sampling new regions) and exploitation (refining high-performing regions), BO efficiently searches high-dimensional spaces with minimal evaluations—ideal for costly tasks such as hyperparameter tuning or model training [65].

At each step, the BO search proposes several DT configurations to evaluate in parallel. Each configuration is used to train a partitioned DT using SpliDT’s custom training algorithm (§3.2.2), which is then evaluated on a test dataset. Subsequent stages assess the model’s hardware resource usage, deployment feasibility, and supported flow count; these metrics are fed back to the BO loop to inform the next iteration. This process continues for a predetermined number of iterations.

Subtree Rule Generation. Given a trained partitioned DT, we generate the corresponding TCAM rules to represent the model in the data plane. We adopt the Range Marking algorithm [85], which efficiently encodes decision tree rules by mapping feature value ranges to compact ternary bit strings. It segments each feature’s domain into non-overlapping ranges and assigns a unique range mark (bit string) to each, ensuring that merged ranges retain distinct and unambiguous encodings. For each feature, its thresholds from the trained partitioned DT are translated into ternary matches, with the associated range mark output actions (i.e., using P4). These TCAM entries are installed in feature tables, producing range marks as match keys for the subsequent model table (Figure˜4).

A second set of TCAM rules is then generated to encode the model logic: these rules match on feature range marks and return either the next subtree ID (for intermediate partitions) or the final prediction class (at the last partition). This encoding maps each DT leaf to a single TCAM rule, effectively avoiding rule explosion.

Both feature and model TCAM entries are generated for each subtree in the partitioned DT and installed into the corresponding tables in the data-plane pipeline (Figure˜4). Each rule also includes an exact match on the subtree ID (SID) to ensure the correct subtree is selected for each partition.

Resource Estimation and Feasibility Testing. To evaluate each candidate configuration, we estimate the number of TCAM blocks and pipeline stages required for the feature and model tables using a target-specific analytical model (e.g., BF-SDE [20], P4Insight [19], NetASM [63], and Nvidia DOCA P4 Developer Toolkit [55]). We then determine the number of remaining stages available for register allocation and bookkeeping logic, which directly impacts the number of flows the model can support concurrently. Recirculation overhead, in terms of in-band control traffic, is estimated using: (1) the number of partitions, which dictates the number of recirculated packets per flow; (2) flow-size distribution observed in real-world datacenter workloads (§5); and (3) the number of active flows concurrently issuing recirculations. A design is deemed feasible if it fits within the target’s TCAM, register, and MAT stage budgets while keeping recirculation traffic within available bandwidth limits.

The feasibility outcome (yes/no), along with the model’s F1 score and supported flow count, is fed back into the BO loop to guide subsequent iterations. At each step, the BO search proposes new parameters (and configurations), gradually converging on models that jointly maximize accuracy and flow scalability, while satisfying the target resource constraints.

3.2.2 SpliDT Custom Training.

Algorithm˜1 outlines the algorithm used to train SpliDT’s partitioned decision trees. Given the overall tree depth, partition sizes, and the number of features per subtree, training begins by learning a single subtree for the first partition using all samples from the corresponding initial window. For each leaf node in this subtree, we identify the subset of training samples that reach that node and use only those samples—along with their associated window for the next partition—to train the corresponding subtree in the following partition. Leaf nodes that do not reach the maximum depth in a partition do not generate further subtrees. This recursive approach allows each subtree to specialize based on the subset of flow packets it receives, enabling window-based inference matching the data distribution observed during training.

3.3 Putting It All Together

To demonstrate how SpliDT’s design search and partitioned inference architecture operate in practice, we walk through a complete end-to-end example.

The process begins with the user providing a labeled dataset, model hyperparameters, resource constraints, and optimization objectives (§3.2.1). SpliDT then invokes Bayesian Optimization (BO) to explore the configuration space and identify a Pareto-optimal model that jointly maximizes F1 score and flow scalability. For one such data point (Figure˜3), the selected model had a tree depth of $D=6$, with 4 features per subtree and partition sizes of $[2,3,1]$ across 3 partitions, supporting up to 1 Million concurrent flows.

Given this configuration (Figure˜3), SpliDT uses its custom training algorithm (§3.2.2) to train 6 subtrees: subtree 1 in the first partition, subtrees 2–3 in the second, and subtrees 4–6 in the third. Some subtrees are skipped due to early exits. SpliDT then generates TCAM rules for each subtree using the Range Marking Algorithm [85], and compiles target-specific code (e.g., P4) for feature extraction and inference logic. These rules are installed into the data plane’s MATs: feature tables encode stateful features into range marks, while model tables use them to either classify flows or select the next subtree (Figure˜4). All subtrees are present on the switch, but only one is active per flow at a time, requiring just 4 feature registers per flow.

Referring to the example in Figure˜3, at runtime, a new flow begins with subtree ID, SID = 1, in partition $P_{1}$. The switch collects the features required by this subtree (e.g., $f_{1}$, $f_{2}$, $f_{4}$) over the first window of packets. Once the window concludes, the subtree predicts SID = 3 in partition $P_{2}$, prompting a single recirculation that updates the flow’s SID and clears its feature and dependency-chain registers. The second window is processed using the features needed by subtree 3 (e.g., $f_{2}$, $f_{6}$, $f_{8}$, $f_{9}$). Following this, the model selects subtree 5 in partition $P_{3}$, and another recirculation updates the SID = 5. The third window is then processed with the features required by subtree 5 (e.g., $f_{4}$, $f_{10}$), and the final prediction is emitted. No further recirculations are needed.

This pipeline executes independently per flow, with register state indexed via 5-tuple hashing to avoid conflicts. In this example, SpliDT supports 1 Million concurrent flows, each maintaining only 4 stateful feature registers and a single SID register—demonstrating its scalability and efficiency under practical resource constraints.

4 Implementation

We implement the SpliDT framework in Python v3.10.13, allowing seamless integration with widely used libraries such as Pandas [58], Scikit-Learn [59], and TensorFlow [69]. At the core of SpliDT’s optimization process is HyperMapper [53] (commit:3dfa8a7), which uses Bayesian Optimization (BO) to efficiently search for the best model configurations. Unlike other BO frameworks such as OpenTuner [3], HyperOpt [7], and GPflow Opt [45], HyperMapper supports multi-objective optimization, diverse parameter types (real, integer, categorical), and feasibility testing [53]. These features allow us to optimize SpliDT models for multiple goals simultaneously, such as maximizing F1 score and flow capacity. The feasibility testing feature also helps eliminate DT configurations that exceed available switch resources (e.g., TCAM limits) or introduce excessive resubmission traffic. The BO experiments are configured via YAML, specifying datasets, parameter space, objectives, and switch constraints.

To train and test SpliDT’s partitioning strategy, we use the DecisionTreeClassifier class from Scikit-Learn [59] and recursively generate subtrees for all partitions. We employ the Range Marking Algorithm, as described in NetBeacon [85], to generate TCAM rules. We parallelize evaluations using Python’s ProcessPoolExecutor from the concurrent.futures module to speed up the BO search. For data-plane inference, we implement SpliDT in P4 [8] and compile it using BF-SDE v9.11.0 for the Tofino1 [41] switch. Finally, to install TCAM rules into the switch, we use bfrt_grpc client API in Python.

In total, the SpliDT framework consists of 4,700 lines of Python code. The P4 implementation and controller required 1,600 and 240 lines, respectively. Each experiment, corresponding to a dataset, was defined using 400 lines of YAML, totaling 2,800 lines for all seven datasets (D1–7).

5 Evaluation

We evaluate SpliDT DTs for their end-to-end classification performance and resource utilization (§5.2) and perform microbenchmark experiments (§5.3).

5.1 Experiment Setup

Testbed Environment. Our testbed includes two servers, one as a traffic generator and the other as a receiver, connected through an Edgecore Wedge 100-32X Tofino1 switch [41] running Stratum OS [28]. Each server has a 64-core Intel Xeon Platinum 8358P CPU @$2.60\text{\,}\mathrm{G}\mathrm{H}\mathrm{z}$, $512\text{\,}\mathrm{G}\mathrm{B}$ RAM, and runs Proxmox VE v8.0.4 [60]. They are equipped with dual-port Intel XL710[40] 10/40G NICs and Nvidia ConnectX-6 [54] 100G NICs, and use MoonGen [26] for traffic generation and reception. The same servers run our Bayesian Optimization (BO) experiments, with 500 iterations and 16 parallel evaluations per iteration for each dataset.

Baselines, and Real-World Applications and Environments. Our baselines include Leo [43] and NetBeacon [85], two state-of-the-art data plane DT implementations. Both baselines support stateful (top-$k$) DTs and improve flow scalability with respect to tree depth via efficient TCAM rule generation. To ensure fairness, we allocate the full switch pipeline (all stages) to each baseline, including ours, and evaluate the best-performing model each can support using all available hardware resources. Table˜2 summarizes the seven real-world datasets (D1–7) used in our evaluation, spanning use cases such as intrusion detection, traffic classification, and modern attack detection (e.g., DoS, botnets, infiltration). Together, these datasets allow for a robust evaluation of the performance of SpliDT DTs against baselines across diverse network security tasks. We also evaluate the volume of resubmitted traffic under two representative data center workloads—E1: Webserver (WS) [61], with many long-lived flows, and E2: Hadoop (HD) [61], characterized by short, bursty mice flows.

Dataset Generation. To generate per-flow packet windows for training SpliDT DTs, we extended the widely-used CICFlowMeter tool [32, 31, 47]. By default, CICFlowMeter identifies flows by 5-tuples, computes 78 flow-level features, and emits statistics only at the FIN packet, discarding intermediate data. We modified it to output statistics at every window boundary (e.g., each quarter of a flow for 4 partitions) and reset flow state after each window. While NetBeacon’s phases resemble SpliDT’s windows, they differ in two key aspects: (a) phase intervals grow exponentially (i.e., 2, 4, 8, 16, …), and (b) flow statistics are retained across phases, resulting in the same top-$k$ features being reused. For each dataset in Table˜2, we generate up to 7 partitions555We experimented with more than 7 partitions, but classification accuracy significantly drops beyond this point. for SpliDT. For NetBeacon [85], we adopt the same phase intervals as their public artifact.

5.2 End-to-End Analysis

Pareto Frontier. Across all datasets, SpliDT DTs consistently outperform the baselines by achieving a superior tradeoff—delivering higher accuracy at the same flow count. This consistent ability to balance model performance (i.e., accuracy) with scalability (i.e., number of flows) allows SpliDT to define the Pareto frontier across all evaluated datasets (Figure˜6). The resulting tradeoff curves are monotonically decreasing: models yield higher accuracy when supporting fewer flows and progressively trade off feature coverage and model complexity to scale to larger flow counts.

Feature Scalability and Resource Utilization* (TCAMs, Registers, and Recirculation Bandwidth)*. As shown in Table˜3, SpliDT DTs consistently deliver the highest accuracy across datasets, balancing tree depth and partitioning effectively. They use less register space while maintaining competitive accuracy, and optimize TCAM usage with manageable register overhead. For instance, in D6, SpliDT supports more features than both baselines across all flow sizes (within 160-, 96-, and 32-bit budgets for 100K, 500K, and 1M flows, respectively), while also deploying deeper trees. As shown in Table˜5 and Figure˜12, SpliDT’s packet recirculation remains well within the 100 Gbps resubmission buffer capacity and does not degrade model responsiveness, matching NetBeacon [85] and Leo [43] in per-flow time-to-detection (TTD). These results underscore the efficiency of SpliDT DTs in maximizing accuracy under the same resource constraints as competing baselines.

Offline Software Overhead. SpliDT’s offline design search reaches maximum accuracy for all datasets within 150 iterations (Figure˜7). As shown in Table˜4, training dominates the per-iteration cost (88% on average), followed by the BO stage (12%), which leverages HyperMapper [53] to guide the search toward high-performing models. Training is computationally intensive due to the recursive nature of SpliDT DTs, which involves deeper trees, more subtrees, and repeated dataset partitioning. Despite this, the full search completes in an average of 3.8 hours per dataset, yielding optimized, deployment-ready models.

5.3 Microbenchmarks

SpliDT framework finds optimal model parameters for partitioned decision trees.** The SpliDT DSE framework navigates the parameter space for tree partitioning to identify high-performing models for a given number of flows. Figure˜9a shows the Pareto frontiers for fixed tree depths (10, 20, 30) as we vary the features per subtree and partition count. For instance, in D2, a depth of 30 generally yields a better frontier than depth 10—except at high flow counts (e.g., 1M flows), where the simpler model performs comparably. Each dataset benefits from a different tree depth depending on its feature complexity and class distribution, and SpliDT effectively adapts to these differences. Figure˜9b fixes the number of partitions (1, 3, 5), showing that fewer partitions often yield better frontiers due to more packets per window, enhancing model accuracy. Lastly, Figure˜9c varies the number of features per subtree (1, 2, 3): using more features improves accuracy but reduces scalability; fewer features increase flow support at the cost of classification performance.

SpliDT DTs outperform baselines for any given TCAM budget.** Figure˜9 shows that SpliDT DTs consistently achieve higher accuracy than the baselines across different TCAM budgets. This improvement stems from reducing the match key size by decreasing the number of features ($k$) used in the model table for the active subtree.

SpliDT DTs recirculate limited traffic without impacting end-to-end flow-level time-to-detection (TTD).** Table˜5 shows that in both environments: WS and HD, packet recirculation by SpliDT DTs stays well within the resubmission buffer capacity of $100\text{\,}\mathrm{G}\mathrm{b}\mathrm{p}\mathrm{s}$. Additionally, as shown in Figure˜12, the per-flow time-to-detection (TTD) for D3—defined as the time from the start of tree traversal to the final inference decision—closely matches that of NetBeacon and Leo, while sustaining 9% and 16% higher F1 score, respectively. This indicates that recirculation overhead in SpliDT DTs does not negatively impact model performance or responsiveness.

SpliDT DTs scale feature count with constant register space.** SpliDT DTs maintain a constant register footprint, based solely on $k$ features per subtree, regardless of the total number of unique features used across the entire tree (Figure˜12). As shown in Table˜3, SpliDT can store up to 30 distinct 32-bit features within a 128-bit register budget. This demonstrates that by dynamically multiplexing features across partitions and subtrees, SpliDT minimizes register overhead. This efficient use of stateful memory enables the deployment of deeper, more complex DTs in the data plane.

SpliDT DTs can accommodate a larger number of flows with reduced feature bit precision while maintaining higher F1 scores.** We lower the bit precision of all features from 32 bits to 16 and 8 bits, respectively, and measure the resulting impact on accuracy and flow scalability. Figure˜12 shows that this reduction affects all models similarly, as they are all decision trees. On average, we observe a 7% (14%) drop in accuracy with 16-bit (8-bit) precision, while the maximum number of supported flows increases to 2M (4M). In both cases, SpliDT continues to yield a better Pareto frontier than the baselines, due to its enhanced feature coverage.

6 Limitations & Future Work

Adaptive Window Sizing. Currently, SpliDT uses fixed per-flow window sizes, which may not always reflect real-world traffic dynamics. This limitation can affect both model accuracy and resource efficiency, particularly under bursty or highly variable traffic patterns. Future work could explore adaptive window sizing across partitions, enabling SpliDT to optimize feature extraction based on observed flow characteristics.

Security & Robustness. SpliDT’s reliance on flow size information stored in packet headers makes it vulnerable to scenarios where this information is hacked or spoofed. If an attacker manipulates this field, it could lead to incorrect window boundaries, resulting in misclassifications, resource mismanagement, or even denial-of-service (DoS) attacks. Enhancing security measures to validate and protect header information is a critical area for future improvement.

7 Related Work

Decision Tree (DT)-Based Inference in the Data Plane. Several efforts have mapped decision trees (DTs) to programmable data planes (i.e., switches) for real-time classification. Leo [43] scales DTs by optimizing match-action table (MAT) representations, while NetBeacon [85] introduces ternary table encodings for large-scale inference. Mousika [75] applies knowledge distillation to optimize binary DTs, reducing resource usage while preserving hardware compatibility. Planter [84] incorporates packet-level summaries into DT-based classification, and IIsy [79] maps ML models to MAT pipelines to improve inference efficiency. pForest [12] generalizes DTs to random forests, enabling dynamic feature selection based on real-time traffic. Unlike these approaches, SpliDT removes the static top-$k$ feature constraint and enables dynamic feature allocation across tree partitions. In contrast to prior single-pass DT execution, SpliDT introduces incremental inference via subtree transitions—optimizing stateful feature storage and scaling to millions of flows with reduced TCAM overhead.

Neural Network (NN)-Based Inference in the Data Plane. NN-based approaches in the data plane prioritize high expressiveness but often encounter resource limitations. Taurus [67] proposes a MapReduce-inspired ML inference framework, while Homunculus [68] automates ML model deployment on switches and SmartNICs. Brain-on-Switch (BoS)[83] incorporates RNNs and transformers for sequential processing, improving accuracy but demanding significant compute resources. N3IC[66] explores binary neural networks for low-latency inference, and ServeFlow [50] introduces neural accelerators to support high-accuracy classification with minimal overhead. AC-DC [44] aims to balance performance and efficiency in dynamic traffic scenarios. To support such workloads, recent converged platforms like NVIDIA BlueField-3 [56] combine embedded CPU cores, network interfaces, and hardware accelerators within a SmartNIC, enabling ML inference offload without host CPU intervention. Similarly, Broadcom’s Trident 5 [10] introduces a rigid on-chip neural inference engine (NetGNT) that performs real-time traffic analysis at line rate without impacting switch throughput or latency. While such architectures are promising for NN-based inference, they rely on dedicated hardware engines and are inherently constrained in terms of general-purpose model flexibility and resource sharing. In contrast, SpliDT is designed explicitly for resource-constrained, line-rate execution in existing data planes. By partitioning decision trees and reusing MAT stages and registers via recirculation, SpliDT supports scalable, high-accuracy inference without requiring external accelerators or specialized NN engines.

Lookaside Accelerators for ML inference. Modern ML inference increasingly leverages high-performance accelerators such as GPUs [21], FPGAs [77, 78], TPUs [18], and SmartNICs [1, 56]. These platforms excel at offline or near-real-time inference, where flexible compute and memory resources enable support for deep neural networks and large-scale models. As a result, many cloud and data center architectures adopt a lookaside model, offloading packets or flow metadata to external co-processors for ML inference [34, 74, 64]. While this approach works well for sampled traffic or background analytics, it is ill-suited for high-speed networks that demand line-rate packet processing with microsecond-level latency [85, 43, 10]. In such environments, external accelerators cannot consistently match the throughput and latency requirements of the data plane, forcing operators to rely on packet sampling, which reduces coverage and weakens real-time detection guarantees. In contrast, SpliDT is explicitly designed for in-network inference, enabling scalable, line-rate DT processing entirely within the programmable data plane. By eliminating the need for lookaside offload and avoiding dependence on specialized external accelerators, SpliDT ensures that line-rate, low-latency inference can be performed directly on the switch fast path—making it a practical solution for modern, high-throughput network environments.

8 Conclusion

We demonstrate in this paper how SpliDT recasts decision tree (DT) deployment in programmable data planes through partitioned, window-based inference. Unlike prior approaches that rely on static top-$k$ feature sets and one-shot execution, SpliDT enables incremental inference via subtree transitions and dynamic feature reuse. Combined with a Bayesian optimization (BO)-based training pipeline, SpliDT balances model accuracy and hardware efficiency, scaling to millions of flows at line rate.

SpliDT shows that complex and expressive in-network ML inference is not only feasible but practical—without compromising model accuracy or performance. As network traffic grows in volume and speed, SpliDT offers a scalable and efficient path to real-time, high-throughput inference under tight hardware constraints. By enabling informed in-network decision-making, SpliDT contributes to ongoing efforts toward a more responsible use of ML in networking.