An Efficient Linux Kernel Implementation of Service Function Chaining for legacy VNFs based on IPv6 Segment Routing

TL;DR

This paper presents an optimized Linux kernel implementation for service function chaining using IPv6 Segment Routing, supporting legacy VNFs with minimal overhead and improved scalability.

Contribution

The authors extend Linux kernel SRv6 support with a new SR-proxy design that enhances scalability and reduces overhead for legacy VNF support.

Findings

SRNKv2 achieves scalability independent of the number of VNFs.

Overhead of SRNKv2 is approximately 3.5%.

Enhanced Linux Policy Routing framework improves performance.

Abstract

We consider the IPv6 Segment Routing (SRv6) technology for Service Function Chaining of Virtual Network Functions (VNFs). Most of the VNFs are legacy VNFs (not aware of the SRv6 technology) and expect to process traditional IP packets. An SR proxy is needed to support them. We have extended the implementation of SRv6 in the Linux kernel, realizing an SR-proxy, referred to as SRNK (SR-Proxy Native Kernel). The performance of the proposed solution (SRNKv1) has been evaluated, identifying a poor scalability with respect to the number of VNFs to be supported in a node. Therefore we provided a second design (SRNKv2), enhancing the Linux Policy Routing framework. The performance of SRNKv2 is independent from the number of supported VNFs in a node. We compared the performance of SRNKv2 with a reference scenario not performing the encapsulation and decapsulation operation and demonstrated that the overhead of SRNKv2 is very small, on the order of 3.5%.

Tables

Table 1. TABLE I: Comparison of SR proxy behaviours

	End.AD	End.AS	End.AM
Generate traffic	Yes	Yes	No
Modify packets	Yes	Yes	No
Stateless	No	No	Yes
State-config	Auto	Manual	N/A
Traffic supported	IPv4/IPv6/L2	IPv4/IPv6/L2	IPv6

Table 2. TABLE II: Throughput (PDR@0.5%) in kpps

SRNKv2	Baseline (End)	SREXT
444.2	461.1	472.3

Table 3. TABLE III: Extended SRv6 policy rule performance

	Baseline	Ext. SRv6 Rule	80 Plain Rules
PDR@0.5% [kpps]	461.1	448.5	328.0
Performance penalty	-	2.7%	28.1%

Full text

I Introduction

Network Operators are facing difficult challenges to keep up with the increasing demand for capacity, the need to support fast service creation and at the same time the goal of reducing the costs. Network Function Virtualization (NFV) [1] [2] and Software Defined Networking (SDN) represent an answer to these challenges and are changing the way IP networks are designed and operated. Leveraging Cloud Computing principles, NFV moves the traditional data-plane network functions from expensive, closed and proprietary hardware to the so-called Virtual Network Functions (VNFs) running over a distributed, cloud-like infrastructure referred to as NFVI (NFV Infrastructure). The SDN architecture splits the data and control planes and moves the intelligence to the SDN controller. SDN aims at simplifying the introduction of new services and fostering flexibility thanks to the centralized network state view.

The concept of services chaining (also known as Service Function Chaining - SFC [3]) is directly associated to NFV. Actually, the idea of creating a processing path across services pre-dates the NFV concept as stated in [4] and [5]. In fact, service chaining has been traditionally realized in a static way by putting hardware functions as middle-points of the processing paths and in some cases by diverting the forwarding paths with manual configuration of VLANs stitching or policy routing. However, these “static” approaches comes with several drawbacks which are detailed in [4]. In particular, they are intrinsically difficult to scale and hard to reconfigure. On the other hand, the current view of SFC applied to NFV is that it has to be highly dynamic and scalable.The IETF SFC Working Group (WG) has investigated the scenarios and issues related to dynamic service chaining [4] and proposed a reference architecture [3]. The main logical elements of this architecture are i) Classifiers; ii) Service Functions Forwarders (SFF), iii) the Service Functions, iv) SFC proxies. The Classifiers match the traffic against a set of policies in order to associate the proper Service Function Chain. The SFFs forward the traffic towards the Service Functions or towards other SFFs and handle the traffic coming back from the Service Functions. The SFC framework proposed in [3] does not pose any restriction on the function that can be chained: they can be both virtualized (VNFs) or physical functions (PFs). For the sake of simplicity, hereafter in the paper we will only refer to the virtualized case (which we believe is the most significant) and will simply use the term VNF instead of Service Function. In this scenario, the forwarding of traffic along a Service Chain needs to be supported by specific protocols and mechanisms that allow the architectural elements to exchange context information. The VNFs can participate to these chaining mechanisms and in this case they are called SFC aware. On the other hand, the legacy VNFs that do not interact with the SFC protocols and mechanisms are called SFC unaware. The SFC proxy elements are needed for the latter type of VNFs. An SFC proxy hides the SFC mechanisms to the SFC unaware VNFs, that will receive and send plain IP traffic. The IETF SFC WG is considering the Network Service Header (NSH) [6] as a specific solution for the realization of the SFC architecture. The NSH header defines the service-level data-plane encapsulation for realizing the VNFs chaining. The NSH header identifies a service chain which is associated to the packets. Moreover, it defines the packet meta-data that can be inserted into the header to exchange state between the nodes of the SFC architecture. In this work we are advocating the use of IPv6 Segment Routing (SRv6) to implement Service Function Chaining [7, 8]. Segment Routing [9, 10] is a form of source routing, which allows to add a sequence of segments in the packet headers to influence the packet forwarding and processing within the network. Segment Routing has been designed and implemented for the MPLS and IPv6 data planes, we only focus here on the IPv6 version, denoted as SRv6. In the SRv6 architecture, the segments are expressed as IPv6 addresses. The SRv6 network programming model [11], leveraging the huge IPv6 addressing space, extends the SRv6 architecture from a simple forwarding mechanism for steering packets to a more general network programming abstraction. A segment can represent an instruction or behavior and not only a network location. Our proposed approach is fully aligned with the network programming model described in [11]. The SRv6 architecture is not limited to Service Function Chaining, which represents only a possible use case. Indeed, SRv6 can support several applications in a network provider backbone like Traffic Engineering, Network Resilience (Fast Rerouting), Virtual Private Networks (VPNs), Multicast, Content Delivery Networks (CDNs). With respect to the MPLS based data plane, SRv6 it has the advantage that it can be better integrated in host networking stack. For this reason Data Center applications could also benefit from SRv6.

A relevant subset of the SRv6 [12] and network programming model [11] specifications have been implemented and integrated in the mainline Linux kernel [13]. In this paper, we rely on this existing work and extend it to focus on the Service Function Chaining of legacy VNFs, which are not able to process the SRv6 headers. The support of legacy VNFs is important for Internet Service Providers (ISP) for different reasons: i) it guarantees a feasible migration strategy saving past investments; ii) it facilitates the interoperability and the multi-vendor scenarios, i.e deployments composed by VNFs of different vendors; iii) the development of SRv6 aware VNFs requires a new implementation cycle which can be more expensive in the short period.

As introduced above, a proxy element needs to be inserted in the processing chain as relay mechanism in order to support SRv6 unaware VNFs (see Figure 1). The latest Linux kernel still lacks of the functionality to implement such SRv6 proxy element. In a prior work (SREXT [14]), we have provided this functionality as an external module not integrated with the most recent SRv6 developments in the Linux kernel. Considering the importance of the support of legacy SR-unaware applications in NFV deployments, the main contribution this paper is the design and implementation of an SR-proxy integrated in the Linux kernel networking components. We refer to this work as SRNK (SR-Proxy Native Kernel). We designed a first version of SRNK and evaluated its performance, identifying a poor scalability with respect to the number of VNFs to be supported. The issue is actually related to the implementation of Policy Routing framework in Linux. Therefore we provided a second design, enhancing the Linux Policy Routing framework, whose performance does not depend on the number of supported VNFs in a node.

The content of the paper is as follows. Section II introduces SFC based on SRv6 considering both SRv6 aware and unaware VNFs. The proposed design and implementation of SRv6 Proxy to support legacy VNFs in the Linux kernel is described in Section III. Our testing environment and methodologies for performance analysis are reported in Section IV. Sections V details the performance evaluation of the implemented solutions. Finally, in Section VII we draw some conclusions and discuss future work.

This work has been performed in the context of the ROSE research project [15] which focuses on the development of an open source SRv6 ecosystem. The source code of all components of SRNK including the patches to the user space utilities are freely available at [16].

II SFC Based on IPv6 Segment Routing

The Segment Routing architecture is based on the source routing approach ([10] and [9]): it is possible to include a list of instructions (the so called segments) in the packet headers. A comprehensive survey on Segment Routing can be found in [17]

This work considers the use of SRv6 for SFC, leveraging its scalability properties.Thanks to the source routing approach, SRv6 is able to simplify network operations. Generally speaking, the advantage of approaches based on source routing lies in the possibility to add state information in the packet headers, thus avoiding or minimizing the information that needs to be configured and maintained by the internal nodes. The possibility to interact only with the edge nodes to setup complex services is extremely appealing from the point of view of simplicity and efficiency. This greatly improves the scalability of services based on SR and allows simpler and faster service setup and re-configuration. In [18] the scaling capability of Segment Routing has been demonstrated considering an use case of 600,000 nodes and 300 millions of endpoints.

By exploiting the SRv6 approach the VNFs can be mapped in IPv6 addresses in the segments list (SIDs list in SRv6 jargon) and we can represent the VNF chain using this list carried in the Segment Routing Header (SRH).

The SR information can be pushed into the packets using two different approaches, denoted as insert and encap modes, respectively.According to the SRv6 network programming document [11], when a node uses the insert mode the SRH is pushed as next header in the original IPv6 packet, immediately after the IPv6 header and before the transport header. The original IPv6 header is changed, in particular the next header is modified according to the value of SRH, the IPv6 destination address is replaced with the IPv6 address of the first SID in the segment list, while the original IPv6 destination address is carried in the SRH header as the last segment of the list. In this work we only consider the encap mode: the original IPv6 packet is transported as the inner packet of an IPv6-in-IPv6 encapsulated packet and travels unmodified in the network. The outer IPv6 packet carries the SRH header with the segments list.111As any tunneling (encapsulation) method, SRv6 introduces overhead the packets. The insert mode introduces an overhead of $8+N*16[bytes]$ where $N$ is the number of segments, while in the encap mode the overhead is $40+8+N*16[bytes]$.

An SR-aware VNF can process the SRH of the incoming packets and can use it to influence the processing/forwarding of the packets. Such VNFs interact with the node Operating System or with SR modules in order to read and/or set the information contained in the SRH. On the other side, the SR-unaware (legacy) VNFs are not capable to process the SRv6 SFC encapsulation. In this scenario an SR proxy is necessary to remove the SRv6 header and deliver a “clean” IP packet to the VNF. Figure 1 provides the reference architecture for a SRv6 NFV node that includes an SR-unaware VNF (VNF1 in the Figure). We refer to packets incoming to the SRv6 NFV node that should be forwarded to the VNF by the SR-proxy as inbound packets. The SR-Proxy needs to intercept the packets coming out from the VNF and re-apply the SRv6 SFC encapsulation. We refer to these packets as fromVNF packets.

In [7], a set of SR-proxy behaviors have been defined, among them we mention: i) static proxy (also called End.AS behavior); ii) dynamic proxy (End.AD behavior); iii) masquerading proxy (End.AM behavior). The first two cases (static and dynamic proxies) support IPv6 SR packets in encap mode. The encapsulated packets can be IPv6, IPv4 or L2 packets. The SR proxy intercepts SR packets before being handed to the SR-unaware VNF, hence it can remove the SR encapsulation from packets. For packets coming back from SR-unaware VNF, the SR proxy can restore the SRv6 encapsulation updating the SRH properly. The difference between the static and the dynamic proxies is that the SR information that needs to be pushed back in the packets is statically configured in the first case and it is learned from the incoming packets in the dynamic case.Instead, the masquerading proxy supports SR packets travelling in insert mode. It masquerades the SR packets before they are sent to the legacy VNF by replacing the IPv6 destination address (the current SID of the segment list) with the original IPv6 destination (i.e. the last segment in the SID list). It is assumed that a VNF compatible with this operating mode is processing IPv6 packets and does not alter the SRH, it just ignores it. In this way, when packets are received back, the SR proxy can restore the correct information in the IPv6 header in a stateless way, just using the information contained in the SRH.

Let us discuss the operational model and the state information that need to be configured and maintained in the SRv6 NFV nodes. Figure 2 illustrates a SRv6 based NFV domain, in which the VNFs are hosted in different NFV nodes. The packets to be associated to VNF chains are classified in ingress nodes, where the SR encapsulation is added. A network operator willing to use SRv6 SFC chaining for SR-unaware VNFs, will first need to associate VNFs to Segment IDs (SIDs) in the hosting SRv6 NFV nodes. We recall that a SID is represented by an IPv6 address. Each SRv6 NFV node has a pool of IPv6 addresses (prefixes) that are available to be used as SIDs for its VNFs. These prefixes are distributed using regular routing protocols, so that the reachability of all VNFs is assured. The association of the IPv6 address SID to a VNF is a configuration operation to be performed in the SRv6 NFV node and it binds the SID to the virtual interface that connects the SR-proxy to the VNF. This operation is performed when a legacy VNF is created in a NFV node. The corresponding state information is used in the inbound direction, when packets directed to the VNF are processed by the SR-proxy. The second step is to configure a VNF chain across the VNFs that are running over the SRv6 NFV nodes. The VNF chain will be applied to a packet by inserting a SID list in the IPv6 SR header in the ingress node. Therefore, the classification of packets and the association with the SID list has to be configured in the ingress node. Each NFV node which runs a legacy VNF needs the proper information to process the packets in the fromVNF direction. This is done differently for the respective types of proxy. In the static proxy case (End.AS behavior), the state information needed to process the packets coming from the VNF is done by statically configuring the SR-proxy with the SID list to be re-inserted in the packet. Both the dynamic proxy (End.AD behavior) and the masquerading one (End.AM behavior) have the good property that they do not need to be configured when a new chain is added. The dynamic proxy “learns” the SID list from the packets in the inbound direction (and so it saves a state information). The masquerading proxy does not even need to save the state information as the SID list is carried along with the packet through the legacy VNF (which has to be IPv6 and needs to accept the SRH header without interfering with it). Table I compares the different SR proxy behaviors.

In this work, we focus on the design and in-kernel implementation of the SR dynamic proxy as it represents the most versatile solution (being able to support legacy VNFs working with IPv6, IPv4 and L2 packet) and it offers a simple operational model.

III Design of the SRv6 Proxy

This section describes the design and implementation aspects of the SR-proxy. We start with subsection III-A that provides a brief introduction of general concepts that will be extensively used in the paper. It briefly describes the network programming model defined in [11] and how it has been implemented inside the Linux kernel. Then, it presents the so called policy routing that introduces a match-action framework for IP routing in Linux and finally it shows our previous SREXT solution. Subsection III-B presents the first implementation of our SR-Proxy integrated in the kernel, referred to as SRNKv1 (Native kernel v1) and elaborates on its operations. In subsection III-C we analyze the performance issues of SRNKv1 and present our second design and implementation (SRNKv2), discussing its performance improvements.

III-A General Concepts and State-of-the-art

III-A1 Network Programming Model

The SRv6 network programming model [11] extends the IPv6 Segment Routing concept from the simple steering of packets across nodes to a general network programming approach. Quoting from [11] “Each segment represents a function to be called at a specific location in the network”, a function can span from a simple action like forwarding or a complex processing defined by the user. Going into the details, each SRv6 capable node maintains the so-called My Local SID Table [11], each entry of this table maps a segment (SID) into a local function. As a consequence, when a packet enters in an SRv6 enabled node with an active segment matching an entry of the table, the associated function is applied to the packet. Leveraging the fact the segments are represented as regular IPv6 addresses, the node can advertise them using any routing protocol. Combining these “network instructions” it is possible to literally program the network and realize very complex network behaviors.

The association of a function to a SID resembles the execution of the nexthop lookup function in the IP nodes. Indeed, My Local SID Table has been realized in the Linux networking stack (from kernel 4.14) using an IPv6 routing table that contains routes on which custom processing function are associated. In recent Linux kernel implementations, lightweight tunnel (LWT) provides the capability of performing a generic operation on the packets (which can span from a simple encap/decap to a general purpose processing). The Linux SRv6 network programming implementation leverages the mechanism offered by the Linux kernel that allows to associate LWTs to routing entries.

The seg6local LWT is the specific type of lightweight tunnel that supports the SRv6 network programming features in the Linux kernel [19]. Starting from Linux kernel 4.14 a subset of the behaviors described in [11] have been implemented, while the SR proxy behaviors are not supported yet. The purpose of this work is to extend the implementation of the SRv6 network programming model currently available in the Linux kernel to support the dynamic proxy (End.AD behaviour). Figure 3 shows the processing of a SRv6 node where a legacy VNF is deployed.

Let us refer to Figure 3 to explain the details of SRv6 processing in an NFV node hosting an SR-proxy. For the packets in the inbound direction the SR-proxy classifies the packets based on the IPv6 destination address, decapsulates them as needed and forwards to the proper interface towards the VNF. For the packets in the fromVNF direction (i.e. sent back by the SR-unaware applications), the SR-proxy needs to restore the SRH header after the identification of the interface from where the packets are coming. Looking at Figure 3 an inbound packet having as destination address which does not correspond to a VNF (e.g. 2000::1) are simply forwarded by the node over an outgoing interface (oif), looking at the default routing table. The packets having FDF1::2 as IPv6 destination address (and active segment in the segment list) is matched by the node in My Local SID Table, hence the SR-proxy behavior is applied and the packet is forwarded to the VNF1. When considering the packets coming from the legacy VNF1, the proxy restores correctly the SRv6 header and delivers it to the IPv6 processing of the node that will forward to the next hop. Note that My Local SID Table and the normal routing table does not need to be separated, this is actually an implementation aspect. In the current Linux implementation the SID entries can be inserted in any routing table, therefore also in the default routing table.

III-A2 Policy Routing

Policy Routing extends the traditional routing based only on IP destination addresses. With Policy Routing the forwarding decision on a packet can be based on different features of the packets, considering packet headers at different protocol levels, incoming interfaces, packet sizes and so on. According to this extended routing model, the Linux kernel implementation of Policy Routing complements the conventional destination based routing table (that leverages the longest prefix match) with a Routing Policy DataBase (RPDB). In general, each Policy Routing entry in the RPDB consists of a selector and an action. The rules within the RPDB are scanned in decreasing order of priority. If the packet matches the selector of an entry the associated action is performed, for example an action can direct the packet to a specific routing table. The most important consideration for our purposes is that the RPDB rules are sequentially processed, so the performance penalty of checking the rules increases linearly with the number of rules.

III-A3 State-of-the-art - SREXT module

The SREXT module ([14]) is our first implementation of the SRv6 network programming model. When it was designed, the Linux kernel only offered the basic SRv6 processing (End behavior). SREXT is an external module that complemented the SRv6 Linux kernel implementation providing a set of behaviors that were not supported yet. Currently most of the behaviors implemented in SREXT are supported by the mainline of Linux kernel (with the exception of the SR-proxy behaviors). So, following this trend, we decided to implement SR-proxy behaviors that were only available using SREXT directly into the kernel avoiding any extra module functionality and dependency.

In the related work (section VI) we analyze the SREXT shortcomings compared to our solution. As for the SR-proxy, SREXT handles the processing of SR information on behalf of the SR-unaware VNFs, which are attached using two interfaces. SREXT provides an additional local SID table which coexists with the one maintained by the Linux kernel. The SREXT module registers itself as a callback function in the pre-routing hook of the netfilter [20] framework. Since its position is at beginning of the netfilter processing, it is invoked for each received IPv6 packet. If the destination IPv6 address matches an entry in the local SID table, the associated behavior is applied otherwise the packet will follow the normal processing of the routing subsystem.

A secondary table (the so called “srdev” table) is used by SREXT for correctly executing the processing of the inbound and fromVNF packets. As regards the former, once the packet has passed the sanity check and the SRv6 behavior has been applied, SREXT stores in this table the fromVNF interface (where SREXT will receive back the packet from the VNF), the applied behavior, the original IPv6 header and its SRH. On the fromVNF side, the receiving interface is used as a look-up key in the table “srdev”, if an entry is found the headers are re-added (IPv6 control traffic like NDP is dropped) and finally the packet will go through the kernel IP routing sub-system for further processing. A new cli has been implemented for controlling SREXT behaviors and showing its tables and counters.

III-B SRNKv1

In this section we present the design of our first kernel implementation of the dynamic proxy (End.AD behavior), referred to as SRNKv1. Most of the following design choices apply also to the static proxy (End.AS behavior), which can be seen as a by-product of the the dynamic proxy implementation. In order to simplify the discussion we just mention the dynamic proxy in the paragraphs and in the images. SRNKv1 design relies on two distinct LWTs which manage respectively the inbound and fromVNF traffic. For each LWT, state information is maintained in order to correctly perform the proxy operations. In particular, the inbound processing needs an entry on the My Local SID Table and uses a per-network namespace hashtable (per-netns hashtable) to store the headers that have to be restored during the fromVNF processing.

As regards the traffic coming from the legacy VNF, a policy routing entry for each VNF is necessary to classify the packets, a routing table with a default route pointing to the LWT is used for the VNF and finally the per-netns hashtable is used to read the headers stored previously by the inbound processing. Figures 4 show an high-level view of the processing inside a SRv6 enabled node and how IPv6 routing network subsystem interacts with the SRv6 dynamic proxy implementation.

III-B1 Inbound processing

The inbound processing is depicted in Figure 4a. As soon as an IPv6 packet arrives at interface eth0 of the NFV node, it enters into the Linux networking stack. After passing the pre-routing stage, the kernel tries to look up the route with the longest prefix that matches the active segment of the packet. Due to policy-routing settings, the Linux kernel looks first at My Local SID Table and if no matching route has been found, it considers the other tables and possibly moves on the next stages of the processing (input or forward). Figure 4a shows this process in details, the packet destination address matches with prefix sid1 and the correspondent route is used. Therefore, the Linux kernel executes the processing function associated with the route: the inbound End.AD operation. The inbound End.AD operates in three different stages: i) it pops the outer IPv6 and SRv6 headers from the incoming packet; ii) it updates the SID pointer of the SRv6 header to select the next one; iii) it stores such retrieved headers into a per-netns hashtable data structure; iv) it sends out the decapsulated IPv6 plain packet to its designated legacy VNF.

Removed headers at step (i) are indexed in the per-netns hashtable by using the identifier of the packet outgoing interface (oif), the one used to communicate with the legacy VNF (veth0 in Figure 4a). Due to the necessity of sharing IPv6 and SRv6 headers between inbound and fromVNF processing, the choice of storing them within a external shared data structure turned out to be the right solution. This design simplifies the access pattern to the stored data, as well as it increases performance. Indeed, the hashtable is well suitable to support fast data retrieving with a very low computational cost and, ideally it is independent with regard to the number of stored entries.

From a configuration point of view, the inbound processing just relies on the plain IPv6 routing through My Local SID Table: the new route is added with the ip -6 route add command of the iproute2 suite, by also specifying the behavior to be activated in the parameters of the command.

Appendix A provides further details on the configuration of the inbound processing.

III-B2 Auto-learning Process

The auto-learning process consists in learning the information related of the VNFs chain (i.e., the list of segments in the SRv6 header) from the inbound packets, without the need of a static configuration. The learned information is saved in a per-netns hashtable. We have introduced an age parameter to control the rate at which the per-netns hashtable can be updated. This parameter can be set during the setup of the LWT routing entry in My Local SID Table. When different from 0, the age parameter represents the minimum interval (in seconds) between two write operations in the per-netns hashtable for the same VNF. Setting the age to 1 second corresponds to a maximum reconfiguration delay of 1 second for a NFV node when the VNF chain is changed by an ingress node and this is the default we used in our experiments. If age equals 0, the per-netns hashtable is updated for every inbound packets, providing the fastest possible reconfiguration time for a VNF chain. In the performance evaluation section, we have analyzed the performance cost for the continuous updating of the per-netns hashtable with respect to the default minimum reconfiguration delay of 1 second. The age parameter registers the last time the headers have been updated and it is used also to determine, when a packet is received, if it is the time to replace stale data with new fresh one. The auto-learning operation is performed only during the inbound processing. The learned information (VNFs chain) is retrieved during the fromVNF processing using the incoming interface222the current implementation of the dynamic proxy assumes that the same interface is used to interact with VNF in the two directions of the packet to rebuild the whole SRv6 packet ready for being forwarded into the network.

Setting properly the age parameter has an important impact on the performance of the system and a proper trade-off is necessary according to the use case to be supported. In a shared-memory producer-consumer context, we can identify the inbound processing as the content producer, and the fromVNF one as the consumer. Indeed, the former is in charge of keeping the per-netns hashtable up-to-date, while the latter accesses the structure for retrieving the headers. Considering this model, the aging parameter can be seen as the upper-bound of data production/refresh rate. By setting it to the maximum limit, it is possible to prevent overloading of the SRv6 NFV node caused by high-rate writing in the shared memory.

This problem is particularly noticeable in all of those systems based on multi-core architectures: the Linux networking stack allows to assign the received packets to all available computing units in order to process them in parallel and to support high data rates. However, this means that several End.AD processing operations may occur at once and, potentially, they may involve updating the same IPv6 and SRv6 headers. Very frequent and simultaneous shared memory updates by multiple CPUs can lead to conflicts that can negatively affect the overall performance of the system. For all these reasons, small values of the age parameter make the system more responsive to chain (SRv6’s segment list) changes, but on the other side they can push heavy and unnecessarily load to the SRv6 NFV node due to high data refresh rate.

III-B3 End.AS design

The End.AD differs from the End.AS just on the way the stored headers are managed. The End.AS behavior is a simplification of the End.AD because it does not need to deal with the auto-learning process. Indeed, it uses chain information which has been saved once during the behavior configuration. The list of segments does not change during the entire life of the End.AS instance unless it is first deleted and then saved with new headers values.

III-B4 FromVNF Processing

The fromVNF LWT tunnel is meant to work in tandem with its inbound counterpart. fromVNF packets do not carry any SID as it happens for the inbound ones. As result, in order to select the correct (fromVNF) LWT tunnel and processing each packet accordingly, we can rely only on the incoming interface between the VNF and the NFV node through which packets come back. Hence, we add an entry in the IPv6 Routing Policy DB (RPDB) for each VNF to be supported. Every RPDB entry is also known as IPv6 rule, as the command used to configure it is ip -6 rule. The rule points to a different routing table for each VNF, in which there is only a default route, pointing to the LWT tunnel associated to the VNF. This means that for $N$ VNFs, we will have $N$ rules and $N$ routing tables. Figure 4b provides a representation of the described fromVNF processing.

Let us analyze with more details the motivation for this design. The fromVNF LWT tunnel can not be tied to any route with a specific prefix because the IPv6 packets sent by VNF can use any destination address and do not have any relationship with the SIDs. Moreover, each End.AD fromVNF tunnel expects to receive traffic by its own layer-2 interface (veth0 in Figure 4), with no regards about the IPv6 destination address of the packets. This means that, in order to apply the fromVNF processing function to an incoming packet, the SRv6 NFV node has to retrieve the route that points to the right LWT tunnel using only the identifier of the interface where such as packet has been received. As a consequence of this, the fromVNF End.AD design has to deal with: i) the problem of designating an IPv6 prefix to be used for creating a route pointing to a custom processing function (LWT), and ii) the issue of steering incoming traffic received on a specific interface through such as route.

The first issue can be easily solved by using as route prefix the any address which is often indicated by ‘‘::’’. Generally, the default route is selected by the routing algorithm when the IPv6 destination address can not be managed by any other. However, this usage gives rise to a new problem. Indeed, creating a LWT on a default route has the side effect that no more than one VNF can be handled by the SRv6 node using a single table. Moreover, control traffic that transits through the SRv6 node and for which there are no given explicit routes may be wrongly handled by the LWT installed on the default route. Thankfully, this problem can be easily solved by installing every default route into a different IPv6 routing table and creating, for each of these, a rule in the IPv6 Routing Policy DB. Such rule is meant to instruct the IPv6 network system to perform route look-up on a specific table based on a specified match. The usage of an IPv6 policy route solves also the issue ii) as at this point we can use the fromVNF interface (veth0 in the above example) as match and a goto-table N as action predicate. In this way we can relate an interface to a specific default route that has attached to a LWT.

Figure 4b shows an high-level overview of proposed solution with the fromVNF LWT tunnel integrated in the IPv6 routing network subsystem. Whenever a plain IPv6 packets, sent by VNF, arrives at SRv6 NFV node, it is handled by the Linux networking stack. After passing the pre-routing stage, the kernel tries to determine the right processing of the packet. It invokes the route look-up operation, but this time the routing algorithm finds first an entry in the RPDB of the node and does not consider IPv6 destination address at first. Indeed, thanks to custom IPv6 rules (one for each fromVNF tunnel) the routing algorithm is capable to retrieve the IPv6 table tied to the incoming interface of the packet. At this point, the routing algorithm makes use of this table to find out the route that matches with the received packet. In this specific case, the routing algorithm selects and returns the only route available, the default one, that is attached to a specific End.AD tunnel. Once the plain IPv6 packet has been received by the fromVNF processing function, it leverages the identifier of the incoming interface of the packet to search for the popped IPv6 and SRv6 headers within the per-netns hashtable. If a result is found, the processing function forges a new packet and sets the headers of such as packet with those that have just been retrieved. The plain IPv6 packet is encapsulated into the newly created one and then the whole packet is delivered towards its destination. This concludes the job of the fromVNF LWT tunnel processing operation.

III-C SRNKv2

After the implementation of SRNKv1, we critically revised its design, by identifying the following main shortcomings: i) two LWT tunnels are used for the two directions inbound and fromVNF related to the same VNF; ii) a different routing table needs to be instantiated for each VNF so that the correct LWT tunnel can be associated to the fromVNF packets; iii) the use of the Linux Policy Routing framework implies to sequentially scan a list of rules to identify the VNF interface from which a packet is coming and associate a specific routing table. In particular, the first two shortcoming correspond to a waste of memory resources, while the third one to a waste of processing resources (see Sec. V).

The revised design of SRNKv2 is shown in Figure 5. The most important improvement is an extension to the Linux Policy Routing framework, in order to avoid the linear scan of the list of rules to match the VNF interface (i.e. one rule for each VNF). A new type of IPv6 rule, called SRv6 extended rule, is added and used in the fromVNF processing (f.1). The new rule (indicated as seg6local-behavior End.AD in the Figure 5) is added to the Routing Policy DB. The selector of this rule performs the lookup of the packet incoming interface (f.2) into the per-netns hashtable that includes all the VNF interfaces handled by the dynamic proxy. In this way, it is possible to understand if the packet is coming from a VNF and to retrieve the needed information (f.3.a, f.3.b). The SID associated to the VNF is used to perform the search (f.5) into the My Local SID Table, which returns the associated tunnel.

In the new design, there is actually a single LWT tunnel for the two directions. In fact, the code that is executed when a routing entry points to the tunnel is able to understand if the packet belongs to the inbound or to the fromVNF direction and behave accordingly. Thanks to the lookup in the per-netns hashtable, which allows to retrieve the SID associated with the VNF, it is not needed anymore to have a separate routing table for each VNF.

III-D Implementation of other SR proxy types

In addition to the implementation of the SR dynamic proxy, we have already implemented the static proxy behavior. This is actually a simple extension of the dynamic one (we only needed to develop command line static configuration tools and disable the “learning” of segment list from inbound packets. In principle, our SR proxy solution can be extended to implement also the masquerading proxy, but this is currently for further work.

IV Testing Environment

IV-A Testbed Description

We built our testbed according to RFC 2544 [21], which provides the guidelines for benchmarking network interconnecting devices. Figure 6 shows the considered scenario. More in depth, the testbed is composed of two nodes, which we call respectively Traffic Generator and Receiver (TGR) and System Under Test (SUT). Both TGR and SUT have two ports. In our experiment we consider the traffic crossing the SUT in one direction. As a consequence, the ports can be identified as in Figure 6: the traffic is generated by the TGR on its Sender port, enters the SUT from the IN port, exits the SUT from the OUT port and then it is received back by the TGR on its Receiver port. In this configuration, the TGR can easily perform all different kinds of statistics on the transmitted traffic including packet loss, delay, etc.

The testbed is deployed on CloudLab [22], a flexible infrastructure dedicated to scientific research on the future of Cloud Computing. In our experiment, each node is a bare metal server with Intel Xeon E5-2630 v3 processor with 16 cores (hyper-threaded) clocked at 2.40GHz, 128 GB of RAM and two Intel 82599ES 10-Gigabit network interface cards. The SUT node hosts the SR-unaware VNF, which is implemented as a Linux network namespace.

The SUT machine is running a compiled version of Linux kernel 4.14 patched with our End.AD proxy behavior implementations (namely SRNKv1 and SRNKv2). It has also a modified version of iproute2 tool [23], which allows the configuration of the dynamic proxy. Focusing on the traffic generator, we exploit TRex [24] in the TGR node. TRex is an open source traffic generator powered by DPDK [25]. We have used the described testbed scenario also in [26], which provides further details on the nodes configuration for correct execution of the experiments.

IV-B Methodology

From the TGR node, we generate SRv6 traffic using TRex. We consider IPv6 UDP packets encapsulated in outer IPv6 packets. The outer packets have an SRH with a SID list of two segments. The first SID points to the SR-unaware VNF running in the SUT, the second SID corresponds to the Receiver interface of the TGR node from the point of view of the SUT. Regarding the packet size, we have followed the indications from the Fast Data I/O Project (FD.io) Continuous System Integration and Testing (CSIT) project report [27]. In particular, the inner IPv6 packets have an UDP payload of 12 bytes, corresponding to 60 bytes at IPv6 level. The SR encapsulation adds 40 bytes for outer IPv6 header and 40 bytes for the SRH with two SIDs. The Ethernet layer introduces 18 bytes for Ethernet header and CRC, plus 20 bytes at the physical layer (preamble, inter frame gap). TRex takes in input a file with the dump of a sample packet and can reply the packet with a configurable packet rate, denoted as $PS$ rate [kpps] (Packet Sending rate). Interestingly, $PS$ can be configured with relatively high precision.

For a given configuration of the SUT, we characterize the SUT performance by generating traffic at a given $PS$ rate [kpps] for a duration $D$ [s] (usually $D=10s$ in our experiments). Let the number of packets generated by the TGR node and incoming to the SUT in an interval of duration $D$ be $P_{IN}$ (Packets INcoming in the SUT). We define the number of packets transmitted by the SUT (and received by the TGR) as $P_{OUT}$ (Packets OUTgoing from the SUT). The throughput $T$ is $P_{OUT}/D$ [kpps]. We define the Delivery Ratio (DR) as $P_{OUT}/P_{IN}=P_{OUT}/(PS*D)=T/PS$. We run a number of test repetition (e.g. 15) to evaluate the average and standard deviation, then we replicate the measurements for different sending rates. We are assuming that the performance is limited by the processing capacity of the SUT, in our experiments we make sure that a single CPU is used for the forwarding of the packets. In particular the same CPU is used for the operation of the NFV node and of the VNF. An example of result is show in Figure 7, for the baseline case that will be described in the next section. For each $PS$ rate reported in the X axis, we plot the Throughput [kpps] (right Y axis) and the Delivery Ratio (left Y axis) as averages of the 15 repetitions. The standard deviation is not shown in the figure, because it is so close to the average that cannot be distinguished. Starting from the left (low $PS$) there is a region in which the Throughput increases linearly with the $PS$ and the Delivery Ratio is 1. Ideally, the Delivery Ratio should remain 1 (i.e. no packet loss) until the SUT saturates its resources and starts dropping a fraction of the incoming packet. This is defined as the No Drop region and the highest incoming rate for which the Delivery Ratio is 1 is defined as No Drop Rate (NDR). On the other hand, in our experiments with the Linux based SUT we measured very small but not negligible packet loss ratio in a region where we have an (almost) linear increase of the Throughput. Therefore, according to [27] we define a Partial Drop Rate (PDR) by setting a threshold for the measured Loss Ratio, typically we used 0.5% as threshold corresponding to a Delivery Ratio of 0.995. The PDR@0.5% is the highest rate at which the Delivery Ratio is at least 0.995. The usefulness of the PDR is that it allows to characterize a given configuration of the SUT with a single scalar value, instead of considering the full relation between Throughput and Incoming rate shown in Figure 7. When the incoming rate overcomes the PDR, the throughput starts to decrease due to trashing phenomena. Actually, finding the PDR of a SUT configuration is in general a time consuming operation as it is needed to scan a broad range of possible traffic rates. For this reason, we designed and developed a PDR search algorithm that optimizes the time needed to find the PDR with a given precision (details in [26]).

V Performance Analysis

Figure 8 reports the performance characterization of our solution (SRNKv2) compared with a baseline reference (End) and with the pre-existing solution (SREXT). The traffic pattern used for the characterization has been described in subsection IV-B. In the baseline scenario, no Segment Routing behavior is configured in the NFV node that simply forwards the inbound and fromVNF packets according to IPv6 destination addresses. On the other hand, the VNF is SRv6 aware and performs the so-called SRv6 End behavior (for this reason the scenario is called End). In the End behavior, a node receives the packets destined for itself and advances the pointer of the segment list to the next segment, changing the IPv6 destination address. As a result, in the baseline scenario the SUT performs two regular IP forwarding operations (each one with a lookup in the routing table) and one SRv6 End behavior (which include a lookup for the incoming SID, an SRv6 header processing and a lookup for the next segment). In the SRNKv2 case, the SUT performs a routing lookup for the incoming SID, it decapsulates the packet according to the dynamic proxy behavior and forwards it to the VNF. The VNF performs a plain forwarding operation (routing lookup) on the inner packet. Moreover, the match on the incoming interface is performed in the NFV node when receiving the packet. The packets are re-encapsulated after retrieving the proper header and finally an IPv6 forwarding operation is performed. The SREXT operations are similar to the ones in the SRNKv2 scenario. The main difference is that the matching on the inbound packets is not performed in the Linux IPv6 forwarding/routing but the packets are captured in the pre-routing phase. Therefore, the regular forwarding operations are skipped, leading to an higher performance. The PDR@0.5% for SRNKv2, baseline and SREXT are reported in Table II. Our SRNKv2 implementation, which also perform decapsulation and re-encapsulation of packets, shows only a 3.7% performance degradation with respect to the baseline forwarding. The SREXT module, which skips the Linux routing operations by capturing packets in the pre-routing hook has a forwarding performance boost of 2.4% with respect to baseline forwarding.

The simplification in routing operations introduced by the SREXT module should be taken into account when making a performance comparison with the in-kernel dynamic proxy variants. The fact that an external module outperforms the in-kernel solution is not surprising in this case. The SREXT logic is tailored to specifically handle the SRv6 case one. Therefore, it can cut off all the generic code that is normally needed to determine the fate of each single packet as well as the protocol handler that should be called to process the data. Indeed, both versions of SRNK have to waste CPU cycles on possible netfilter hooks and rule lookup before being able to handle a SRv6 packet during the routing operation in Linux kernel. So, performance penalty of SRNK is not the result of a poor design. Instead, it is the side effect of the design choice of the Linux kernel networking stack which wants to be as generic as possible for dealing with a very broad range of different protocols.

Figure 9 analyzes the poor scalability of our first design (SRNKv1) based on the regular Linux Policy Routing framework. We show the PDR@0.5% versus the number of Policy Routing rules that are processed before the matching one. Consider that the number of rules corresponds to the number of VNFs to be supported and that the rules are scanned sequentially until the matching one. Therefore, the performance with $N$ rules can be read as the worst case performance when $N$ VNFs are supported, or as the average case with $2N$ VNFs (because $N$ rules needs to be checked on average with $2N$ VNFs). A linear degradation of the performance with the number of rules is clearly visible, for example when there are 80 rules the PDR@0.5% is 28.4% lower than the PDR@0.5% for SRNKv2 or for SRNKv1 with a single rule (for 160 rules the PDR@0.5% is 50.6% lower).

As for the impact of the auto learning feature of the dynamic proxy, in all the experiments shown so far we have evaluated the SRNK performance by setting the age parameter to 1 second (hence limiting the update rate to one update/s). We run an experiment by setting it to 0 (no limitation on the update rate, so that the VNF chain is updated for each incoming packet). Under this condition, we were not able to consistently achieve a delivery ratio higher than 0.99 even for low packet rates. Therefore, we evaluated the PDR@2%, which was around 392 kpps, if we compare to the PDR@0.5% (444.2 kpps) of SRNkv2 with aging 1 second, we can estimate a decrease of performance not less than 11% for updating the VNF chain at the highest possible rate (i.e. for every incoming packet).

Finally, we analyzed the cost of performing the interface lookup with the new extended SRv6 policy rule, separately from the decapsulation and encapsulation operations which are executed in the SRNKv2 scenario. There are two motivations for this analysis. First the policy rule needs to be executed for all IPv6 packets, so it introduces a performance degradation also for non-matching packets that it worth to be evaluated. The second reason is that the proposed mechanism could be reused in scenarios with multiple policy rules based on the incoming interfaces. This would require an extension to the Linux Policy Routing framework, the performance evaluation is a part of the cost-benefit analysis for this extension. For this performance analysis, we start from the baseline (End) scenario in which the packets are only forwarded in the NFV according to plain IPv6. We consider two scenarios: i) Ext. SRv6 Rule with an extended SRv6 rule with no matching interfaces, the rule will be checked for all inbound and fromVNF packets; ii) 80 Plain Rules with 80 rules which try to match an interface are added (with no matching interfaces), these 80 rules will be checked for all inbound and fromVNF packets. The PDR@0.5% is reported for the baseline and the described scenarios. The performance degradation in the packet forwarding for adding the lookup with the extended SRv6 rule is only 2.7%. On the other hand, adding 80 policy rules implies a big degradation of the forwarding performance, which becomes 28.1% smaller.

VI Related work

VI-A Service Function Chaining

Network Service Header [6] (NSH) has been proposed as solution to implement the SFC architecture defined in [3], which specifies that the SFC encapsulation transports all the necessary information to map the packets to a specific sequence of Service Functions (VNFs) that will process the packets along the path. Segment Routing Header (SRH) is inline with the SFC architecture defined in [3]. Moreover, it offers optional TLVs (Type-Length-Value) to carry on additional information (like NSH metadata). The most important difference with respect to the SRv6 solution is the need of state information in the SFF forwarders (further discussion on the NSH solution and its differences with SRv6 can be found in Appendix D).

VI-B SRv6 implementations

From kernel 4.10, Linux supports SRv6 processing including also the implementation of several local behaviors. However, at the time of writing there is lack of support of proxy behaviors. As already mentioned in Section III-A3, the SREXT module [28] provides a complementary implementation of SRv6 in Linux based nodes. A specific shortcoming of the SREXT module is that it does not support the Linux network namespaces. Therefore, it cannot coexist with the frameworks and tools that rely on network namespaces (e.g. Linux Containers, Dockers…). More in general, an external kernel module is not able to directly access most of the internal structures and usually it is necessary to re-implement them with the risk of realizing inefficient implementations and risking to introduce bugs in the code. The goal of our SRNK implementation is to be integrated in the Linux kernel mainline, so that it can evolve and be maintained together with the Linux kernel.

Another SRv6 implementation is included in the VPP (Vector Packet Processing) platform, which is the open source version of the Cisco VPP technology. The open source VPP is developed under the umbrella of the FD.io project [29]. VPP implementation of SRv6 supports most of the behavior defined in [11] including also the dynamic proxy behavior. As reported in [27], the forwarding performance of VPP is in general very high. For example a NDR (No Drop Rate) of around 5.5 Mpps is reported for a dynamic proxy setup similar to SRNK. These VPP performance evaluations are not directly comparable with our measurements, because they only focus on the SR-Proxy operations, while we have included the processing inside the VNF in our evaluation. In any case with comparable setups, VPP will outperform our implementation (as VPP also outperforms Linux kernel forwarding). Nevertheless, there is still value in enhancing the SRv6 functionality in the Linux kernel as we have proposed, because VPP is not ubiquitously deployed and there are scenarios in which it is simpler to use a kernel based feature rather than depending on an external framework (more discussion in [30]). VPP is built on top of DPDK [25]. Systems, willing to leverage such framework, need to use DPDK compatible NICs. Furthermore, a very broad number of embedded devices, used as network nodes, do not have the capability to run such frameworks (for example resource constrained devices) or run Linux distributions (for instance LEDE [31] or OpenWrt [32]) without the support of kernel bypass functionality. Last but not least, contrary to the Linux kernel, DPDK and thus VPP require reserved memory (hugepages for DPDK) and CPUs that could not be used for any other tasks.

VII Conclusions

In this paper, we have described SRNK, a dynamic proxy for Linux that supports Service Function Chaining based on IPv6 Segment Routing for legacy VNFs, a use case of great importance for service providers. The SRNK implementation is open source (available at [16]) and extends the current Linux kernel implementation of the SRv6 network programming model. SRNK is well integrated in Linux ecosystem, as it can be configured through the well known iproute2 [23] utility. We plan to submit the SRNK code to the Linux kernel mainline.

We have thoroughly analyzed several performance aspects related to our implementation of the dynamic SRv6 proxy. We went through two design and implementation cycles, referred to as SRNKv1 and SRNKv2. We identify a scalability issue in the first design SRNKv1, which has a linear degradation of the performance with the number of VNFs to be supported. The root cause of the problem is the Linux Policy Routing framework. The final design SRNKv2 solved the problem, by introducing an new type of rule in the Policy Routing framework. This rule, called extended SRv6 rule, allows using an hash table to associate an incoming packet with its interface, verifying if the packets is coming from a legacy VNF and retrieving the information needed by the dynamic SRv6 proxy to process the packet (e.g. re-encapsulating it with the outer IPv6 header and the Segment Routing Header).

As a final remark, we want to stress that even if the SRNKv2 performs a little less than the SREXT dynamic proxy implementation (< 6%), it is well integrated in the kernel code and it could be maintained with less effort with respect to the external module. SREXT takes its advantage on SRNK from the fact that it is highly designed to cut off most of the generic code that slows down the kernel networking performance by hooking itself directly in the prerouting path. Anyway, SREXT is not capable to deal with network namespaces, it can not access to some crucial internal structures and unexported functions (such as the ones offered by the kernel used for encap/decap SRv6 packets) and it does not rely on a standard configuration tool in userspace (whereby SRNK is well integrated with a patched version of iproute2). SRNK should be thought as an effort to bring inside the Linux kernel an extra feature for supporting legacy applications and leveraging the power of network programming introduced by SRv6.

Acknowledgment

This work has received funding from the Cisco University Research Program.

Appendix A SRNKv1 configuration

Hereafter we report the configuration procedure of the SRNKv1 implementation.

A-A Inbound processing

The inbound End.AD tunnel leverages only the classic IPv6 routing. With reference to the testbed network which has been depicted in Figure 6, each packet with destination fdf1::2 that comes into SFF/SR node from any interface will be handled by the LWT installed on that route. In order to create an inbound End.AD tunnel, which is able to handle packets destined to the VNF with SID fdf1::2, the command used is:

$ ip -6 route add fdf1::2/128 \  encap seg6local action End.AD chain inbound \  oif veth0 nh6 fdf1::2 age 5 \  dev veth0

It is easy to identify in its structure three different parts:

• •

ip -6 route add fdf1::2/128 is used to add the route fdf1::2 in the IPv6 routing tables;

• •

encap seg6local is used for specifying to the IPv6 networking subsystem to create a seg6local tunnel which handles packets for fdf1::2

• •

action End.AD chain inbound oif veth0 nh6 fdf1::2 age 5 is used for specifying the behavior of the seg6local tunnel.

In this sub-command, the action is defined as End.AD and the direction of the data flow is towards the VNF (chain inbound). As we explained in Section 4a, each packet that comes into this tunnel is subjected to an outer IPv6 and SRv6 headers de-capsulation. The nh6 param is used to inform the inbound tunnel about the next hop to which each packet has to be sent, and in this case is the SID of the legacy VNF.

A-B FromVNF processing

The fromVNF End.AD tunnel has to perform the inverse operation realized by the inbound counterpart. It has to restore the IPv6 and SRv6 headers using only the incoming interface of the packet as key search. At this point, we need to instruct the network subsystem on how it should send traffic to the right fromVNF tunnel. Using the ip -6 rule tool we are able to manipulate the Routing Policy DB of the nodes and issue a command that informs the system to make use of a given IPv6 routing table when traffic arrives at some specific interface. The ip rule command is the following:

$ ip -6 rule add iif veth0 table 100

After the execution of this command, every time a packet arrives at the ingress interface veth0, the routing system will try to find a route in table 100 that matches with the destination address of that packet.

Instead, to create an fromVNF End.AD tunnel on router SFF/SR with the purpose of managing packets coming from the legacy VNF at interface veth0, the following command is used:

$ ip -6 route add default \      encap seg6local action End.AD \      chain fromVNF iif veth0 \      dev veth0 table 100

Also for the case of inbound tunnel creation, the ip command can be seen split into three different parts:

• •

ip -6 route add default is used to add the default route ‘‘::’’ in the IPv6 routing table 100;

• •

encap seg6local is used for specifying to the IPv6 network subsystem to create a seg6local tunnel which handles packets destined for default address;

• •

action End.AD chain fromVNF iif veth0 is used to specify the attributes of the behavior that is intended to be created.

The action is End.AD and the direction of the data flow is specified by the chain attribute which is set to chain fromVNF. The iif keyword indicates packets coming from interface veth0. This means also that the tunnel is allowed to listen for incoming traffic only from the interface specified by iif. If it receives packets from another interface, packet is discarded automatically.

Appendix B SRNKv2 configuration

Hereafter we report the configuration procedure of the SRNKv2 implementation.

B-A Tunnel creation

In our second design (Section III-C) we have introduced the notion of bi-directional tunnel within the definition of the End.AD behavior. The creation of the desired behaviour can be achieved through the following command:

$ip -6 route add fdf1::2/128 \      encap seg6local action End.AD \      oif veth0 nh6 fdf1::2 age 5 \      dev veth0

As it is possible to appreciate, the command closely resembles the one that we have used in the Section A to setup the inbound tunnel, but this time, we are creating only one tunnel that manages the inbound/fromVNF processing for a given VNF.

With reference to the scenario depicted in Figure 6, traffic that arrives at SFF/SR node with destination the VNF’s SID (fdf1::2) on any interface except veth0 is sent to seg6local LWT associated with the End.AD behaviour for de-capsulation purposes (inbound processing). After that, packets are delivered to VNF using the output interface (for short called oif) veth0. On the other side, traffic that arrives from the interface veth0 has to be redirected to the right End.AD tunnel for the encapsulation (fromVNF processing). To accomplish that, the IPv6 rule, described in the next paragraph, is used.

B-B IP rule configuration

The idea behind RPDB configuration relies on the ability to forward packets from a specific incoming interface to the associated End.AD LWT and process them similarly to the inbound process. With this idea in mind we have designed, and implemented the changes described in Section III-C and the following ip rule command to redirect traffic from VNFs towards the right End.AD tunnels for fromVNF processing:

$ ip -6 rule add seg6local-behaviour End.AD

The command allows us to add a rule to the RPDB of the node, seg6local-behaviour End.AD is used to specify the local SRv6 behaviour that should be taken into account when the rule is picked up. If the priority of the above rule is not superseded by the priority of other rules, the extended RPDB is able to compare the packet’s incoming interface with the oif of the existing LWTs through the per-netns hashtable. This means that: if the inbound interface is equals to (exactly) one oif, the patched IP rule subsystem gets the correspondent VNF’s SID that is used as the destination address for the packet (in place of the real’s one) during the IPv6 routing lookup on “My Local SID Table”. As soon as the route is resolved, the associated End.AD tunnel is also retrieved and it is exploited for applying fromVNF operations on the incoming packet. Otherwise, if the packet’s incoming interface does not belong to any End.AD instance, the packet is treated as usual and the route lookup is performed, by default, on the destination address.

Appendix C Detailed experiments results

In this appendix we report additional experimental results, shown in Fig. 10, Fig. 11, Fig. 12

Appendix D NSH comparison with SRv6

The SRv6 solution brings a simplification of the network operations offering the possibility to implement SFC in the network without the need of having completely separated overlay and underlay operations. Indeed, with SRv6 these differences are “blurred” and we can literally program SFC in the network with several advantages. Figure 13 shows a comparison between NSH and SRH solutions in terms of: i) packet encapsulation; ii) packet forwarding; iii) state to be maintained/configured in the network nodes.

The NSH header encapsulates user traffic and then uses tunneling mechanisms to steer packets over the data center fabric. VXLAN tunnels have to be configured and maintained manually or using management protocols like ovsdb [33]. On the other side traffic encapsulated with the SRH does not leverage any tunnel and can be directly forwarded by the underlay fabric.

Most of the times in a NSH based deployment underlay, tunnels and SFC need to be controlled by different control planes. Usually it is necessary to implement these functions in two different network nodes: i) software switch or virtual router in the virtualization server; ii) hardware switch of the fabric. Having such as VXLAN tunnels introduces further overhead in the virtual nodes. One possible optimization would be to offload the tunneling part to the fabric but this would require the VTEP offloading functionality which could not be available on the underlay devices.

Network visibility would also benefit of the simplification introduced by SRv6, it would be easier to debug and troubleshoot a SRv6 based infrastructure with respect to one leveraging tunneling mechanisms and having a protocol stack of 7 or more headers.

Finally we can state that the SRv6 based approach requires less state, NSH protocol differently from SRv6 just transports a chain identifier, the so called Service Path Identifier (SPI), and the position along the chain of the packet through the Service Index (SI) but then specific state is needed in the SFC nodes in order to proper forward the packet to the next hop. On the other side, the SRH overhead results to be bigger than the overhead introduced by NSH solutions in the worst case.

We note that SRv6 does not necessarily need to be an alternative to NSH, indeed there are proposals that deal with the inter-working of NSH and SRv6 as envisaged in [34].