SDN as Active Measurement Infrastructure

TL;DR

This paper proposes integrating active network measurements directly into SDN infrastructure, enabling scalable, accurate, and diverse vantage points without dedicated measurement nodes.

Contribution

It introduces SAAMI, a framework that embeds active measurements into SDN, allowing measurements from any SDN-enabled location and simplifying deployment.

Findings

SAAMI is accurate in measurement results.

SAAMI is scalable to large networks.

SAAMI is extensible for different measurement protocols.

Abstract

Active measurements are integral to the operation and management of networks, and invaluable to supporting empirical network research. Unfortunately, it is often cost-prohibitive and logistically difficult to widely deploy measurement nodes, especially in the core. In this work, we consider the feasibility of tightly integrating measurement within the infrastructure by using Software Defined Networks (SDNs). We introduce "SDN as Active Measurement Infrastructure" (SAAMI) to enable measurements to originate from any location where SDN is deployed, removing the need for dedicated measurement nodes and increasing vantage point diversity. We implement ping and traceroute using SAAMI, as well as a proof-of-concept custom measurement protocol to demonstrate the power and ease of SAAMI's open framework. Via a large-scale measurement campaign using SDN switches as vantage points, we show that SAAMI is accurate, scalable, and extensible.

Full text

1 Introduction

Software Defined Networking (SDN) has emerged as a powerful architectural paradigm, enabling innovations in network virtualization, provisioning, verification, and security (e.g., [8, 9, 13]). Within the context of measurement, SDNs are commonly instrumented to monitor network utilization and can quickly modify their forwarding behavior in response to dynamic workloads [30]. In contrast to such passive measurements (e.g., packet or flow-level statistics, heavy-hitters, or anomaly detectors), this work considers the feasibility of performing active network measurement using SDNs.

Intuitively, SDNs provide the basic building blocks for facilitating programmable active measurement vantage points. Via the standardized OpenFlow protocol [20], SDN controllers can generate any arbitrary packet (e.g., measurement probes) and instruct a switch to emit the packet out a specified interface. Similarly, controllers can instantiate fine-grained flow rules that match measurement probe responses. Controllers may then perform arbitrary computation over probe responses as they arrive encapsulated in an OpenFlow message from the switch. In this work, we introduce and advocate for an architecture using these primitives which we term “SDN as Active Measurement Infrastructure” (SAAMI).

While active measurements today are performed using end-hosts, we believe SAAMI provides compelling advantages. First, tighter integration of active measurements within the network allows operators, administrators, and researchers to place measurements anywhere an SDN switch exists – without the traditional cost of configuring, certifying, securing, installing, powering, managing and maintaining a dedicated measurement host. Often, installing a measurement host within the core or edge of the network presents an insurmountable administrative or policy hurdle, or is not physically possible (due to space, availability and expense of consuming an interface on the router, etc). By leveraging existing production equipment, SAAMI lowers deployment and vantage point diversity barriers.

Second, placing active measurements within SDN more closely couples the ability to programatically enable action to be taken in response to measurement results. In a similar fashion to using passive measurements to drive the behavior of the SDN data plane, active measurements can provide actionable information for network reconfiguration and adaptability.

Third, SAAMI follows the philosophy of inexpensive commodity hardware, centralized control, and compositional network architecture. SAAMI uses the standardized OpenFlow API to drive commodity switches, thereby removing the traditional need to develop, configure, and deploy custom APIs for requesting measurement tasks and receiving measurement results. As we will show, using OpenFlow prevents over-specialization of the measurement platform, making it easy to extend to new and unanticipated measurement tasks.

Toward the goal of converging on a standards-based strategy for active measurement platforms for operators and researchers, we investigate the feasibility and limitations of SAAMI. Our primary contributions include:

The SAAMI architectural vision to converge active measurement facilities into commodity hardware using standardized protocols. 2. 2.

A large-scale Internet measurement study using SAAMI to source active probes from commodity SDN switches, and a corresponding analysis of the resulting fidelity of round-trip latency measurements. 3. 3.

Implementation of a new measurement protocol using the SAAMI primitives as a demonstration of the ability to easily and rapidly innovate and deploy measurements within the SAAMI framework. 4. 4.

A fundamentally different approach to Internet measurement, in which core nodes (in addition to edge systems) participate as vantage points for active measurements, potentially in reaction to events in the traffic streams they observe.

The remainder of this paper is organized as follows. Section 2 describes SAAMI, while Section 3 details results and accuracy from using SAAMI in real-world large-scale Internet measurement experiments. We discuss related work in Section 4, and conclude with a discussion of deployment scenarios and suggestions for future work in Section 5.

2 Implementation

Active measurement is widely used for network management and debugging. Canonical examples include the ping and traceroute utilities that provide reachability, round-trip time, and forward path information. Not only are such tools used for troubleshooting, they are integral to the operation of large providers, content distribution networks, and at-scale services.

Myriad other active measurement techniques and tools have been developed, for instance host and service fingerprinting [19], capacity and bandwidth estimation [7], censorship detection [5], network neutrality [6], residential broadband performance [3], and congestion localization [16], to name only a few.

It is well-known that the accuracy and generality of inferences from these active measurements can depend strongly on where in the network the measurements are performed. For instance, censorship may only affect nodes behind a certain middlebox, congestion may affect only a single network, or a BGP hijack event may affect reachability for only a subset of vantage points. To this end, the research community has developed several active measurement platforms consisting of varying (but relatively small) numbers of nodes distributed (in an ad-hoc fashion) across the Internet, e.g., Archipelago [12], Bismark [29], Dasu [26], and Atlas [23] to name a few. Unfortunately, these platforms are frequently: i) designed for a specialized task; ii) under-powered (either in terms of compute, memory, or network abilities); and iii) lacking in network or geographic diversity.

Rather than the current fractured environment of incompatible network measurement platforms, abilities, APIs, and output formats, we consider the feasibility of performing active measurement using existing SDN standards and capabilities. Our vision that by using SDN, the measurement and operational community can utilize a standard and open API to avoid over-specialization, lower deployment barriers, and facilitate vantage points otherwise not possible.

While significant prior work utilizes the passive network measurement capabilities afforded by SDNs, e.g., [28, 30, 21, 14, 33] and novel methods to gather measurements from SDNs, e.g., [32], our work seeks to understand the feasibility of SDN for active measurement – i.e., where the SDN switch generates specialized measurement probes and gathers their responses.

Figure 1 illustrates the high-level architectural view of SAAMI within an SDN. We envision SAAMI co-existing with other applications on an SDN controller that is responsible for one or more SDN switches within a provider or enterprise network.

Our implementation is based on the popular open-source Ryu [25] controller. In response to measurement tasks (§2.3) SAAMI instructs the SDN controller to send various OpenFlow [20] messages to the SDN switch(es) in order to induce active IP measurement probes. In particular, we utilize the OFPacketOut message to instruct a switch to send a particular packet. Further, SAAMI installs flow table rules such that it can receive and process measurement replies; we rely on OFPacketIn messages from the switch that encapsulate data-plane probe packets matching particular criteria. In this fashion, SAAMI acts as the abstraction layer between measurement tasks and the SDN.

2.1 Calibration

We note that many measurement tasks, including ping and traceroute, require accurate timing information, e.g., for round-trip time (RTT) latency estimation. SAAMI must thus address two timing challenges related to using commodity switches and the OpenFlow protocol: approximating the time at which measurement packets are sent and received. With respect to probe transmission, both the delay in sending the OpenFlow instruction (the “OFPacketOut”) from the controller to the switch, and the switch’s processing delay in executing that instruction, contribute potentially variable latency. Further, while OpenFlow provides a mechanism to forward probe responses from the switch to the controller (the “OFPacketIn”), there is no standardized way to obtain the time when the packet was received at the switch. We must therefore estimate these values via an assessment of the RTT between the controller and the switch, and via empirical analysis of commodity SDN switch behavior. While it is possible to design specialized hardware and implement API changes to address these sources of inaccuracy, our goal in this work to determine the feasibility of current SDN hardware and software implementations to support SAAMI.

2.1.1 Switch Processing Delays

Let $R(X)$ be the time at which some packet $X$ is received at a switch port and $T(X)$ be the time at which some packet $X$ is emitted from a switch port.

• •

Assume a switch receives a probe within a OFPacketOut at $R(pktout)$. To determine the switch’s delay is emitting the probe, we compute: $T(probe)-R(pktout)$.

• •

Further, we wish to determine whether any output reordering occurs, i.e., $\exists i\text{ s.t.\ }T(probe_{i+1})<T(probe_{i})$ and $R(pktout_{i})<R(pktout_{i+1})$.

• •

Assume a switch receives a response packet that matches a flow rule. The switch’s delay in generating the OFPacketIn message is: $T(pktin)-R(response)$.

To isolate sources of delay and calibrate our SAAMI measurements, we create an isolated testbed using a Linux machine with two physical Ethernet interfaces as shown in Figure 2. One Ethernet directly connects to the Out-of-Band Management (OOBM) port on a commodity commercial SDN switch, while the other Ethernet is connected to one of the switch’s data-plane interfaces. In our experiments, we use a commodity HP2920 commercial switch running OpenFlow 1.3. We then perform a packet capture from the Linux machine to time OpenFlow packets in relation to dataplane packets (either generated or received). Because the packet capture is performed on two interfaces of the same physical machine, time is synchronized. Since the host and the SDN switch are directly connected via a $<1$ meter cable, propagation delay is a negligible component of the measured delay.

We send 100 OFPacketOut messages to the switch, and measure the switch’s delay in emitting the corresponding packet. Figure 3 displays the cumulative fraction of delays $T(probe_{i})-R(pktout_{i})$ over the 100 OFPacketOut requests. We see that 97% have a delay between 1.5 and 2.0ms. Two OFPacketOut messages require $\approx 23$ms, while the switch took approximately 50ms before emitting one of the packets. Overall, the delay is both small and tightly bounded. Further, we observe no packet reordering.

Similarly, we evaluate the switch’s delay in emitting a OFPacketIn message in response to receiving a packet that matches a flow rule, $T(pktin_{i})-R(response_{i})$. We see a qualitatively similarly shaped distribution in Figure 3, however 95% of the OFPacketIn messages are generated in 1.0ms or less.

2.1.2 Bundled Messages

We observe instances of multiple OFPacketOut messages bundled into single TCP segments by the SAAMI controller. Such effects are due to operating systems and their corresponding TCP stack implementations. Quantifying the effect that bundling has on time-sensitive measurements is therefore important for e.g., RTT estimation. Thus, we must assess the variation in time for packets to be emitted by an SDN switch, whether arriving at the switch as OFPacketOut messages in separate TCP segments, or bundled with other OFPacketOut messages in a single TCP segment.

In order to measure this effect, we use OFPacketOut messages to instruct the switch to emit ICMP packets. We calculate the difference in time between an ICMP Echo Request leaving the SDN switch and its corresponding OFPacketOut message arriving at the switch, accounting for OFPacketOut messages in individual segments and multiple OFPacketOut messages contained in a single segment separately. We sent 15,000 OFPacketOut probe requests with 5 probes apiece to a host running Open vSwitch [22] acting as the SDN switch. Of the 75,000 probes generated, only 61 probes are sent as OFPacketOut messages contained in a TCP segment with other OFPacketOut messages. The time difference between ICMP Echo Requests exiting the switch and the time the OFPacketOut message entered the switch is characterized in Figure 4. On average, the overall switch delay is approximately 10% higher for probes sent in multiple-OFPacketOut-containing segments – a 206 $\mu s$ delay was incurred by multipart messages, whereas individually sent OFPacketOut pings had only a 189 $\mu s$ mean delay.

2.1.3 Controller to Switch RTT Estimation

The SAAMI controller measures the RTT between controller and associated SDN switch by issuing and timestamping an OFEchoRequest message, a built-in OpenFlow message type that is used by default as a “heartbeat” between the controller and switch. When the corresponding OFEchoReply returns from the switch, the controller timestamps this reply, allowing for the calculation of the RTT between the controller and switch $RTT_{C-S}$. For our OpenFlow implementations of ping (Section 2.4.1) and traceroute (Section 2.4.2) we estimate $RTT_{C-S}$ for each target separately before generating probes. In order to account for isolated, drastic changes in controller-to-switch latencies, we keep an exponential moving weighted average of $RTT_{C-S}$ times for use in true RTT estimation calculations.

Note that performing the controller to switch latency estimation within the OpenFlow protocol, as opposed to, e.g., using a simple ICMP echo, provides a more reliable approximation of the delay (and processing) incurred by OpenFlow messages.

2.2 Configuration

For experiments in which probes must eventually return to the switch that emitted them (e.g., ping and traceroute), these probes must use a source address that is routed to or through the switch that generated the probe111We discuss more complicated scenarios where a different switch in the network receives probe responses in §5.. Thus, SAAMI must know what source IP address to use when generating OFPacketOut messages and installing flow rules. While this requires specific addressing information to be known, such knowledge is integral to SDN controllers.

In our testing, we chose an unused address on a subnet that is routed to a network on which the SDN switch is connected. Because it was not possible to assign an IP address to the commodity SDN switch port, SAAMI deliberately generates gratuitous ARPs. This allows the router to which our SDN switch is connected to pre-populate its ARP cache and prevent any additional delay. We expect that generating these ARPs will not be necessary in other deployments where SDN switches act as layer-3 forwarders.

Thus, for correct operation, SAAMI must at a minimum be configured with: i) the IP and MAC addresses to use when sourcing measurement probes; and ii) the IP and MAC address of the next hop such that the probe is properly forwarded. As is typical in SDN installations, the switches are configured with the IP address of the controller, and SAAMI listens for incoming OpenFlow connections.

2.3 Measurement API

Following our general philosophy of leveraging existing protocols, SAAMI utilizes HTTP and JSON as its measurement API with which the consumers of measurement tasks interface. In this way, SAAMI acts as the interface between high-level measurement tasks, and the marshaling of OpenFlow messages within the SDN plane.

The SAAMI controller therefore runs an HTTP server that listens for incoming measurement instructions. We leverage HTTP as a standard RPC-like mechanism with the ability to easily support encryption, integrity, and authentication.

As a concrete example, we detail here the REST API calls necessary for our ping implementation described in Section 2.4.1. Many common measurement tasks may leverage or require the emission of ICMP Echo Request packets, such as RTT estimation or to elicit responses containing IP Identifier values. To enable these experiments, the SAAMI controller has a defined “probe URL”, a URL that when requested with the required parameters included in a JSON array via an HTTP PUT, will emit Echo Requests destined for a particular target. For instance, the controller uses URL http://example.com:8080/ping/ as the Echo Request-emitting resource locator (by default, Ryu listens for API calls on port 8080). Our ping implementation requires a JSON array be sent with the HTTP PUT request containing three keywords extending the JSON schema: i) tgt, the IP address of the target; ii) num, the number of Echo Requests to be emitted; and iii) payload, an optional payload for inclusion in the Echo Requests SAAMI emits from the SDN switch. Upon receipt of the PUT request, the SAAMI controller parses the JSON array and passes the fields to the method responsible for creating the ICMP Echo Requests, encapsulating them in IP datagrams and Ethernet frames, and delivering OFPacketOut messages to the SDN switch. Switch for packet emission and output port selection are specified in OFPacketOut message data fields, ensuring the correct switch emits the packet from the proper output interface. The use of the REST API creates an extensible measurement framework – in our ping example, it is trivial to add an additional optional source IP address parameter for Echo Request generation or to define a default TTL for the emitted datagrams – that is easily automated using scripts (e.g., with curl or wget). Furthermore, the API enables the experimenter to operate independently from the measurement platform, without needing to log into the controller to start or stop measurements.

Experiment output is retrieved via REST API calls; we obtain state maintained by the SAAMI controller again through the use of HTTP GET requests to predefined URLs. For example, an HTTP GET request to http://example.com:8080/ping/dump returns a JSON array containing the packet emission and arrival times associated with each ICMP ID and Sequence Numbers sent in our ping implementation, using the ICMP ID as the JSON schema keyword. ICMP Echo Requests that receive no corresponding Echo Reply simply contain a null entry in the response timestamp position in its corresponding field, allowing unresponsive destination hosts to be treated differently than responsive destination addresses. This raw data can then be manipulated by the experimenter according to their own needs. A practical application using this data (estimating RTTs) is demonstrated in Section 2.4.1.

Figure 5 is a high-level view of SAAMI’s ping implementation.

2.4 Common Measurement Tasks

2.4.1 Ping

SAAMI implements the ping network utility through an HTTP PUT message to the SAAMI controller from a SAAMI client. This PUT message includes a JSON array with target IP address, count and payload fields. In our implementation, the SAAMI client is a trivial 12 line Python program, illustrating the ease with which it is possible to develop within the SAAMI framework.

Upon receipt of the ping request via the PUT, the controller first sends an ARP reply to the router containing the MAC and IP addresses of the controller’s interface to the switch as described in §2.2. This gratuitous ARP reply prevents any ARP cache expiration; without this gratuitous ARP reply, we risk SAAMI needing to reply to ARP requests and thereby negatively affecting time-sensitive RTT estimation.

Next, SAAMI initiates RTT estimation to the switch as outlined in Section 2.1.3. The controller then creates an ICMP Echo Request destined for the target IP address, encapsulates it in an Ethernet frame addressed to the next hop’s MAC address, and emits an OFPacketOut message to an SDN switch. Concurrently, the controller creates a map between the ICMP Identifier field value used and target IP address to maintain state of in-flight probes. SAAMI initially sets the ICMP Sequence Number field to 0 for the first ICMP Echo Request. The payload of our Echo Request is empty, but can be specified by the requester in the JSON array sent with the HTTP PUT.

SAAMI timestamps the packet and delivers a OFPacketOut message to the SDN switch, which emits the ICMP Echo Request out of the switch port specified as an argument to the OFPActionOutput() method. SAAMI then increments the ICMP Sequence Number field value and repeats until the requested number of ICMP Echo Requests have been delivered to the switch via OFPacketOut messages. For each subsequent ping target, SAAMI increments the ICMP Identifier field value, establishing a linkage between ICMP Identifier values and ping destination IP addresses.

Concurrently, SAAMI is delivered ICMP Echo Replies for the probes it has transmitted by the switch via OFPacketIn messages, due to the installation of a flow rule during SAAMI initialization directing the switch to forward these to the controller. When a probe reply is received SAAMI notes the time the ICMP Echo Replies were received by the controller and stores this value. For a given target $Tgt$, we can therefore calculate $RTT_{C-Tgt}$, the total time elapsed between OFPacketOut transmission from SAAMI to the switch and OFPacketIn messages received by SAAMI from the switch for each ICMP Sequence Number. $RTT_{C-Tgt}-RTT_{C-S}$ then approximates the true RTT between the SDN switch and probe target. Targets that do not respond to SAAMI probes have no ICMP Echo Reply timestamp and leave us unable to calculate the RTT; we therefore discard any targets with no probe replies. A REST API call retrieves the timestamp values stored by SAAMI for analysis by the SAAMI client.

Because the ICMP ID field is 16 bits, this implementation allows for approximately 65 thousand ping targets before SAAMI’s state table is full. We work around this limitation by pulling the current state from SAAMI via a REST API call; specifically a GET request to a specific URL returns a JSON array of SAAMI’s state table to the SAAMI experimenter. Another REST API call clears the state table and measurements can be resumed, thus allowing for a potentially unlimited number of ping targets. An alternative implementation might leverage the payload field of the ICMP Echos to maintain the state of ping targets; we abstain from this approach in order to maintain consistency of sent packet sizes for all probes when conducting large-scale measurements.

2.4.2 Traceroute

Another common measurement task we implement in SAAMI is traceroute, initiated by a specific REST API call222While there are several traceroute variants, we provide basic implementation here to demonstrate the ease with which measurements can be created.. The keywords contained in the JSON array of the HTTP PUT message are the target, and number of probes per TTL. SAAMI’s traceroute implementation operates by first determining $RTT_{C-S}$ as described in Section 2.1. SAAMI then creates ICMP Echo Request messages with TTL values beginning at 1, delivering these to the switch via OFPacketOut messages. Initially, the ICMP Identifier value is set to 0, as in our ping implementation. SAAMI maintains TTL and target information for each probe by ICMP identification and sequence numbers. When intermediate routers respond with ICMP Time Exceeded messages, SAAMI correlates the response with its corresponding probe by parsing the ICMP quote, and thereby determines the IP address and RTT for each hop along the forward path. SAAMI increments the IP TTL value after each user-specified number of probes at each TTL value, until one of two conditions occurs: i) the traceroute target is reached and responds with an ICMP Echo Reply, or ii) the TTL value reaches 30 without having reached the traceroute target. The SAAMI controller then increments the ICMP Identifier value used for correlating messages bound for the same traceroute target together, as in our ping implementation in Section 2.4.1. Our traceroute implementation is asynchronous; each successive ICMP packet is sent to the switch in a OFPacketOut message to be emitted as soon as it is created by the controller. SAAMI is capable of handling these multiple packets in flight simultaneously due to the state table it maintains of ICMP ID/Sequence Number values. Like our ping implementation, traceroute probes for which intermediate routers or destination addresses are unresponsive simply contain a null value as the return timestamp. A REST API call returns the SAAMI traceroute state to the SAAMI experimenter, who can then reproduce the path from source to destination from the IP addresses.

2.5 Custom Measurements

In addition to performing existing active measurement tasks from within SDN (e.g., ping and traceroute as previously described), SAAMI facilitates the creation and execution of new and novel measurements in the spirit of network innovation. As a proof-of-concept, we partially implement the “router ID” primitive from our nascent work in developing and using within protocol measurements [2].

Today’s method for active router discovery has remained essentially unchanged for almost 30 years: traceroute induces routers along the forward path to a destination to return ICMP messages with one of the router’s interface IP addresses as the source. Unfortunately, IP addresses are a poor proxy for a router identifier. Our reliance on traceroute leads to the cumbersome and error-prone processes of: i) alias resolution, to determine the set of IP addresses belonging to the same physical router; and ii) router ownership inference. In the case of alias resolution, the existing techniques require intensive active probing, only work for a subset of addresses, and can both produce false aliases and miss true aliases [15]. Similarly, the naïve method of using the autonomous system (AS) that originated the address space to which a router’s interface belongs is often not indicative of the AS that owns or maintains the router, due to delegated and off-path addresses [18, 17].

Our intent in implementing the router ID is not to perform exhaustive measurements, but rather to provide a concrete example of the types of measurement primitives that could be created with SAAMI, and their immediate benefit to network management and diagnosis.

For debugging and management purposes, we imagine that a provider wishes to extend the functionality of their core networking devices such that they respond to an identification query with a unique router identifier and the AS number to which the device belongs. Using SAAMI, we instantiate a rule in the switch to encapsulate and forward to the controller any ICMP packets destined to the switch with type 200 and code 0333corresponding to a currently unused ICMP type. The controller parses this router ID query message and creates a OFPacketOut response with the device’s AS and identifier.

This trivial example highlights several important characteristics of SAAMI. First, while any database, e.g., the DNS, could conceivably provide an identical functionality, doing so requires keeping different portions of the namespace consistently updated – a significant practical difficulty. Instead, SAAMI provides a much closer coupling where the control plane of the network device knows its AS and router identifier. Second, router ID demonstrates the ease with which a new protocol or protocol extension can be implemented to provide substantial measurement benefit. In this case, router ID effectively solves both the aliasing and ownership problems with an explicit mechanism, rather than the brittle and error-prone inferences the measurement community is currently forced to employ.

2.6 Deployment Scenarios

We envision core network operators facilitating SAAMI experiments by allowing researchers access to the SAAMI application running on their production controllers. This immediately introduces several implementation challenges. First, how might service providers arbitrate access to their SAAMI instance? Ryu supports a robust PKI-based authentication system, allowing network operators to permit SAAMI access to authorized users and specific switches via public key authentication. Second, allowing SAAMI users the ability to inject an arbitrary number of packets as quickly as the controller can generate them into their network is likely an unappealing prospect for service providers. But because SAAMI is a controller application, policies limiting maximum packet rates, connection rates, and bitrates can be specified and tailored to individual operators’ risk assessments. Third, the potential exists for measurement experiments to install flow rules on SDN switches that conflict with production dataplane forwarding rules. This potential conflict can be overcome by assigning a lower priority to SAAMI user-generated flow rules, which would then be ignored if a conflicting flow rule installed by the network operator exists, thereby preventing unintended and potentially harmful consequences; OpenFlow already supports this type of flow rule prioritization. Finally, SAAMI should support multiple experimenters concurrently; because the controller is simply a production server, partitioning of multiple users is a benefit inherited from the deployment environment itself. Conflicting flow rules generated by individual experimenters can be handled by assigning higher priority to certain experiments, in a first-come, first-served manner, or according to operator-specific instructions. Finally, to mitigate risk assumed by network administrators running SAAMI instances, SAAMI can be configured to only allow certain types of packets to be generated via its API. For instance, the ability to send ICMP-based ping or traceroute probes at a low datarate, or packets to TCP port 80 might be enabled, while the emission of more unusual packets (like those discussed in Section 2.5) could be restricted only to certain users or disabled entirely.

3 Results

3.1 Ping

In order to measure the accuracy of SAAMI’s ping implementation, we capture traffic entering and leaving the SDN switch through a hub connected to the machine running SAAMI with tcpdump. In this manner, we obtain ground truth for the times our SAAMI-generated ICMP Echo Requests and Replies exit and enter the switch. We account for various hardware and software implementations, using both an HP 2920 SDN switch and a Linux machine running Open vSwitch, as well as remote versus local controller scenarios. Both the HP switch and machine running Open vSwitch are located in Monterey, California, while the remote controller is located in Boston, Massachusetts. Figure 6 shows the CDF of absolute error (between SAAMI-recorded RTT and actual RTT) using an HP 2920 series switch, as well as using the machine running SAAMI as the switch itself with Open vSwitch (labeled as “Local controller HP2920” and “Local Controller OVS”, respectively). Additionally, the software and hardware switches were controlled from a SAAMI controller operating on the machine in Boston, shown in the “Remote Controller” curves. All trials probed the same 15,000 IP addresses and received $\sim$11,000 replies each. Our results show that the SAAMI instance controlling the remote Open vSwitch SDN switch has the least amount of absolute error, with nearly 99% of errors at 10 $ms$ or less. The lower bound is the remote SAAMI instance controlling the HP switch, with only $\sim$70% of errors at the 10 $ms$ of less mark. Figure 7 is a CDF of the absolute error as a fraction of the total RTT. In our ping implementation, the Open vSwitch implementations consistently produce a lower $\frac{error}{RTT}$ value, with the locally controlled Open vSwitch trial obtaining an error of $<10\%$ of the total RTT for more than 95% of all probes. Both HP switch implementations produce the most error in RTT measurement, with the remote controller architecture achieving the same error percentage for approximately $20\%$ fewer RTTs. One method that can be used to reduce the absolute and relative RTT errors is simply to send more Echo Requests to the targets. In Figure 8, we show the SAAMI error relative to the actual RTT in two trials, one in which five Echo Requests were sent to the destination, and the other in which only one Echo Request was emitted. For the trial in which five Echo Requests were sent to the target, we record the mean RTT for all probes to each target. This strategy dramatically reduces the overall RTT error.

4 Related Work

Zeng *et al.*first posited the notion of SDN controllers instructing switches to send test packets in the context of their Automated Test Packet Generation system in [34]. We extend this idea to the variety of active measurements in use today, and implement such a system in real SDN hardware and software.

Our work relies on RTT measurements of probes to various network targets generated and received by a controller from SDN switches; quantifying latencies in SDNs is therefore fundamental to our work. Rostos *et al.*develop the OFLOPS platform to evaluate OpenFlow switch implementations in various use-cases [24]. OFLOPS examines processing delays in OpenFlow switches when performing specific actions such as forwarding and packet modification. Our work also examines processing delays in both hardware and software OpenFlow implementations of modern devices. In [10], He *et al.*measure inbound (switch to controller) and outbound (controller to switch) latencies induced by OFPacketIn events and flow-modification rule insertions, deletions, and modifications, respectively. Our timing analysis is impacted by delays caused by OFPacketIn events as well, which we similarly analyze on our devices using a passive tap. SLAM [31], a tool used to monitor and estimate latencies in data centers, sends packets between SDN switches to estimate the latency along the path between them. SLAM generates notifications to a controller via OFPacketIn notification messages sent by the first and last switches along a path when a probe is received. As in our work, in [31] the authors account for controller to switch latency by continuously monitoring it via OFPEchoRequest messages.

The notion of using network devices to perform active measurements was first standardized in [27] and [11], protocols for performing one and two-way active measurements of delay and loss. Our work is broader, generalizes these protocols, and provides a means for such schemes to be implemented in network switches without explicit vendor support. Most closely related to our own work is SDN traceroute [1], a technique to reveal the sequence of switches and ports that actual data-plane packets traverse in an SDN network. SDN traceroute relies on injecting measurement probes via OFPacketOut messages, and installing rules to match and retrieve tagged probe traffic via OFPacketIn messages. While SDN traceroute and SAAMI rely on the same primitives, SAAMI is designed to provide a platform for existing measurements, e.g., IP-level ping and traceroute, using SDNs.

5 Conclusions and Future Work

We introduce SAAMI, an active measurement application for SDNs. SAAMI enables active measurement experiments to be conducted wherever SDNs are deployed, bringing measurement capability to the core of the network as opposed to traditional edge-based platforms. We implement two common active network measurement utilities – ping, to determine RTTs from an SDN switch to network target, and an ICMP-based traceroute. To validate our RTT measurements, we conduct multiple experiments to identify sources of error and quantify their effects when using SAAMI as a measurement platform. We find that these effects vary significantly according to the location of the SAAMI controller and between hardware and software SDN switches, but can be mitigated by sending several packets to each target, and using the mean RTT from all probes. Finally, we demonstrate the ability to conduct custom measurements by designing a capability to perform active router discovery by querying routers for an imagined “router ID” value containing that device’s ASN and an identifying string. By sending an ICMP packet with an unused ICMP type and code value to a router supporting this capability, SAAMI obtains topological data about a device directly, rather than via current, error-prone traceroute methods.

5.1 Future Work

With SAAMI instances deployed at the network core, we aim to understand forwarding behavior not accessible to measurement experiments performed at the network edge. For example, do traceroutes initiated from the network edge follow the same terminal path to a destination as those initiated from a tier 1 AS? That is, do operator policies pertaining to source IP address affect the path selection in ways that are not discernible to edge-based vantage points? Additionally, as we note in Section 2.5, SDN controllers are capable of arbitrary packet creation. These packets need not adhere to the constraints placed upon ordinary hosts; for example, a SAAMI controller may generate IP datagrams with any source IP address. If destined for another SAAMI switch, these spoofed source address packets can reveal source address validation policies implemented by network operators unmeasurable by platforms at the edge of the Internet. Finally, SAAMI’s unique contribution of offering measurement vantage points from the core of the network affords the opportunity to conduct certain estimates, like bandwidth usage, closer to the quantity being measured. We envision using SAAMI, or a similar system, to “tag” portions of the traffic flowing through vantage points to better understand traffic characteristics as it is routed through network operator-administered infrastructure.

Finally, while we focus on OpenFlow-based SDNs in this work due to the ubiquity of their real-world deployment, we note that the SDN space is rapidly evolving. For example, recent work on programmable network processors [4] promises a much richer set of abstractions and capabilities that SAAMI can utilize if realized.

Acknowledgments

We thank Mark Allman, Steve Bauer, and Ethan Katz-Bassett for valuable early feedback. Views and conclusions are those of the authors and should not be interpreted as representing the official policies or position of the U.S. government.