A QUIC Implementation for ns-3

TL;DR

This paper presents a native ns-3 implementation of the QUIC protocol, enabling researchers to simulate and analyze its performance and design choices in a controlled environment.

Contribution

It provides the first detailed implementation of QUIC in ns-3, including features, assumptions, and example scenarios, facilitating further research.

Findings

Implementation enables performance testing of QUIC in ns-3

Highlights differences between implementation and IETF drafts

Provides example scenarios for protocol evaluation

Abstract

Quick UDP Internet Connections (QUIC) is a recently proposed transport protocol, currently being standardized by the Internet Engineering Task Force (IETF). It aims at overcoming some of the shortcomings of TCP, while maintaining the logic related to flow and congestion control, retransmissions and acknowledgments. It supports multiplexing of multiple application layer streams in the same connection, a more refined selective acknowledgment scheme, and low-latency connection establishment. It also integrates cryptographic functionalities in the protocol design. Moreover, QUIC is deployed at the application layer, and encapsulates its packets in UDP datagrams. Given the widespread interest in the new QUIC features, we believe that it is important to provide to the networking community an implementation in a controllable and isolated environment, i.e., a network simulator such as ns-3, in which it is possible to test QUIC's performance and understand design choices and possible limitations. Therefore, in this paper we present a native implementation of QUIC for ns-3, describing the features we implemented, the main assumptions and differences with respect to the QUIC Internet Drafts, and a set of examples.

Figures

Full text

QUIC, ns-3, transport protocols

††journalyear: 2019††copyright: none††conference: 2018 Workshop on ns-3; June 19–20, 2019; Firenze, Italy††booktitle: WNS3 ’19: 2019 Workshop on ns-3, June 19–20, 2019, Firenze, Italy††price: xxx††doi: xxxxxxxxxxxxxxxxxxxxxxx††isbn: xxxxxxxxxxxxxxxxxxxxxxxxxx††ccs: Networks Transport protocols††ccs: Networks Network simulations††ccs: Networks Network protocol design

1. Introduction

This paper has been submitted to WNS3 2019. Copyright may be transferred without notice.

The infrastructures, technologies and protocols that support communications over the Internet have significantly evolved over the years to support the needs of the connected society and the increase in mobile, desktop and machine-generated traffic (Cisco, 2017). In particular, the research community has recently shown a renewed interest in topics related to the transport layer, with novel transport protocols and extensions to TCP such as, e.g., Stream Control Transmission Protocol (SCTP), Quick UDP Internet Connections (QUIC), and Multi-path TCP (MPTCP) (Polese et al., 2018). Furthermore, novel congestion control algorithms have also been proposed to address novel communication challenges, such as those introduced by new cellular Radio Access Networks (RANs) (e.g., 3GPP LTE, NR) (Zhang et al., 2019).

One of the most important novelties is QUIC, a transport protocol implemented at the application layer, originally proposed by Google (Langley et al., 2017) and currently considered for standardization by the Internet Engineering Task Force (IETF) (Iyengar and Thomson, 2018). QUIC addresses some of the issues that currently affect transport protocols, and TCP in particular. First of all, it is designed to be deployed on top of UDP to avoid any issue with middleboxes in the network that do not forward packets from protocols other than TCP and/or UDP (Papastergiou et al., 2017). Moreover, unlike TCP, QUIC is not integrated in the kernel of the Operating Systems (OSs), but resides in user space, so that changes in the protocol implementation will not require OS updates. Finally, QUIC provides a number of improvements with respect to TCP, such as the integration of the security of the connection, and the reduction of the initial handshake delay.

Despite being still under discussion by the IETF, QUIC has been deployed in a number of commercial products (e.g., the Google Chrome browser). According to the estimates provided in (Langley et al., 2017), 7% of the overall Internet traffic and 30% of Google’s egress traffic is currently generated by QUIC connections. The performance of different versions of QUIC has also been studied in a number of recent papers (Rüth et al., 2018; Kakhki et al., 2017; Carlucci et al., 2015), with experiments in real networks based on different open source implementations of QUIC. Nonetheless, given the important role that this protocol will play in the evolution of the networking stack in the near future, we believe that it is fundamental for researchers and network engineers to have the possibility of studying its design and performance in a controlled environment, i.e., in a network simulator. Therefore, in this paper we present a native implementation of QUIC for ns-3, which is compliant with version 13 of the IETF QUIC draft, and is integrated with the existing applications and TCP/IP stack of ns-3.111The code is publicly available at https://github.com/signetlabdei/quic-ns-3 Our implementation is based on the design of the ns-3 TCP implementation (Casoni and Patriciello, 2016; Patriciello, 2017), and heavily extends it to account for the novelties of QUIC.

Notice that, while there exist multiple real implementations of QUIC (Seemann and Clemente, 2018; Tikhonov and Prodoehl, 2017; Shade et al., 2016; Lee et al., 2016), their integration with ns-3 has several issues.222An integration of the QUIC implementation in (Shade et al., 2016) can be found in (Yan Soares Couto, 2018). For example, QUIC also implements the Transport Layer Security (TLS) functionalities and thus requires the generation and setup of security certificates. Installing and integrating the QUIC implementations and ns-3 can also be complex and cause compatibility issues on some operating systems. Furthermore, a direct integration would not allow researchers to easily change the parameters of the protocol or test new features, requiring them to alter the code-base directly. Finally, ns-3 applications and existing implementations of congestion control algorithms are based on the structure of the ns-3 TCP socket, and cannot be reused without a native implementation. Therefore, we advocate that a QUIC implementation for ns-3 can benefit the networking research community due to its ease of use, despite introducing the burden of re-implementing and updating the protocol. Moreover, we designed the QUIC stack so that it is possible to re-use the various implementations of TCP congestion control in QUIC. This makes it possible to directly compare different congestion control designs for QUIC, while real implementations only offer adaptations of the NewReno and CUBIC schemes (Kakhki et al., 2017).

In the remainder of the paper, we will first provide an overview of the main features of QUIC in Sec. 2. Then, we will describe the ns-3 implementation in Sec. 3, with details on the code structure, the compatibility with TCP and the missing elements with respect to the QUIC Internet Drafts. In Sec. 4 we will illustrate some examples of QUIC’s performance evaluation. Finally, Sec. 5 will conclude the paper.

2. QUIC: protocol Description

As mentioned in Sec. 1, QUIC is a transport protocol implemented on top of UDP, initially proposed by Google in 2013. Since 2016, the IETF has started a process to standardize the protocol into multiple Internet Drafts, e.g., for the main transport functionalities (Iyengar and Thomson, 2018), recovery and congestion control (Iyengar and Swett, 2018), security (Thomson and Turner, 2018) and the bindings to the protocols at the application layer, e.g., the HyperText Transfer Protocol (HTTP) (Bishop, 2018).

Fig. 1 reports the configuration of the protocol stack (above the network layer) when considering a secure HTTP connection over QUIC or TCP. QUIC’s design indeed builds on that of TCP, but the two protocols also present significant differences. Like TCP, QUIC uses acknowledgments, duplicate acknowledgments, and the lack of acknowledgments to infer the status of the end-to-end connection and react by updating the congestion window or performing the necessary retransmissions. However, loss recovery is improved with respect to TCP: acknowledgments explicitly differentiate between out-of-order and lost packets, and intrinsically support Selective Acknowledgment (SACK) with a higher number of blocks than in TCP (Iyengar and Thomson, 2018). Moreover, QUIC can provide more information to the congestion control algorithms compared to TCP, consequently allowing better estimation of connection Round Trip Times (RTTs) (e.g., by explicitly signaling the delay between the reception of a packet and its ACK) and more efficient loss detection and recovery (Polese et al., 2018).

QUIC is also designed to support the multiplexing of several data (and control) streams over a single connection, with flow control implemented at both the stream and the connection level. The multiplexing is based on a novel packet structure, in which multiple frames (belonging to different streams, or carrying control information such as acknowledgments) are combined to create a QUIC packet. Thanks to the stream support, QUIC naturally combines with the streaming interface of HTTP/2,333Notice that the integration of HTTP on top of QUIC will be defined as HTTP/3. so that different application-layer streams can be mapped to different and independent streams that do not require in-order delivery, while still being part of the same transport layer flow. Therefore, even if one of the streams is affected by packet loss, the others can still reliably deliver their packets to the application at the receiver, preventing the so-called Head-of-Line (HOL) blocking issue that affects TCP. With TCP, indeed, the multiple streams that are generated in an HTTP/2 connection are bundled together at the transport layer. If a single packet is lost, TCP cannot release any packet to the receiver’s application layer, consequently blocking all the streams.

Moreover, QUIC integrates the functionalities of TLS 1.3, making it possible to authenticate and encrypt both the payload and (at least part of) the header. Thanks to this integration, QUIC makes it harder to perform injection attacks, eavesdropping or traffic analysis in middleboxes (Thomson and Turner, 2018). Additionally, this allows QUIC to reduce the latency of the initial connection establishment. Indeed, while TCP needs to perform a protocol handshake first, and then the TLS cryptographic handshake, with QUIC it is possible to embed the relevant cryptographic parameters into the first message sent between the two endpoints of the connection , thus reducing the initial handshake latency to a single RTT. If two endpoints have already connected in some previous interactions, it is also possible to perform a 0-RTT handshake, in which the encrypted data can already be sent in the first exchanged packet.

Finally, QUIC supports a connection-level identifier that provides robustness against updates in the underlying layers of the protocol stack, e.g., IP address updates caused by Network Address Translation (NAT) and/or mobility in cellular and Wi-Fi networks (Iyengar and Thomson, 2018).

3. Implementation of QUIC in ns-3

This Section describes the main features of the native QUIC implementation for ns-3. Fig. 2 reports the UML diagram for the QUIC socket, congestion control, headers and buffers code. As the diagram shows, we developed the main classes in the ns-3 internet module, which already models other transport protocols such as UDP and TCP. Moreover, the structure of the code resembles that of the TCP implementation, replicating the separation of congestion control from the main socket logic and the presence of stand alone buffer classes (Casoni and Patriciello, 2016), but at the same time introduces new elements to account for the novelties of QUIC with respect to TCP.

3.1. Code Structure

QuicSocketBase (which extends QuicSocket), the main class of the QUIC implementation, models the basic functionalities of a QUIC socket, like in the TCP implementation. Each client will instantiate a single QuicSocketBase object, while the server will fork a new socket for each incoming connection. A QuicSocketBase object receives and transmits QUIC packets and acknowledgments, accounts for retransmissions, performs flow and congestion control at a connection level, takes care of the initial handshake and exchange of transport parameters, and handles the life cycle and the state machine of a QUIC connection. An instance of the QuicSocketBase class holds pointers to multiple other relevant items, including:

• •

the socket transmission and reception buffers, implemented by QuicSocketTxBuffer and QuicSocketRxBuffer, respectively;

• •

the QuicSocketState object, which extends TcpSocketState (Casoni and Patriciello, 2016) with additional variables that are used by the QUIC state machine and congestion control;

• •

an object extending the TcpCongestionOps class, which performs the congestion control operations and provides a basic compatibility with the TCP congestion control implementations, as we will discuss later.

The QUIC socket is bound to an underlying UDP socket through a QuicL4Protocol object, which also handles the initial creation of the QuicSocketBase object, triggers the UDP socket to bind and connect, and handles the delivery of packets between the UDP socket and QuicSocketBase.

A stream is instead modeled by the QuicStreamBase class, which extends the basic QuicStream class. It buffers application data, performs stream-level flow control, and delivers the received data to the application. Similarly to the full socket, a QuicStreamBase object also has pointers to transmission and reception buffers, which are implemented by the QuicStreamTxBuffer and QuicStreamRxBuffer classes.

Multiple QuicStreamBase objects are connected to a single QuicSocketBase through an object belonging to the QuicL5Protocol class. A QuicSocketBase holds a pointer to a QuicL5Protocol object, and the latter contains a vector of pointers to multiple QuicStreamBase instances. The QuicL5Protocol class creates and configures the streams, and takes care of delivering packets or frames to be transmitted and received across the streams and the socket.

3.2. QUIC Packets, Frames and Headers

According to the QUIC Internet Drafts (Iyengar and Thomson, 2018; Iyengar and Swett, 2018), a QUIC packet is composed by a header and a payload. The QUIC packet is then encapsulated into a UDP datagram to be transmitted over the wire.

The header of a QUIC packet is modeled by the QuicHeader class, which extends the ns-3 Header class. It takes care of serializing and deserializing the header when attached to an ns-3 packet, and represents the information that a QUIC header carries. One of the main novelties of the QUIC header design with respect to the traditional TCP headers is that it can have two different formats (i.e., long or short) according to the amount of information that needs to be exchanged. Moreover, for short headers, the packet number entry in the header can have a variable length, i.e., one, two or four bytes according to how many bytes are actually necessary to represent the packet number. Long headers (17 bytes) are used for the message exchanges which are relevant to the connection life cycle (e.g., the initial handshake, version negotiation) and are sent prior to the establishment of the TLS keys. Short headers (from 2 to 13 bytes) are instead used for data packets and acknowledgments. The QuicHeader class also provides various static methods that can be used to create different kinds of headers (short, long, for handshake messages) and set the relevant parameters.

The payload of a QUIC packet, instead, is composed of multiple frames, with each frame mapped to a stream and/or a control operation. Each frame carries an additional subheader, which is implemented by the QuicSubheader class, and specifies the type of frame. Data frames are explicitly associated to a stream, through a stream identifier, and their subheader may carry the size of the frame, the offset with respect to the first stream byte, and a flag that signals the end of the stream. Control frames, instead, are used to perform specific actions and have custom formats. The most relevant control frames for the ns-3 QUIC code are the ACK frames, implemented according to the format specified in (Iyengar and Thomson, 2018; Iyengar and Swett, 2018), and used to acknowledge the packets received by an endpoint of the connection. Each ACK specifies the largest acknowledged sequence number and can carry multiple selective acknowledgment blocks if gaps in the received packet sequence are identified. Moreover, it is possible to explicitly specify in the ACK the delay between the reception of the acknowledged packet with the highest sequence number and the time at which its ACK was sent.

3.3. QUIC Buffers

The current implementation of QUIC for ns-3 features 4 different kinds of buffers, as mentioned in Sec. 3.1. We decided to keep separate buffer implementations between the socket and stream level because the functionalities of the various buffers are slightly different, and the buffers contain different items.

At the socket level, the transmission buffer, implemented in the QuicSocketTxBuffer class, stores stream or control frames to be delivered by QuicSocketBase and returns packets (composed by one or multiple frames) of the desired size to the socket. This buffer holds two different lists of packets, one for those not yet transmitted, and one for those transmitted (until they are acknowledged), in case retransmissions are needed. Each item stored in these lists is encapsulated in a QuicSocketTxItem, which associates the smart pointer to the packet with a possible sequence number, information on the timing of transmission and ACK reception, and a number of flags that identify the packet as lost, retransmitted, acknowledged or released. The QuicSocketTxBuffer implements a prioritization mechanism for the special frames of stream 0, which should be transmitted first whenever there is a transmission opportunity. Moreover, it is capable of splitting and assembling frames according to the actual packet size that the socket requires. Once a packet is delivered to the socket for the first transmission, the associated frames are removed from the unsent list, and the whole packet is added to the sent list. In this way, it is possible to correctly handle the reception of ACKs, release correctly received packets, handle retransmissions, and compute the number of bytes in flight.

At the stream level, the sender-side buffering is handled by QuicStreamTxBuffer instead. The transmission buffer stores application packets until they are acknowledged, but does not perform retransmissions, which, according to the QUIC Internet Draft, are only handled at the socket level for full QUIC packets. The packets in the stream buffer are identified by an offset with respect to the beginning of the stream. Moreover, the stream transmission buffer needs to gracefully handle the case in which it tries to deliver a packet to the underlying socket buffer, but the socket buffer is full and rejects the packet. In this case, the frame is re-added to the unsent list of the stream buffer, with the correct offset.

Receiver-side buffering follows the opposite paradigm: packets received by the socket are read, disgregated into individual subframes and stored in the QuicStreamRxBuffer. If the received bytes are out of order, they are stored in the stream buffer until the packet containing the missing bytes is received correctly. If the stream buffer gets some bytes in the correct order, it transfers them to the QuicSocketRxBuffer, which holds the received application packets released by the streams until the application requests them. Moreover, in case the stream has received the FIN bit (i.e., the transmission has ended), the receiver stream-level buffer can return the total amount of received bytes in the stream.

Given that the buffers are always initialized in either QuicSocketBase or QuicStreamBase, and following the ns-3 TCP implementation approach, it is possible to set the size of the buffer (i.e., the number of bytes they can buffer before rejecting packets) using attributes of these classes for the socket (SocketSndBufSize and SocketRcvBufSize) and the stream buffers (StreamSndBufSize and StreamRcvBufSize), respectively.

3.4. Data Flow in a QUIC Connection

In the following, we will describe the chain of events and calls that are triggered whenever an application sends a packet to the socket, and when the UDP socket at the other endpoint receives a datagram with a QUIC payload.

QuicSocketBase implements the ns-3 Socket Application Programming Interfaces (APIs), thus the application calls the Send method to deliver a packet to the socket. Notice that possible extensions of the classic BSD socket interface are being discussed for the interfacing between HTTP/3 streams and QUIC (Bishop, 2018), e.g., to explicitly signal the mapping between a packet and a stream to the socket. In order to make this possible in ns-3 without modifying the Socket implementation, we exploit the piggybacking paradigm and embed the information in the flag integer parameter that can be set when using the Send call. QuicSocketBase will then use this parameter as the identifier of the stream on which the packet should be sent, and the packet will be delivered to that stream through the QuicL5Protocol object. If it is the first packet of that stream, the corresponding QuicStreamBase instance is also initialized. Otherwise, if the application does not express a preference, then QuicL5Protocol will assign the packet to the first stream. The maximum number of streams can be set through attributes of QuicSocketBase.

If the stream flow control allows the transmission of the packet, then QuicStreamBase will create the QuicSubheader and deliver the frame to QuicL5Protocol, which will relay it to QuicSocketBase. If the socket buffer has enough space, the packet is added to the QuicSocketTxBuffer. The frame is then possibly aggregated to other frames and sent to the underlying UDP socket through QuicL4Protocol, according to the possible transmission opportunities enabled by the socket flow and congestion controls.

At the other endpoint, when the UDP socket receives the packet, it will trigger a callback to the ForwardUp method of the associated QuicL4Protocol. This method handles the possible cloning of the server socket if the packet is received in a listening state, and will eventually trigger an additional callback to deliver the packet to the ReceivedData method of QuicSocketBase. This method checks if the packet is a control packet (e.g., for the initial handshake and configuration) or a packet with a mixed payload with one or more data and ACK frames. If this is the case, QuicSocketBase will deliver the packet to the DispatchRecv method of the associated QuicL5Procotol object. This method cycles through the frames of the QUIC packet, triggering the relevant methods either in QuicSocketBase or in QuicStreamBase. The first is called whenever a control frame that needs to be handled by the socket is received (e.g., an ACK), while the second is called for stream-related control or data frames. The latter are buffered in the QuicStreamRxBuffer and released in the correct order to QuicSocketBase, which will notify the application through the NotifyDataReceived callback. The application can then call the RecvFrom method to extract the packets from the socket and consume them.

3.5. Retransmission Process

The retransmission process of QUIC works at the socket level, i.e., it retransmits complete packets (composed of one or multiple frames) that were lost, by leveraging the ACK transmission and reception flow that is embedded in the QUIC operations. In this implementation, the QuicSocketBase class handles the reception of ACKs through the OnReceivedAckFrame method, which is triggered from QuicL5Protocol when a received packet contains an ACK frame.

As mentioned in Sec. 3.2, an ACK frame always specifies the largest acknowledged packet number, and possibly contains several additional ACK blocks, which also specify missing packets. The socket relays this information to the associated QuicSocketTxBuffer object, which iterates over the list of sent packets, marks them as acknowledged or lost, following the procedure specified in (Iyengar and Swett, 2018, Sec. 4.2.1), and returns the list of the acknowledged packets to the socket. The socket then performs the relevant congestion control operations (e.g., by updating the socket state in case of losses, or increasing the congestion window), and triggers the retransmission process.

In QUIC, the retransmissions are always associated with new, monotonically increasing sequence numbers. Therefore, for each retransmission, QuicSocketBase increases the sequence number by one, and notifies the socket buffer. The latter moves the sent packets marked as lost back to the buffer of not-yet-transmitted packets, updating their packet number. Finally, QuicSocketBase retrieves the packets to be retransmitted and forwards them to QuicL4Protocol.

3.6. Compatibility with TCP Congestion Control implementations

One of the main features of the ns-3 TCP redesign described in (Casoni and Patriciello, 2016) is the modularity of the congestion control algorithm with respect to the main socket code. This is achieved by using a separate object, which extends a class with the basic congestion control functionalities, i.e., TcpCongestionOps. A number of congestion control algorithms have been implemented and added to the internet module, including, for example, TCP NewReno, HighSpeed, BIC, Illinois, Vegas (Nguyen et al., 2016). Therefore, QuicSocketBase can be used in legacy mode, with which it is possible to use the TCP congestion control algorithms to drive the congestion window of the QUIC connection as well. Besides, it is also possible to use more refined algorithms, which exploit the additional information that the QUIC socket provides, and develop congestion control methods for QUIC only.

The compatibility with legacy TCP congestion control algorithms is achieved by having a TcpCongestionOps instance as the basic congestion control object in QuicSocketBase. Then, we introduced a new class QuicCongestionControl, which extends TcpNewReno.444As expected, TcpNewReno extends TcpCongestionOps itself. QuicCongestionControl features additional methods that inject additional information specified by the QUIC Internet Draft in the congestion control algorithm (e.g., the transmission of a packet, more refined information on the RTT) if needed. When the congestion control algorithm is set in QuicSocketBase (i.e., after the socket is created by QuicL4Protocol), the SetCongestionControlAlgorithm checks if the specific algorithm extends QuicCongestionControl, and in this case sets the m_quicCongestionControlLegacy flag to false. Otherwise, the latter is set to true and the legacy mode is activated. Then, every time the socket needs to trigger the methods of the congestion control algorithm, e.g., when an acknowledgment is received, or a Retransmission Timeout (RTO) expires, it will check whether the operations are in legacy mode to call the relevant methods.

3.7. Connection Establishment

A QUIC connection is a single conversation between two QUIC endpoints (Iyengar and Thomson, 2018), each unequivocally identified by the tuple of IP addresses and UDP ports. We implemented a connection opening mechanism that follows QUIC’s Internet Draft, and performs the version negotiation and the transport handshake between the client and the server. Moreover, the procedure we implemented models the cryptographic handshake, but does not perform actual encryption, as we will discuss later. Fig. 3 reports a diagram of the QUIC state machine, with the transitions that happen during connection establishment and termination.

Notice that QUIC can execute the handshake in two different ways, according to the shared history of the two endpoints. If the endpoints have recently authenticated, then a 0-RTT handshake can be used. We model this by adding to QuicL4Protocol a list of authenticated endpoints, which is checked before sending the first packet and when it is received at the other endpoint. Moreover, the 0-RTT can be forced using the 0RTT-Handshake attribute in QuicL4Protocol. If 0-RTT is not allowed, the socket falls back to the 1-RTT handshake using the client initial message. Finally, if there is a version mismatch555Given the availability of early QUIC implementations, the Internet Draft defines a version field in the initial handshake header. between the two endpoints, then a 2-RTT handshake is performed to settle to a common QUIC version between endpoints.

**2-RTT handshake - ** In the ns-3 QUIC implementation, the 2-RTT handshake is forced by setting the 0-RTT attribute to false, and by configuring the InitialVersion attribute of QuicSocketBase to QUIC_VERSION_NEGOTIATION.666Other supported versions are defined in quic-socket.h. By setting this attribute to another version, the client and the server will assume to use the same version and avoid the 2-RTT handshake. The client (unaware of the fact that the server does not support the same QUIC version) sends a client hello initial message using the SendInitialHandshake method of QuicSocketBase. The server then rejects the connection and replies with a packet carrying a version negotiation frame, whose payload contains the list of all the versions the server supports encoded in 32 bits. The client then matches the list with one of the versions it supports and updates its version; finally, the handshake continues as a 1-RTT handshake.

**1-RTT handshake - ** This handshake is very similar to TCP’s, but it also embeds the setup of the TLS parameters. As in the 2-RTT handshake, the client sends a client hello initial message, i.e., a packet with a long header of type client initial, a certain QUIC version (which was possibly just negotiated) and a first packet number.777While according to the QUIC specification the initial packet number should be random, in the ns-3 implementation it is systematically set to zero in order to simplify the debugging of the protocol stack. Upon reception of the client initial packet, the server replies with a packet carrying a handshake header containing the relevant transport and cryptographic parameters and the connection identifier chosen by the server. Once the client receives this packet, it can conclude the transport and cryptographic handshake sending a new client hello. Its connection ID field will contain the server-selected connection ID, while its packet number has to be one higher than the last packet that was sent. From now on the packet number will be monotonically increased for each subsequent packet. The payload of this packet can contain stream frames, padding and ACK frames.

**0-RTT handshake - ** As mentioned in Sec. 2, this handshake is one of QUIC’s novelties with respect to TCP, which allows the setup of a new connection with just a one-way packet exchange, from the client to the server. Once the application triggers the socket, and if the aforementioned conditions are met, then the client will send a packet with a QuicHeader of the ZRTT_PROTECTED kind, carrying the previous connection identifier, the version and the cryptographic parameters.

3.8. Connection Termination

Besides the initial handshake, the implementation of QUIC for ns-3 also features a simplified connection termination mechanism, that triggers a draining period with an idle timeout or an immediate close message.

The draining period is a specific time interval in which the two endpoints synchronize on the closing of the connection and handle all the packets that could be received in the meantime. In fact, even if a socket has entered the CLOSING state, it could still receive some packets, because of imprecise synchronization between the idle timers of the two endpoints, or retransmissions and delayed packets still in flight in the network. The immediate close is the standard mechanism through which an application can close a connection. For this purpose, the QuicSocketBase has a specific Close method which, once called by the application, triggers the transmission of a specific frame signaling the end of the connection to the other endpoint. Once the closing packet is sent, the socket moves to the CLOSING state and enters the draining period. The same is done by the other endpoint upon receiving this packet. Additionally, both the client and the server keep an idle timeout timer, which is reset at every packet transmission or reception. The timer is initialized at a fixed value (300 s) and keeps running during all the connection lifetime. If it expires, both endpoints enter the draining period and move to the CLOSING state.

The draining period makes it possible to receive and acknowledge packets which were in flight after the closing of the connection, while avoiding new transmissions, except for those with application or connection close frames. Once the draining timer expires, the connection is definitively closed and all incoming packets are discarded. Both the client and server sockets are destroyed.

In our specific implementation, the Close method in QuicSocketBase is invoked either by the application to terminate the connection or by the socket itself when the idle timeout expires. This method moves the connection to the CLOSING state and starts the draining period timer. When the draining period timeout expires, the DoClose method is called. The socket callbacks are revoked, and then the socket is removed from the socket list of QuicL4Protocol calling the RemoveSocket method. If the closed socket was a server’s listening socket, then any other forked sockets are closed at the same time. In any case, when the socket list is empty, the UDP socket is closed as well.

3.9. Missing Features With Respect to QUIC Internet Drafts

This implementation is aligned with version 13 of the QUIC Internet Drafts, and it will be possible to update it in future releases to add new features and align it with the final version of the Request for Comments (RFC). Moreover, we list here the features that are missing from this implementation with respect to draft 13, and discuss the main assumptions that were made.

First of all, we decided not to implement the TLS stack or rely on external cryptographic libraries. While this represents a significant difference with respect to the Draft-compliant QUIC design, it will not affect the evaluation of the end-to-end performance of QUIC in terms of congestion control behavior and retransmission processes. Besides, we implemented the connection establishment, as described in Section 3.7, so that the need for the cryptographic handshake is accounted for, even though not actually performed. Moreover, given that this implementation targets a network simulator, there is no explicit need for the privacy and security features that TLS is expected to provide in a real QUIC implementation. Finally, researchers who want to use the ns-3 QUIC implementation will not need to install additional external libraries and security certificates.

In terms of transport capabilities, we do not allow the deletion of a stream, and some special frames (e.g., STREAM_BLOCK frames) are not currently supported, but can be easily added in future releases. Finally, there is a difference with the Draft with respect to how the server and client exchange the transport parameters, which specify a number of properties for the connection. According to the QUIC Internet Draft, the server sends its transport parameters to the client during the initial handshake, and the client applies them. While this exchange is actually simulated, it has no relevant effect on the configuration of the QUIC stack in ns-3, whose parameters are already shared by the server and the client in the implementation of QuicSocketBase. The reason is that QuicSocketBase itself is a class that represents both a client and a server, and has a single set of attributes for both. Therefore, by setting the attributes for the server, the user also sets those for the client.

3.10. Testing and Applications

Finally, we implemented a basic set of unit tests, which will be extended in future releases. The first test suite (i.e., quic-tx-buffer) checks the correctness of the behavior of the socket and stream transmission buffers, implemented by QuicSocketTxBuffer and QuicStreamTxBuffer, respectively. For both the socket and the stream, the test checks different insertion and removal events from the buffer, including test cases in which the insertion is not successful due to a full socket buffer and the packet needs to be re-inserted in the stream buffer. Moreover, the socket test verifies the behavior in case of retransmissions. Similarly, the second test suite (i.e., quic-rx-buffer) performs add and remove operations on the reception buffers of the socket and stream. Finally, the third test suite (i.e., quic-header) verifies the correctness of the QuicHeader and QuicSubheader implementations.

Moreover, in order to explicitly use the stream multiplexing capabilities, we implemented an adaptation of the UDP client/server applications (i.e., QuicClient and QuicServer) which call the Send API of the socket using a stream identifier. In this first implementation, the data is scheduled over the available streams (whose number can be set with an attribute) in a round robin fashion.

4. Examples

We now provide a set of preliminary results to validate the behavior of the ns-3 QUIC implementation when using different congestion control algorithms, both in legacy and non-legacy modes. We consider a classic dumb-bell topology, often used to evaluate TCP performance (Mishra et al., 2016), with two pairs of clients and servers sharing a common bottleneck of 2 Mbps. The minimum RTT is 100 ms, and the example we used to run this experiment is quic-variants-comparison, adapted from the respective TCP example.

Fig. 4 reports the evolution of the congestion window and the RTT for the two flows, for two legacy congestion control algorithms (i.e., NewReno and Vegas) and the non-legacy option, which adapts NewReno with a slightly different congestion window increase in the congestion avoidance phase (Iyengar and Swett, 2018). The evolution of the congestion windows matches the expected behavior of the algorithms, with the two flows that settle in steady state to the Bandwidth-Delay Product (BDP) (i.e., 25 kB). Moreover, when comparing the behaviors of delay-based (i.e., Vegas) and loss-based (i.e., NewReno and QUIC) congestion control, it can be seen that, as expected, Vegas manages to maintain a smaller RTT. Finally, NewReno and QUIC congestion control exhibit similar trends, with the latter being characterized by a faster window ramp-up during the congestion avoidance phase.

5. Conclusions

In this paper, we presented a native ns-3 implementation of QUIC, a new transport protocol that is being standardized by the IETF and already accounts for a significant portion of the Internet traffic. The implementation of QUIC described in this paper extends the code structure of the TCP ns-3 code base with the features that characterize QUIC, e.g., stream multiplexing, low-latency initial handshake, improved SACK through ACK frames. Moreover, we designed the QUIC socket implementation so that it is possible to plug both the legacy TCP and new QUIC-only congestion control algorithms. We validated the performance of our implementation through simulations on a dumb-bell topology, which showed the typical behavior of the legacy congestion control algorithms and compared their performance to a non-legacy, Internet-Draft-based QUIC congestion control. The code is publicly available, and is compatible with the latest version of ns-3 at the time of writing.

Acknowledgements.

This work was partially supported by the U.S. Commerce Department/NIST through the project “An End-to-End Research Platform for Public Safety Communications above 6 GHz.” The authors would also like to acknowledge Davide Marcato and Stefano Ravazzolo for their initial contribution to the project.