A Performance Evaluation Tool for Drone Communications in 4G Cellular Networks

TL;DR

This paper presents a measurement tool for evaluating drone communication performance over 4G cellular networks, capturing LTE parameters, throughput, and GPS data to assist in network assessment.

Contribution

It introduces an Android-based measurement tool specifically designed for assessing drone communication performance in 4G cellular networks, including throughput and LTE parameter analysis.

Findings

The tool effectively records LTE parameters and throughput data.

Sample measurements demonstrate its practical application.

The tool aids in identifying network performance issues for drones.

Abstract

We introduce a measurement tool for the performance evaluation of wireless communications with drones over cellular networks. The Android software records various LTE parameters, evaluates the TCP and UDP throughput, and tracks the GPS position. Example measurement results are presented.

Full text

I Introduction

The commercial and industrial market for small-drone appli-

cations is rapidly growing and provides opportunities in terms of surveillance, monitoring, transport, emergency management, and agriculture, among many other domains. In terms of wireless connectivity, off-the-shelf drones are usually equipped with Wi-Fi. Whereas this is sufficient for applications with low coverage requirements, cellular networks could provide wide area coverage, which is especially important for autonomous flights extending beyond line of sight (LoS). However, today’s cellular networks — including Long Term Evolution Advanced (LTE-A) — are not optimized for aerial devices [1]. For example, the base station antennas are typically tilted downwards to serve users on the ground rather than flying devices. Technical enhancements are necessary to optimize the connectivity of drones, e.g., to manage interference and handovers [2]. Upcoming cellular systems — with advanced beamforming, beam steering, and full dimensional multi-input multi-output (FD-MIMO) — may alleviate shortcomings to provide better aerial connectivity [3, 4].

Several experimental studies have explored drone communication with Wi-Fi (see [5, 6, 7, 8]) but few relate to cellular-based drone communication (see [2, 1, 9, 10, 11]). In order to study cellular-connected drones, the research community needs an evaluation tool to analyze the performance in current and forthcoming 3rd Generation Partnership Project (3GPP) releases and to design drone applications accordingly. This paper introduces such a tool, namely the cellular drone measurement tool (CDMT). It integrates several important features to evaluate cellular communication in aerial networks, such as monitoring of signal strength and cell information, assessing Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) performance, and tracking Global Positioning System (GPS) coordinates. Some results obtained with CDMT are presented in this paper. We demonstrate the tool in action by flying an AscTec Pelican drone with an Android smartphone. In our experiment, we observe some ping-pong handovers and recognize that the downlink (DL) throughput is higher on the ground, whereas the uplink (UL) throughput is higher in the air. More results on handovers and throughput using CDMT are given in [12] and [13].

The remainder of the paper is organized as follows: Section II provides an overview of available LTE measurement tools, before Section III describes LTE signal metrics. Section IV introduces CDMT and presents its features. Section V shows some example experiments, and finally Section VI concludes the paper.

II Related Tools

While some LTE measurement tools with various features are freely available, to the best of our knowledge, none of them integrates all of the parameters supported by CDMT. Moreover, they usually focus exclusively on TCP as the transport layer protocol. Commercial solutions offering similar parameters such as CDMT are expensive and often reside within the operator’s network [14, 15]. The performance and power characteristics of LTE, 3G, and Wi-Fi are analyzed in [16]. The Android tool 4GTest [17] is employed to characterize the LTE network performance. It uses a client-server model with multiple servers deployed at different locations and allows users to switch between network types to capture performance measurements. A model to characterize the round trip time and throughput over the signal strength is derived from measurements with a customized measurement tool in [18]. The tool uses a client-server model that records the UDP performance in the DL. It is not publicly available and can track only a few parameters of cellular networks.

The Nemo Handy Handheld Measurement Solution [19] is a commercial Android-based network testing application. It provides many relevant parameters, including connection setup delay, download time, time-to-connect delay, throughput, reference signal received power (RSRP), reference signal received quality (RSRQ), latitude, longitude, altitude, and information about neighboring cells. This tool has been used for LTE measurements with a drone in [10]. It was observed that the signal strength in that scenario increases from $-93$ dBm to $-74$ dBm until the drone reaches an altitude of $34$ m, before subsequently declining again with rising flight height. Another measurement campaign with a drone and Nemo was carried out to analyze LTE signals (RSRP and signal-to-interference ratio) in dense environments up to an altitude of $350$ m [11]. Here, the signal strength increases as the drone clears the height of the buildings due to LoS links with the base stations. However, higher path loss and interference from neighboring cells adversely affects the received signal power.

The study [20] addresses LTE and Wi-Fi and analyzes whether transport protocols should select the best network or better use multipath TCP. The “Cell vs WiFi” [21] app measures and compares the performance of LTE and Wi-Fi on Android devices. The app is currently unavailable.

In conclusion, available tools are either proprietary and not freely available or designed in simplicity, not capturing all important parameters necessary to analyze cellular-based drone communication. Thus, we developed a measurement tool that supports both TCP and UDP and captures information about signal strength, frequency bands used, and neighboring cells.

III LTE Signal Metrics

The three metrics related to LTE signal strength available on Android devices are RSRP, RSRQ, and RSSNR (reference signal signal-to-noise ratio) [22]. First, RSRP provides the average power received by the resource element carrying the reference signal in any symbol. It ranges from $-44$ dBm to $-140$ dBm[23]. It is primarily used for cell selection and handover. However, a high throughput can be observed even with low RSRP. Second, RSRQ provides additional information about when to perform a handover. It indicates the quality of the reference signal with typical values ranging between $-19.5$ dB (worst) and $-3$ dB (best)[23]. Third, RSSNR measures the signal-to-noise ratio of the received signal. It is used to assess the the signal quality and is an additional parameter to make handover decisions. Typical values range from $30$ dB (best) to $-20$ dB (worst)[24].

The UE sends these parameters to the serving base station (eNodeB) as a measurement report. Acoordingly, the eNodeB makes the handover decision based on the reports received [25].

IV Cellular Drone Measurement Tool

CDMT is a performance evaluation tool (see Fig. 1) developed for the Android platform to be used for aerial devices connected with 4G cellular networks. It is available at

https://www.lakeside-labs.com/cdmt

and can be utilized for academic research. The CMDT records LTE parameters such as RSRP, RSRQ, RSSNR, cell signal quality (CSQ), serving physical cell identity (PCI), channel quality indicator (CQI), E-UTRA absolute radio frequency channel number (EARFCN), neighboring cell information including PCI, EARFCN, RSRP, and RSRQ. It supports throughput measurements for TCP and UDP for DL and UL, packet delay for UDP, and it logs GPS information (time, latitude, longitude, altitude, number of available satellites, speed, and acceleration). All parameters are logged at a rate of $1$ Hz.

In order to measure throughput and round trip time (RTT), CDMT exploits a client-server model, where the client application is executed on Android. On the client side, the server network configuration (e.g., IP address and ports) and — depending on the type of measurement — some further parameters (e.g., UDP segment size) have to be configured (Fig. 1(a)). Once the parameters are configured and the server application is running, measurements can be recorded (Fig. 1(b)). Graphical representations of RSRP and throughput measurements are shown once the recording has been finished (Fig. 1(c)).

The server application is written in Java. It comprises two modules: the control module starts and stops measurements and reports results to the client, while the data module sends or receives data via TCP and UDP sockets. The UDP data rate is measured on the client for downlink tests and on the server for uplink tests. In the latter case, the server reports the data rate achieved to the client, which stores it in a log file. The TCP data rate is always calculated on the client.

It is assumed that the server is accessible via a public Internet Protocol (IP) address. Hence, we can always perform tests for the TCP uplink, TCP downlink, and UDP uplink. UDP downlink tests may not be available depending on the Network Address Translation (NAT) configuration of the mobile operator. CDMT only provides a simple NAT traversal technique: the client first sends a UDP packet to the server to create a mapping at the network address translator, and the server then uses the external IP address and port combination of the received packet as an endpoint for the UDP downlink. However, this mechanism does not work if the NAT changes the port mapping during the measurement. Since this was the case for our mobile operator, we selected an Access Point Name (APN) setting that does not use carrier NAT but rather a public IP address configuration at the end device instead.

The TCP throughput is measured by downloading or uploading a random stream of data created at the server or client, respectively. Alternatively, the TCP downlink test can be made using the Hypertext Transfer Protocol (HTTP) via a Uniform Resource Locator (URL), pointing to a file that is then (repeatedly) downloaded by the client until the measurement is manually stopped. In this configuration, the custom measurement server is not involved. The UDP throughput is measured by sending data packets of configurable size. The first four bytes contain the sending timestamp (to calculate the transmission delay) and the remaining bytes are chosen randomly. This requires synchronized clocks at the server and the Android device. The calculated delay is reported back using the TCP control socket.

The user starts and terminates a test manually. Measurement values are recorded every second and stored on the client platform for offline analysis. Most existing tools do not support such recording but rather only provide averaged values at the end.

V Experimental Study

This section describes an experimental study that showcases the possibilities of the CDMT. Further experiments with CDMT — along with more comprehensive results on throughput and handover rates — are presented in [12] and [13].

V-1 Setup

A Sony Xperia H8216 smartphone running Android $8.0$ is used. It has two quad-core processors with $4$ GB RAM and the Qualcomm Snapdragon 845 chipset supporting LTE carrier aggregation (CA). It is mounted on an AscTec Pelican drone, which can carry a payload of $650$ g with a flight time of about $16$ minutes. We evaluate the throughput over LTE-A (3GPP Release 13) with ground and aerial measurements. CA is activated in the DL on four frequency bands (LTE800, LTE1800, LTE2100, and LTE2600), and no CA is used in the UL. The maximum antenna output power is $46$ dBm.

For UDP experiments, a packet size of $8,\!192$ bytes is chosen. TCP downlink tests use the file download mode (i.e., a $1$ GB file is repeatedly downloaded from a public server). All other experiments (TCP UL, UDP UL and DL) are performed by configuring CDMT to connect to a measurement server located at our lab and running the aforementioned Java module. Measurements are performed as line tests from a reference point to a distance of $150$ m. The drone is set to fly at an altitude of $50$ m. Fig. 2 shows the flight path trajectory and the locations of the serving PCIs.

V-2 Throughput

Fig. 3 shows the measured throughput over flight distance. Eight curves are given for the throughput in the DL (eNodeB to UE) and UL (vice versa) using either TCP or UDP for both aerial and ground measurements. The presented curves are the mean of three experimental runs. It can be observed that the TCP throughput is higher on the ground than in the air (for both DL and UL), while the reverse is true for the UDP throughput in the UL. The UDP throughput in the DL on the ground and in the air are about the same.

The behavior of the TCP throughput can be explained by the fact that the RSRP is always better on the ground (Fig. 4). For UDP, the RSRP on the ground is much worse than in the air. However, an actual relationship between RSRP and throughput cannot be established.

V-3 Handovers

In the aerial scenario, in the TCP DL we observed a ping-pong handover from PCI 130 to 388 and back to 130 once in one run. No handover was observed during the other two runs as the UE was always connected to PCI 130 or 92, respectively. In the ground scenario, we observed several ping-pong handovers: in the three TCP DL runs, changes were from PCI 263 to 56 and back to 263, from 130 to 295 and back to 130, and from 92 to 109 and back to 92. In the TCP UL, handovers happened from PCI 263 to 56 and back to 263 (in two runs) and from 92 to 109 and back to 92 (in one run). No handover was observed in the TCP aerial UL. The PCIs selected were 92, 263, and 130 for the three runs.

In the UDP DL, no handover was observed on the ground nor in the air. The UE connected to PCI 263 in the air and to PCI 359 on the ground. In the UDP UL, the UE remained connected to PCI 263 in all three aerial experiments and performed a ping-pong handover from PCI 263 to 56 and back to 263 in all three ground experiments.

VI Conclusions and Outlook

The cellular drones measurement tool (CDMT) targets aerial devices connected to 4G networks. It can be used to record LTE performance parameters available on Android platforms, including TCP and UDP throughput, and supports tracking via GPS. We demonstrated its feasibility with example results from a measurement campaign over LTE-A. These results show better TCP downlink and uplink throughput on the ground and higher UDP uplink throughput in the air. Accompanying and future work includes comprehensive experimental evaluations at different flight heights and an adaptation of CDMT for 5G networks.