Remote Estimation of Free-Flow Speeds

TL;DR

This paper introduces a deep learning method to estimate free-flow speeds of roads using overhead imagery and minimal road attributes, reducing reliance on costly detailed labels.

Contribution

It presents a CNN-based approach that effectively estimates free-flow speeds from imagery alone or combined with simple road features, improving estimation accuracy.

Findings

Imagery alone nearly matches the accuracy of detailed road features.

Combining imagery with road features yields the highest estimation accuracy.

The method performs well on a large, diverse dataset.

Abstract

We propose an automated method to estimate a road segment's free-flow speed from overhead imagery and road metadata. The free-flow speed of a road segment is the average observed vehicle speed in ideal conditions, without congestion or adverse weather. Standard practice for estimating free-flow speeds depends on several road attributes, including grade, curve, and width of the right of way. Unfortunately, many of these fine-grained labels are not always readily available and are costly to manually annotate. To compensate, our model uses a small, easy to obtain subset of road features along with aerial imagery to directly estimate free-flow speed with a deep convolutional neural network (CNN). We evaluate our approach on a large dataset, and demonstrate that using imagery alone performs nearly as well as the road features and that the combination of imagery with road features leads to the highest accuracy.

Tables

Table 1. Table 1 : Within-5 accuracy for each method.

Method	Within-5 Acc.
Imagery Only	37.60
Road Features Only	40.07
Combined	49.86

Equations

$$L=-\frac{1}{N}\sum _{i=1}^N\sum _{k=1}^K{y}_{i,k}\log {\hat{y}}_{i,k}$$

Full text

**Index Terms— ** urban understanding, transportation engineering, roadway safety

1 Introduction

The behavior of an average automobile driver is based on numerous factors. In this work, we focus on one aspect of behavior: the speed of travel. We specifically focus on the free-flow speed, which is the average vehicle speed along a roadway when there is no congestion or adverse weather conditions. The free-flow speed is used in a wide variety of planning and regulatory contexts. Standard practice for estimating free-flow speeds relies on the availability of many road features, such as lane widths and curvature. While some of these features are easy to gather, others require expensive surveying equipment and expert annotators, restricting their availability. As a result, in many states, roads are assigned the default speed limit of 35 mph for urban areas and 55 mph for rural areas, which can often be inconsistent with actual operational speeds and lead to undesired driver behavior.

To automate the costly process of estimating free-flow speeds, we propose a deep convolutional neural network (CNN) that takes as input the overhead imagery of the roadway and coarse-grained road features, including the posted speed limit, the functional classification, and the type of area. As output, our CNN generates a probability distribution over integer free-flow speeds. For training data, we use aggregated data collected from real-world drivers. At inference time, since the input features are all easily obtainable, it should be possible to rapidly, and inexpensively, estimate free-flow speeds over large spatial regions.

Using a large-scale evaluation dataset, we find that using only image features result in the poorest performance, but that combining imagery and road features result in significantly better performance than road features alone.

2 Related Work

Different studies have been proposed to estimate and map properties of the visual world using overhead images. Several authors have proposed different deep learning based approaches for vehicle detection [1, 2] and road extraction [3, 4, 5, 6] from aerial images. Salem et al. [7] introduced an approach for mapping soundscapes of geographic regions using overhead imagery. Greenwell et al. [8] proposed a model that is capable of predicting object histograms from overhead imagery. Several works have addressed the problem of speed estimation. Shuai et al. [9] introduced an approach to estimate the vehicle speed in traffic videos. Most similar to our work, Song et al. [10] proposed a model for road safety estimation based on the usRAP Star Rating Protocol. While this star rating is based on approximately 60 road safety features [11], their network works directly on ground level panorama images. We propose a new method that instead uses overhead imagery with the addition of auxiliary features.

3 Approach

We utilize a CNN architecture to estimate the free-flow speed of a given road segment. The neural network uses both aerial imagery and relevant road features as input, and outputs a probability mass function over $K$ possible free-flow speeds. We begin by describing the dataset that we use, followed by a more detailed description of the proposed network architecture.

3.1 Dataset

Our free-flow speed dataset is obtained through HERE Technologies and further annotated to incorporate coarse-grained road feature data needed for training. To calculate the free-flow speed for a particular road segment, driving speeds of vehicles were monitored during the year 2014. As free-flow speed refers to the speed that a driver can achieve without traffic congestion, we only consider data during non-holiday weekday periods from 9am to 3pm. The speeds are then averaged to obtain the ground-truth for each road segment. We rounded each ground-truth speed to the nearest integer to obtain a discrete label for training. This results in $K=79$ unique free-flow speeds.

For each road segment, we obtain an aerial image through the National Agriculture Imagery Program (NAIP) data. Each image is centered at the beginning of the segment, and covers an aerial view of $400\times 400$ m2 of land. We orient the images such that the direction of travel is up, and we reshape it to $224\times 224$. In total, our dataset consists of $1\,335\,132$ road segments and their corresponding aerial images.

Figure 2 shows the distribution of the free-flow speeds along with road features used for training. The drastic difference between the distribution of free-flow speeds and speed limits is evident here. In fact, over half of the roads have free-flow speeds significantly slower than the speed limits (more than 10-miles-per-hour). We will examine a few roads with such differences in Section 4.2, using predictions obtained from our model.

We split our dataset into training and testing sets. To ensure the sets are disjoint, we partition the dataset by location. The dataset is divided into counties, eight of which are selected for evaluation. The resulting test set is roughly $7\%$ of the total data, and contains a variety of roads with different area types, terrain, and free-flow speeds. A validation set ($1\%$ of the training set) is reserved for model selection.

3.2 Network Architecture

The backbone of our architecture is the Xception [12] model, which is used to extract high-level features from the aerial image. The resulting feature vector of length $2048$ is passed to a dense layer with output size $512$, denoted as $S$, as shown in Figure 1. In addition to aerial imagery, the network is trained using three integer features related to driving speed: area type, functional classification, and posted speed limit. The three features are concatenated with $S$ before feeding the combined feature vector into the final dense layer with output size $K$. We denote the output of the final layer as $\hat{y}$.

3.3 Loss Function

We formulate the task of predicting the free-flow speed of a road segment as a multiclass classification problem with ground-truth free-flow speed label $l$. With the output of the network defined in 3.2, we compute the training loss as the standard cross-entropy loss between the predicted distribution $\hat{y}$ and the target distribution $y$, as shown in the following equation:

[TABLE]

where $N$ is the number of training examples.

3.4 Implementation Details

Our model is implemented using Tensorflow. The network is optimized using the Adam optimizer with default parameters. We initialize the Xception network with weights pre-trained on ImageNet for image classification. During training, we freeze the weights of the Xception network and optimize only the last two dense layers with learning rate $0.001$. We decay the learning rate exponentially by a factor of 10 every 5 epochs. ReLU activation layers are used throughout the network except for the last layer, which uses the Softmax activation instead. We apply $L_{2}$ regularization to the two dense layers with scale $0.00005$. We train with batch size 16, for a total of 15 epochs.

4 Evaluation

Using the dataset described in 3.1, we trained our proposed model along with two other variations: imagery-only model and road-features-only model. We conducted quantitative and qualitative evaluation on our reserved test set.

4.1 Quantitative Analysis

We aim to discover the effect of each input modality in predicting free-flow speeds. We calculate the accuracy of the three trained models (overhead imagery only, road features only, and both) on the test set. Table 1 displays the within-5 accuracy for each method, where we consider a prediction to be positive if it is within five-miles-per-hour of the ground-truth speed. As we can see, road feature data is better than aerial imagery for predicting free-flow speeds, but we obtain the best performance when we combine both modalities.

4.2 Qualitative Evaluation

Free-flow speed should be similar to the speed limit for a given road, but that is often not the case. We are interested in finding roads where the predicted free-flow speed is drastically different than the speed limit. Figure 3 shows such examples. The top two roads have free-flow speeds much lower than the speed limits, most likely due to the curvature and number of intersections present. The bottom two roads have free-flow speeds much higher than the speed limits, which is understandable since the roads are straight, wide, and without congestion. Detection of these discrepancies is very useful, as traffic engineers can quickly filter through millions of roads to identify ones that need to have their speed limits re-evaluated.

We also compared predictions from our top two models: one trained on both aerial imagery and road features, and the other trained only on road features. Figure 4 shows the images that had drastically different predictions from the two models. The images in the right column received higher free-flow speed predictions from the combined model, where the images in the left column received higher free-flow speed predictions from the feature only model. These differences further support the importance of image features for prediction, as imagery can provide fine-grained information about the road segment. For the left three roads, it is natural for people to drive slower in residential areas or at intersections; for the right three, straight roads and ones parallel to interstates allow drivers to achieve greater speeds. Fine-grained labels such as presence of intersections, population density, and curvature of the road are hard to obtain, but they can easily be inferred from aerial imagery.

5 Conclusion

We introduced a method for estimating the free-flow speed of a road segment, which is important for understanding driver behavior. We demonstrated that a combination of aerial imagery and related road features as input is best for prediction, since it obtained higher accuracy than models trained on either features alone. We also performed qualitative evaluation, and obtained insights on the effect of input modalities along with the relationship between free-flow speeds and speed limits. We hope to extend this work and include ground-level imagery as an additional input, since information such as roadside hazards may not be visible in aerial imagery.

Acknowledgements

We thank Mei Chen and Xu Zhang, both from the Univ. of Kentucky Dept. of Civil Engineering, for their assistance in obtaining and understanding this dataset. We also acknowledge the support of NSF CAREER (IIS-1553116).