Simulation and Learning for Urban Mobility: City-scale Traffic Reconstruction and Autonomous Driving

TL;DR

This paper proposes combined simulation and machine learning techniques for city-scale traffic reconstruction and autonomous driving to address global traffic congestion issues.

Contribution

It introduces novel methods integrating simulation with machine learning for large-scale traffic analysis and autonomous vehicle development.

Findings

Enhanced traffic reconstruction accuracy

Improved autonomous driving performance

Potential reduction in traffic congestion

Abstract

Traffic congestion has become one of the most critical issues worldwide. The costs due to traffic gridlock and jams are approximately $160 billion in the United States, more than {\pounds}13 billion in the United Kingdom, and over one trillion dollars across the globe annually. As more metropolitan areas will experience increasingly severe traffic conditions, the ability to analyze, understand, and improve traffic dynamics becomes critical. This dissertation is an effort towards achieving such an ability. I propose various techniques combining simulation and machine learning to tackle the problem of traffic from two perspectives: city-scale traffic reconstruction and autonomous driving.

Tables

Table 1. Table 1.1: The absolute errors in the recovered travel time computed using my technique vs. Lou et al.  Lou et al., ( 2009 ) by using GPS traces from various percentages of the traffic population. My technique results in much smaller errors as of Lou et al.  Lou et al., ( 2009 ) .

	Absolute errors to the ground truth
	My technique		Lou et al. Lou et al., (2009)
Traffic percentage	mean ($s$)	std. ($s$)	mean ($s$)	std. ($s$)
20%	8.29	5.31	29.74	21.90
40%	4.25	3.38	29.85	22.66
60%	3.67	3.67	29.33	22.06
80%	3.40	3.32	29.55	22.46
100%	3.58	4.04	29.66	22.35

Table 2. Table 1.2: Test Results of On-Road Scenarios: my policies O s ​ t ​ r ​ a ​ i ​ g ​ h ​ t subscript 𝑂 𝑠 𝑡 𝑟 𝑎 𝑖 𝑔 ℎ 𝑡 O_{straight} & O f ​ u ​ l ​ l subscript 𝑂 𝑓 𝑢 𝑙 𝑙 O_{full} can lead to robust collision avoidance and lane-following behaviors.

	Test Policy and Success Rate (out of 50 runs)
Scenario	$B_{follow}$	$O_{follow}$	$B_{straight}$	$O_{straight}$	$B_{full}$	$O_{full}$
Straight road / Curved road	100%	100%	100%	100%	100%	100%
Straight road + Static obstacle	0%	0%	0%	100%	0%	100%
Curved road + Static obstacle	0%	0%	0%	0%	0%	100%

Table 3. Table 2.1: The absolute errors in the recovered travel times computed using my technique vs. Lou et al.  Lou et al., ( 2009 ) , using GPS traces from various percentages of the traffic population. My technique results in much smaller errors as of Lou et al.  Lou et al., ( 2009 ) .

	Absolute errors to the ground truth
	My technique		Lou et al. Lou et al., (2009)
Traffic percentage	mean ($s$)	std. ($s$)	mean ($s$)	std. ($s$)
20%	8.29	5.31	29.74	21.90
40%	4.25	3.38	29.85	22.66
60%	3.67	3.67	29.33	22.06
80%	3.40	3.32	29.55	22.46
100%	3.58	4.04	29.66	22.35

Table 4. Table 4.1: Training Data Summary: my method can achieve over 200 times more training examples than DAGGER  Ross et al., ( 2011 ) at one iteration leading to large improvements of a policy.

	Training Module (Data)					Other Specs
Scenarios	Following (#Images)	Avoidance (#Images)	Detection (#Images)	Total	Data Augmentation	#Safe Trajectories	Road Type	Obstacle
Straight road	33 642	34 516	32 538	97 854	$212$x	74	on-road	static
Curved road	31 419	33 624	71 859	136 855	$98$x	40	on-road	static
Open ground	30 000	33 741	67 102	130 843	$178$x	46	off-road	dynamic

Table 5. Table 4.2: Test Results of On-Road Scenarios: my policies O s ​ t ​ r ​ a ​ i ​ g ​ h ​ t subscript 𝑂 𝑠 𝑡 𝑟 𝑎 𝑖 𝑔 ℎ 𝑡 O_{straight} & O f ​ u ​ l ​ l subscript 𝑂 𝑓 𝑢 𝑙 𝑙 O_{full} can lead to robust collision avoidance and lane-following behaviors.

	Test Policy and Success Rate (out of 50 runs)
Scenario	$B_{follow}$	$O_{follow}$	$B_{straight}$	$O_{straight}$	$B_{full}$	$O_{full}$
Straight road / Curved road	100%	100%	100%	100%	100%	100%
Straight road + Static obstacle	0%	0%	0%	100%	0%	100%
Curved road + Static obstacle	0%	0%	0%	0%	0%	100%

Equations

$$\hat{\pi }=\underset{\pi \in \Pi }{\mathrm{argmin}}\sum _{t=1}^{T}C\left(s_t,a_t\right),$$

$$\hat{\pi }=\underset{\pi \in \Pi }{\mathrm{argmin}}{\mathbb{E}}_{\varphi \sim d_{\pi },a^{\ast }\sim {\pi }^{\ast }(\varphi )}\left[l\left(\varphi ,\pi ,a^{\ast }\right)\right].$$

$$J\left(\hat{\pi }\right)\le T{ϵ}_{min}+T{ϵ}_{regret}+\mathcal{O}(\frac{f\left(T,l_{max}\right)}{N}),$$

$$J\left(\hat{\pi }\right)\le J\left(\bar{\pi }\right)\le T{\widehat{ϵ}}_{min}+T{\widehat{ϵ}}_{regret}+\mathcal{O}\left(T{l}_{max}\sqrt{\frac{\log \frac{1}{\mu }}{KN}}\right)$$

$$J\left(\hat{\pi }\right)\le J\left(\bar{\pi }\right)\le T{\widehat{ϵ}}_{min}+\mathcal{O}\left(ϵ\right)$$

$$P_c(w)=min\{1,\frac{T_{c,max}^{prev}+T_{c,max}}{T_a^{prev}+T_a}\cdot \frac{1}{w}\},$$

$$L_{s,e}(w)=\beta w+(1-\beta )\frac{e.k}{e.k_{max}},$$

$$e.k=e.k_{max}\left(1-\frac{e.v}{e.v_{max}}\right).$$

$$P_{s,e_i}(w)=L_{s,e_i}(w)\prod _{e_j\in p_{est},j\ne i}(1-L_{s,e_j}(w)).$$

$${\tau }_{e,c}={\int }_0^{w_{max}}{\tau }_{e,f}\frac{w}{1-w}\frac{P_c(w){\sum }_e{P}_{s,e}(w)}{Q_s}dw,$$

$${\tau }_{e,s}={\int }_0^{w_{max}}{T}_s\frac{P_c(w)P_{s,e}(w)}{Q_s}dw.$$

$$\mathbf{\Phi }{f}_{dct}=f,$$

$$\mathbf{A}{\hat{f}}_{dct}=\mathbf{\Psi }\mathbf{\Phi }{\hat{f}}_{dct}=b.$$

$$\text{min }\Vert {\hat{f}}_{dct}{\Vert }_{{\mathcal{L}}_1},$$

$$\text{s.t. }\mathbf{A}{\hat{f}}_{dct}=b.$$

$$u_{od}=\sum _{k\in {\mathcal{K}}_{od}}u(k),u(k)\ge 0,\forall o\in \mathcal{O},d\in \mathcal{D}.$$

$$f(e)=\sum _{(o,d)\in \mathcal{O}\times \mathcal{D}}\sum _{k\in {\mathcal{K}}_{od}}{\delta }_e^{k}u(k),$$

$$f(e)={\mathbf{P}}_e\mathbf{u}.$$

$${\mathbf{f}}_{\mathcal{E}}={\mathbf{P}}_{\mathcal{E}}\mathbf{u},$$

$$\mathcal{W}={\left\{t(k)\ge t(s_1,s_2)\right\}}_{\forall (s_1,s_2)\in {\mathcal{S}}_p,\forall k\in {\mathcal{K}}_{s_1,s_2}}.$$

$${\bar{\mathbf{t}}}_{\mathcal{E}}=\underset{{\mathbf{t}}_{\mathcal{E}}}{\mathrm{argmin}}$$

$$\mathcal{W},{\mathbf{t}}_{\mathcal{E},min}\le {\bar{\mathbf{t}}}_{\mathcal{E}}\le {\mathbf{t}}_{\mathcal{E},max}.$$

$${\bar{k}}_{s_1,s_2}=\underset{\bar{k}\in {\bar{\mathcal{K}}}_{s_1,s_2}}{\mathrm{argmin}}\Vert t(s_1,s_2)-\bar{t}(\bar{k})\Vert ,$$

$$\text{with}{\bar{\mathcal{K}}}_{s_1,s_2}={\left\{\underset{k\in {\mathcal{K}}_{q_1,q_2}}{\mathrm{argmin}}\bar{t}(k)\right\}}_{\forall (q_1,q_2)\in {\mathcal{Q}}_1\times {\mathcal{Q}}_2}.$$

$$\underset{\mathbf{\theta }}{\mathrm{maximize}}\mathcal{L}(\mathbf{\theta }|\mathcal{N})=\sum _{\forall (s_1,s_2)\in {\mathcal{S}}_p}\text{log}\pi (t(s_1,s_2)|{\bar{k}}_{s_1,s_2};\mathbf{\theta }),$$

$$\underset{{\theta }_e}{\mathrm{maximize}}\sum _{\omega \in \Omega }{w}_{\omega }(e)\text{log}{\pi }_e(t_{\omega }(e);{\theta }_e),$$

$$\begin{array}{rl}\forall \omega \in \Omega ,\forall e\in {\bar{k}}_{s_1,s_2},t_{\omega }(e)& =t(s_1,s_2)\frac{A_{\omega }(e)}{{\sum }_{e\in \mathcal{E}}{A}_{\omega }(e)}\\ \text{with}{A}_{\omega }(e)& \sim \Gamma ({\alpha }_e,\frac{{\beta }_e}{t(s_1,s_2)}),\end{array}$$

$$t(e)=t_{min}(e)\left(1+1.5\left({\frac{f(e)}{c(e)}}^4\right)\right),$$

$$c(e)=\{\begin{array}{ll}1700+10t_{min}(e)& \text{if }{t}_{min}(e)\le 70\text{leavevmode}mph,\\ 2400& \text{otherwise}.\end{array}$$

$$\underset{\hat{\mathbf{u}}}{\mathrm{minimize}}$$

$${\hat{\mathbf{f}}}_{\mathcal{E}}=\mathcal{M}(\hat{\mathbf{u}}),\hat{\mathbf{u}}\ge 0,$$

$$\bar{\mathbf{u}}=\hat{\mathbf{u}}+{ϵ}_1,{\bar{\mathbf{f}}}_{\mathcal{E}}={\hat{\mathbf{f}}}_{\mathcal{E}}+{ϵ}_2.$$

Full text

1. 1 INTRODUCTION

2. 1.1 City-Scale Traffic Reconstruction

3. 1.2 Autonomous Driving

4. 1.3 Main Results

5. 1.3.1 City-Scale Traffic Reconstruction

6. 1.3.1.1 Deterministic Estimation of Traffic Conditions

7. 1.3.1.2 Temporal Missing Data Completion

8. 1.3.1.3 Iterative Estimation and Spatial Missing Data Completion

9. 1.3.2 Autonomous Driving

10. 1.3.2.1 Theoretical Results

11. 1.3.2.2 Experimental Results

12. 1.4 Thesis Statement

13. 1.5 Organization

14. 2 CITYWIDE ESTIMATION OF TRAFFIC DYNAMICS VIA SPARSE GPS TRACES

15. 2.1 Introduction

16. 2.2 Related Work

17. 2.3 Traffic Velocity Field Reconstruction

18. 2.3.1 Notation and Definitions

19. 2.3.2 Velocity Field Estimation

20. 2.3.3 Travel Time Allocation

21. 2.3.4 Evaluation on A Synthetic Network Using Traffic Simulation

22. 2.4 Missing Value Completion

23. 2.5 Estimating Traffic Dynamics Via GPS Data

24. 2.6 Summary and Future Work

25. 3 CITY-SCALE TRAFFIC ANIMATION USING STATISTICAL LEARNING AND METAMODEL-BASED OPTIMIZATION

26. 3.1 Introduction

27. 3.2 Related Work

28. 3.3 Overview

29. 3.3.1 System

30. 3.3.2 Notation

31. 3.4 City-Scale Traffic Reconstruction

32. 3.4.1 Initial Estimation

33. 3.4.2 Iterative Estimation

34. 3.4.2.1 Map Matching

35. 3.4.2.2 Travel Time Estimation

36. 3.4.3 Bilevel Optimization

37. 3.5 Dynamic Data Completion

38. 3.5.1 Metamodel-Based Simulation Optimization

39. 3.5.2 Algorithmic Steps

40. 3.6 Results

41. 3.6.1 Evaluation of Initial Traffic Reconstruction

42. 3.6.1.1 Road Network and GPS Dataset

43. 3.6.1.2 Traffic Conditions via System Optimal Model

44. 3.6.1.3 Traffic Conditions via Timestamp Model

45. 3.6.1.4 Synthetic GPS Traces

46. 3.6.1.5 Evaluation and Comparison

47. 3.6.1.6 Analysis of Bilevel Optimization

48. 3.6.2 Evaluation of Dynamic Data Completion

49. 3.6.3 Traffic Visualization and Animation

50. 3.7 Summary and Future Work

51. 4 ADAPS: AUTONOMOUS DRIVING VIA PRINCIPLED SIMULATIONS

52. 4.1 Introduction

53. 4.2 Related Work

54. 4.3 Preliminaries

55. 4.4 ADAPS

56. 4.4.1 Theoretical Analysis

57. 4.4.2 Framework Pipeline

58. 4.5 Policy Learning

59. 4.5.1 End-to-end Imitation Learning

60. 4.5.2 Training Data Collection

61. 4.6 Learning from Accidents

62. 4.6.1 Solving Accidents

63. 4.6.2 Additional Data Coverage

64. 4.7 Experiments

65. 4.7.1 Experiment Setup

66. 4.7.1.1 Scenarios

67. 4.7.1.2 Vehicle Specifications

68. 4.7.1.3 Obstacles

69. 4.7.1.4 Training Data

70. 4.7.2 Evaluation Results

71. 4.7.2.1 On-road Scenarios

72. 4.7.2.2 Off-road Scenario

73. 4.7.2.3 Data Heterogeneity

74. 4.8 Summary and Future Work

75. 5 CONCLUSION

76. 5.1 Summary of Results

77. 5.2 Future Work

78. 5.2.1 Macroscopic Level

79. 5.2.2 Microscopic Level

80. 5.2.3 Connection Between The Two Levels

1. 1.1 The absolute errors in the recovered travel time computed using my technique vs. Lou et al. Lou et al., (2009) by using GPS traces from various percentages of the traffic population. My technique results in much smaller errors as of Lou et al. Lou et al., (2009).
2. 1.2 Test Results of On-Road Scenarios: my policies $O_{straight}$ & $O_{full}$ can lead to robust collision avoidance and lane-following behaviors.
3. 2.1 The absolute errors in the recovered travel times computed using my technique vs. Lou et al. Lou et al., (2009), using GPS traces from various percentages of the traffic population. My technique results in much smaller errors as of Lou et al. Lou et al., (2009).
4. 4.1 Training Data Summary: my method can achieve over 200 times more training examples than DAGGER Ross et al., (2011) at one iteration leading to large improvements of a policy.
5. 4.2 Test Results of On-Road Scenarios: my policies $O_{straight}$ & $O_{full}$ can lead to robust collision avoidance and lane-following behaviors.

\cftsetrmarg

1.0in

1. 1.1 Illustrations of procedures required to process GPS data for traffic estimation and reconstruction. LEFT (map-matching): off-the-road GPS points need to be mapped onto a road network. MIDDLE (map-matching): after determining the matching points, the traversed path of a vehicle needs to be inferred. RIGHT (travel-time estimation): after determining the traversed path, the timestamp difference needs to be distributed to individual road segments.
2. 1.2 Illustration of the sparsity issue embedded in GPS data. One-day GPS data of downtown San Francisco from the Cabspotting project Piorkowski et al., (2009) are plotted, marked regions showing the lack of data.
3. 1.3 Recovery of average travel time on different percentages of the traffic population using my approach (TOP) vs. Lou et al. Lou et al., (2009) (BOTTOM). My technique consistently outperforms Lou et al. Lou et al., (2009) in estimating the average travel time over 30 different congestion levels.
4. 1.4 Relative improvements measured in MSE of my technique over Lou et al. Lou et al., (2009) on travel time. My technique outperforms Lou et al. Lou et al., (2009) as the congestion level increases or as more GPS traces become available for the recovery task.
5. 1.5 The average speed measurements from loop-detector data are interpreted as a traffic signal, which exhibits a clear periodic pattern (TOP); the spectral analysis reveals that the most prominent frequency is one cycle per day (i.e., 24 hours) (MIDDLE); the traffic signal is approximated by a frequency-domain linear regression model in which 95% energy is retained by keeping the 10 largest frequencies (BOTTOM).
6. 1.6 The top panel shows the decaying rates of frequency magnitudes of all traffic signals; the bottom panel shows the locations and normalized magnitudes of the frequency components of all traffic signals. The rapid growth of decaying rates and randomly distributed frequency structures indicate that Compressed Sensing is applicable for recovering a traffic signal.
7. 1.7 A recovered traffic signal using my technique highly resembles its original form.
8. 1.8 My technique shows robustness when the number of samples used in recovering a traffic signal decreases.
9. 1.9 Estimated traffic pattern of downtown San Francisco (TOP) and its spectral analysis (BOTTOM). The result from my technique demonstrates a clear daily trend, which is consistent with the periodic feature observed in loop-detector data from the same area.
10. 1.10 Correlation between every pair of days in a week. The left panel lists normalized similarity scores calculated using the cosine distance, and the right panel provides the qualitative results. For both, the upper triangular matrix is derived using the estimated traffic conditions from GPS data, and the lower triangular matrix is computed using loop-detector data from the same area. When data and patterns are compared across the diagonal line, my estimated results exhibit high similarity to the loop-detector data.
11. 1.11 My algorithm on map-matching and travel-time estimation achieves consistent improvements over the previous techniques Rahmani et al., (2015); Hunter, (2014) in various measures.
12. 1.12 The error level of my technique vs. simulation-only approach: For a given road network and a specific origin-destination demand, I first compute the differences between the two methods with respect to the ground truth. Then, I subtract these two differences to obtain one error difference measure (indicated by a gray cross). The mean, minimum, and maximum values of the average error level (indicated by the solid line) are respectively 7.8%, 0%, and 13%. In many cases, my technique even outperforms the simulation-only approach with much smaller differences to the ground truth, shown by the negative values.
13. 1.13 The performance speedup of my technique over the simulation-only approach: my technique is on average about $27.2x$ faster, with maximum and minimum performance gains of $35.6x$ and $16.1x$. The maximum observed performance gain of a single speedup measure is over $90x$ (at origin-destination demand = 9000).
14. 1.14 2D traffic animation of regions in San Francisco: Northeast (top left), Central-East (top center), Central (top Right), Northwest (bottom). I have exaggerated the headlights and adopted an evening time period (i.e., Friday 7PM) to make vehicles more visible.
15. 1.15 3D traffic animation: a perspective overview (left), a topdown view (center), and a driver’s view (right).
16. 1.16 Visualization of traffic patterns in San Francisco and Beijing. Four time periods of a week, namely Sunday 9AM, Tuesday 9AM, Thursday Noon, and Friday 7PM, are selected to illustrate weekend vs. weekday and morning vs. evening traffic. The traffic is measured by Volume Of Capacity (VOC). All computations are conducted in epoch time.
17. 1.17 The estimated traffic conditions measured in average volume over capacity (VOC) of San Francisco and Beijing for various types of roads. All computations are conducted in epoch time. My technique successfully recovers the periodic phenomena in all cases. In San Francisco, saddle shapes appear on motorway and truck and overall roads for several days indicating mid-day traffic relief. Such phenomena are not observed in Beijing, which suggests the similar usage of different types of roads and the congestion forming throughout daytime.
18. 1.18 LEFT and CENTER: the comparisons between my policy $O_{full}$ (TOP) and Bojarski et al. Bojarski et al., (2016), $B_{full}$ (BOTTOM). $B_{full}$ causes collision while $O_{full}$ steers the AV away from the obstacle. RIGHT: the accident analysis results on the open ground. I show the accident caused by an adversary vehicle (TOP); then I show, after additional training, the AV can avoid the adversary vehicle (BOTTOM).
19. 1.19 The visualization results of collected images using t-SNE Maaten and Hinton, (2008). My method generates more heterogeneous training data compared to DAGGER Ross et al., (2011) in one learning iteration.
20. 2.1 Pipeline of my framework. Map Matching and Travel Time Allocation are applied on individual time intervals, while Missing Value Completion is performed over all time intervals.
21. 2.2 An example illustrating a failure using the shortest distance criterion for map-matching trace $S$ to $T1$, $T2$, or $T3$. LEFT: By matching travel time of trace $S$ and road conditions, the correct path $T1$ is identified. RIGHT: By only considering the shortest distance path, $S$ is mismatched to $T3$.
22. 2.3 An illustration of the relaxation process of two traces $S1$ and $S2$. The actual traffic conditions and trace matching results are shown in the TOP-LEFT panel. The input listed in the TOP-RIGHT panel contains the initial network and trace information (i.e., sources, targets, and travel time). After relaxation of trace $S1$, paths $T1$ and $T3$ have their travel times increased to $S1.t$. While $S1$ can be matched to either $T1$ or $T3$, the situation is resolved after relaxation of $S2$ due to further increase in the travel time of the road segment $e$.
23. 2.4 Recovery of average travel time on different percentages of the traffic population using my approach (TOP) vs. Lou et al. Lou et al., (2009) (which uses the shortest-distance criterion) (BOTTOM). My technique consistently outperforms Lou et al. Lou et al., (2009) in estimating the average travel time over 30 different congestion levels.
24. 2.5 Relative improvements measured in MSE of my technique over Lou et al. Lou et al., (2009). My technique outperforms Lou et al. Lou et al., (2009) as the congestion level increases or more GPS traces become available in estimation.
25. 2.6 The average speed measurements from a loop detector are interpreted as a traffic signal, which exhibits a clear periodic pattern (TOP); the spectral analysis reveals that the most prominent frequency is one cycle per day (MIDDLE); the traffic signal is approximated by a frequency-domain linear regression model, in which 95% energy is retained by keeping the 10 largest frequencies (BOTTOM).
26. 2.7 The top panel shows the decaying rates of frequency magnitudes of all traffic signals; the bottom panel shows the locations and normalized magnitudes of the frequency components of all traffic signals. The rapid growth of decaying rates and randomly distributed frequency structures indicate that Compressed Sensing is applicable for recovering a traffic signal.
27. 2.8 Solution elements (in total 168) by solving an underdetermined system via convex optimization. Both the actual and the $\mathcal{L}_{1}$-norm based recovery demonstrate sparsity (i.e., most solution elements are approximately zero).
28. 2.9 Recovery of a traffic signal via Compressed Sensing. The actual signal and its 90 random measurements are plotted (TOP). The $\mathcal{L}_{1}$-norm based recovery (BOTTOM) shows high similarity to the actual signal.
29. 2.10 Errors between recovered and actual traffic signals. As more samples are used in signal’s recovery, smaller errors are observed. This analysis also demonstrates the robustness of my approach as the error only increases linearly when the number of samples used in recovery decreases.
30. 2.11 Estimated traffic pattern of downtown San Francisco using the Cabspotting dataset Piorkowski et al., (2009) (TOP) and its corresponding spectral analysis (BOTTOM). The recovery from my technique shows a clear daily trend, which is consistent with the features observed in loop-detector data from the same area.
31. 2.12 Correlation between every pair of days in a week. The left panel lists the normalized similarity scores calculated using the cosine distance, and the right panel provides the qualitative results. In both, the upper triangular matrix is derived using the estimated traffic conditions, and the lower triangular matrix is computed using loop-detector data from the same area. When data and patterns are compared across the diagonal line, my estimated results exhibit high similarity to the loop-detector data.
32. 3.1 The systematic view of my framework. Trip records are optional as they can be inferred from GPS traces on a digital map.
33. 3.2 LEFT: Sample GPS points from the Cabspotting dataset. MIDDLE: Road maps of downtown San Francisco overlaid with traffic analysis zones (TAZs). RIGHT: A heuristic traffic condition established via the Timestamp model (the travel times are converted to flows).
34. 3.3 The relationship between the normalized convergence rate and the number of iterations of the outer loop of my iterative process is shown. The convergence rate decreases quadratically as the number of iterations increases and tends to flat after 10 iterations.
35. 3.4 From TOP to BOTTOM, the left diagrams show the results generated using network travel times via the system optimal (SO) model, while the right diagrams show corresponding results of network travel times via the Timestamp model. TOP: The error rates (%) of various methods of aggregating travel time across the network. MIDDLE: The relative improvements (%) of travel times of all road segments measured in MSE. BOTTOM: The relative improvements (%) of map-matching accuracy measured using successfully identification rates of road segments. In summary, my technique achieves consistent improvements over the other two methods as the number of GPS traces used in recovery increases.
36. 3.5 TOP and MIDDLE: The normalized noisy levels (%) of target road-segment flows $\bar{\mathbf{f}}$ computed according to the MSE of estimated network travel times to ground truth. In general, for both models, my technique produces lower noisy levels than the other two techniques. BOTTOM: The normalized MSE of target OD pairs $\hat{\mathbf{u}}$ (%) under different values of the weighting factor $\eta$ and various noisy levels of $\bar{\mathbf{f}}$ (%). When $\eta$ is small, the error is more sensitive to perturbations on $\bar{\mathbf{f}}$. Overall, the error increases as the noisy level of $\bar{\mathbf{f}}$ increases. For all studies, the normalized noisy level of target OD pairs $\bar{\mathbf{u}}$ has been set to $50\%$. My method has achieved consistently lower error rates compared to the other two methods.
37. 3.6 The error level of my technique vs. simulation-only approach: For a given road network and a specific OD demand, I first compute the differences between the two methods with respect to the ground truth. Then, I subtract these two differences to obtain one error difference measure (indicated by a gray cross). The mean, minimum, and maximum values of the average error level (indicated by the solid line) are respectively 7.8%, 0%, and 13%. In many cases, my technique even outperforms the simulation-only approach with a smaller difference to the ground truth, as indicated by a negative value in the diagram.
38. 3.7 The performance speedup of my technique over the simulation-only approach: my technique is on average about $27.2x$ faster, with maximum and minimum performance gains of $35.6x$ and $16.1x$. The maximum observed performance gain of a single speedup measure is over 90x (at OD demand = 9000).
39. 3.8 (a) TOP: several loop-detector signals are plotted showing phase shifts among them; BOTTOM: the average signal of the signals shown in the top panel. (b) TOP: aligned loop-detector signals according to their phase responses; BOTTOM: the average signal of the aligned signals shown in the TOP panel. (c) TOP: the averaged frequency of the signals in (a) TOP, which shows several frequency component are getting degraded; MIDDLE: the frequency of the signal in (a) BOTTOM, which shows inconsistent magnitude ratios; BOTTOM: the frequency of signal in (b) BOTTOM, which shows prominent frequencies and magnitude ratios.
40. 3.9 Qualitative (visualization) and quantitative analysis of traffic in San Francisco. Top: four time periods, namely Sunday 9AM, Tuesday 9AM, Thursday Noon, and Friday 7PM, are adopted to illustrate weekend vs. weekday and morning vs. evening traffic. The traffic is measured by volume over capacity (VOC) (computed as $\sum_{e\in\mathcal{E}}\left(f(e)/c(e)\right)$ where $c(e)$ is the capacity of the road segment $e$ defined in Equation 3.11). Bottom: I compare my reconstructed results using GPS data (after the filtering process explained in this section) to the data from three loop detectors in San Francisco. The results show small losses (around $1\leavevmode\nobreak\ m/s$) in the speed measurement.
41. 3.10 2D traffic animation of regions in San Francisco: Northeast (top left), Central-East (top center), Central (top Right), Northwest (bottom). I have exaggerated the headlights and adopted an evening time period (i.e., Friday 7PM) to make vehicles more visible.
42. 3.11 3D traffic animation: a perspective overview (left), a topdown view (center), and a driver’s view (right).
43. 4.1 Collision-free trajectories generated by the expert algorithm for a vehicle traveling on the right lane of a straight road, with an obstacle in front: in total, 74 trajectories span from the first moment the vehicle perceives the obstacle (green, progressive avoidance) to the last moment the collision can be avoided (red, sharp avoidance).
44. 4.2 Illustration of important points and DANGER/SAFE labels from Section 4.6 for a vehicle traveling on the right lane of a straight road, with an obstacle in front. Labels are shown for four points $\{\mathbf{l}_{u1},\mathbf{l}_{u2},\mathbf{l}_{u3},\mathbf{l}_{u4}\}$ illustrating the four possible cases.
45. 4.3 LEFT and CENTER: the comparisons between my policy $O_{full}$ (TOP) and Bojarski et al. Bojarski et al., (2016), $B_{full}$ (BOTTOM). $O_{full}$ can steer the AV away from the obstacle while $B_{full}$ causes collision. RIGHT: the accident analysis results on the open ground. I show the accident caused by an adversary vehicle (TOP); then I show after additional training the AV can avoid the adversary vehicle (BOTTOM).
46. 4.4 The visualization results of collected images using t-SNE Maaten and Hinton, (2008). My method can generate more heterogeneous training data compared to DAGGER Ross et al., (2011) in one learning iteration as the sampled trajectories progress.

CHAPITRE 1:  INTRODUCTION

Throughout history, human mobility not only characterizes our way of life but also plays an essential role in socio-economic development. In the past century, as a result of rapid motorization and urbanization, increasing human-mobility patterns are stemmed from automobiles and are appearing in urban environments. While this phenomenon reflects better accessibility to societal resources and an increase in quality of life, the resulting traffic congestion has become one of the most infamous problems across the globe. The annual costs due to traffic gridlock and jams are nearly $160 billion in the U.S., £13 billion in the U.K., and exceeding one trillion U.S. dollars worldwide.

As urbanization, motorization, and vehicle production rates keep climbing—especially in Asia and Africa—we will witness many more metropolitan areas forming, growing, and experiencing severe traffic conditions. The ability to understand and improve traffic dynamics is thus critical more today than ever.

Traffic can be studied at various scales. Traffic is an aggregate phenomenon, which can appear at a large spatial-temporal scale. Study of its dynamics requires a holistic view. Among various traffic engineering tasks, one crucial task is the efficient and accurate estimation and reconstruction of city-scale traffic. This task can enable many Intelligent Transportation System (ITS) applications such as identifying congestion causes, detecting network bottlenecks, planning traffic flows, and analyzing transport policies.

Additionally, traffic is formed by individual vehicles propagating through space and time. If we can improve the efficiency and coordination of them, jointly, we have the opportunity to alleviate severe traffic conditions in metropolitan areas. While the behaviors of human-driven vehicles can be only altered through regulations and policies, the behaviors of autonomous vehicles can be precisely directed and optimized via control algorithms. Thus, the switch from human-driven vehicles to autonomous vehicles has the potential to revolutionize our transportation systems by reducing the number of crashes and alleviating traffic jams which both, by a large degree, attribute to human factors.

This dissertation summarizes my early efforts towards understanding and improving traffic conditions via studying traffic at both the macroscopic level and the microscopic level. Macroscopically, I study city-scale traffic reconstruction: I have combined physics-based simulation for modeling aggregate traffic behaviors and machine learning techniques for distilling travel patterns from a large volume of traffic data in order to estimate and reconstruct traffic at a metropolitan scale. Microscopically, I study autonomous driving: I have developed simulation platforms and an online learning mechanism for effectively and efficiently learning and testing control policies for autonomous driving.

City-Scale Traffic Reconstruction

In order to analyze congestion causes, identify network bottlenecks, and experiment with novel transport policies, we need to be able to reconstruct and simulate city-scale traffic at the macroscopic level. I have developed methods to efficiently and accurately estimate and reconstruct large-scale traffic using mobile-sensor data while generating visual analytics in various forms.

Estimation and reconstruction of large-scale traffic is difficult due to fundamental challenges: 1) traffic dynamics is intrinsically stochastic as a result of individual drivers’ behaviors, and 2) the availability and quality of traffic data are usually limited. Conventionally, traffic data are collected via in-road sensors such as loop detectors and video cameras. While these sensors produce accurate measurements, they are mostly installed on highways and major roads, which only constitute a small portion of a city. Mobile-sensor data, such as GPS reports, are more promising sources for the estimation and reconstruction task due to their broader coverage. However, GPS points usually embed a low-sampling rate, meaning that the time lapse between two consecutive reports is large (e.g., greater than 60 seconds), and exhibit spatial-temporal sparsity, meaning that the data are scarce in certain areas and time periods.

In order to adopt GPS data for reconstructing full traffic dynamics, several procedures are required to address the abovementioned features: 1) map-matching, which maps off-the-road GPS points (due to inevitable measurement noise) onto a road network and infers the traversed path of a vehicle (illustrated in Figure 1.1 LEFT and MIDDLE); 2) travel-time estimation, which estimates the travel time of a road network through GPS timestamps (illustrated in Figure 1.1 RIGHT); 3) missing-data completion, which interpolates spatial-temporal missing measurements (illustrated in Figure 1.2).

Many state-of-the-art map-matching approaches use the shortest-distance criterion Lou et al., (2009); Quddus and Washington, (2015), which treats the shortest-distance path between two mapped GPS points as the traversed path. This assumption, however, can lead to a considerable bias in a congested network where the shortest-distance path potentially differs from the shortest travel-time path between two GPS points, while the latter is preferred by GPS devices and experienced drivers. If we adopt a sequential pipeline to process GPS data, travel-time estimation (the subsequent step of map-matching) will produce inaccurate results, since the timestamp difference between GPS reports will be distributed to a wrong set of road segments.

GPS data are usually scarce in certain time periods, such as early-morning hours, and certain areas, such as suburbs (illustrated in Figure 1.2). We need to interpolate these spatial-temporal missing traffic data in order to gain a holistic view of a city’s traffic dynamics. In addition, in order to account for the dynamic nature of traffic, we need to develop methods that can take both local interactions among cars and global propagation of traffic into account. In theory, the spatial missing data can be approximated using traffic simulation. However, we have to ensure the flow consistency on the boundaries that separate areas with GPS data coverage and areas without GPS data coverage in order to accept a particular simulation result. This requires us to be able to dynamically adjust traffic simulation for matching traffic flows.

Chapter 2 and Chapter 3 are dedicated to address the limitations of GPS data and explain how to accurately and efficiently estimate traffic conditions, interpret spatial-temporal missing traffic data, and dynamically reconstruct traffic flows at a city scale.

Autonomous Driving

While traffic dynamics can only be studied collectively, it is also formed by individual vehicles. Given over 90% crashes are due to human errors, the conversion from human-driven vehicles to autonomous vehicles (AVs)—with improved safety and efficiency features—has the potential to alleviate the severe traffic conditions Wu et al., (2018).

Widespread research on autonomous driving has been conducted as a result of recent advancements in automation and machine learning algorithms. Before the deployment of AVs, we need to scrutinizingly test and improve their safety, control, and coordination. I have developed a framework, named ADAPS (Autonomous Driving Via Principled Simulations), which not only can be used to simulate, analyze, and produce driving data in various scenarios including accidents, but also represent a more efficient online learning mechanism for learning robust control policies for autonomous driving Li et al., (2019).

Learning to drive is a sophisticated task considering the complexity and dynamics of an environment and the lack of precise descriptions of a desired driving behavior. These features pose challenges to conventional planning and control methods, since they usually require us to specify the dynamics of the world—which can be used to model the cost of an action at each time step and consequences of an action in the future Ratliff, (2009); Silver, (2010). Describing and parameterizing an uncontrolled and unpredictable driving environment is a daunting and often impractical task, thus preventing the conventional techniques from succeeding.

Nevertheless, driving, as a complex task, can be easily demonstrated by humans. The intricate traffic flow manifests such success. This phenomenon has inspired imitation learning—a data-driven machine learning technique that leverages expert demonstrations to synthesize a controller for achieving a desired behavior. A number of studies have shown that imitation learning is an effective approach capable of learning policies for a juggling robot Atkeson, (1994), a quadruped robot maneuvering on rough terrains Ratliff et al., (2007), and autonomous driving on highway Pomerleau, (1989), among many others.

A straightforward way to achieve imitation learning is via standard supervised learning: learning a training dataset that is a collection of observations and their corresponding behaviors. In the context of driving, this training dataset could be images from a front-facing camera and the steering angle applied while each image was taken. The effectiveness of supervised learning is commonly shown by running a learned model on a test dataset and analyzing the results.

Supervised learning operates based on one assumption—the examples in both training and test datasets are independently and identically distributed (i.i.d.) Friedman et al., (2001). While this assumption is reasonable for many problems, it fails to apply on driving, in which task the training and test examples are not i.i.d. Ross et al., (2011).

To be specific, autonomous driving is a sequential prediction and controlled (SPC) task, meaning the system has to predict a sequence of control commands over time to achieve a goal. Given the sequential nature, in such a system, any predicted control commands will affect the following observations being taken by the system and, consequently affect the future predicted control commands (since the predictions are drawn based on the observations). Because the predictor and the observer are entangled, their resulting training and test examples violate the i.i.d. assumption.

Although we can still use standard supervised learning to achieve small training errors, the problem occurs during the test phase. In any learning process, approximation is inevitable, which, when combined with the entanglement of the predictor and the observer, can lead a system to large test errors: a small disturbance from either the predictor or the observer is likely to lead to compounding errors. To give an example, in driving, a small disturbance to the sensor or the control module can result in a vehicle encountering an “unseen” observation (i.e., an observation not appearing in the training dataset). Consequently, this observation will confuse the control module, causing an unpredictable maneuver of the vehicle and lead the vehicle to more “unseen” observations. This mismatch between the observations used for training (i.e., training distribution) and the observations encountered during testing (i.e., test distribution) is termed covariate shift Sugiyama and Kawanabe, (2012).

In essence, a driving policy from supervised learning by treating all training examples as i.i.d. will not be robust to its own mistakes and likely lead a vehicle to dangerous situations including accidents. To alleviate this issue, first, we need driving data from both safe and, especially, dangerous situations. This is critical because most expert demonstrations (i.e., human driving data) are from safe driving. However, if a learning algorithm is not trained on recovery demonstrations in dangerous situations, when encountering, the algorithm’s behavior is undefined. Second, we need an efficient online learning mechanism. For solving an SPC task, interactions with experts in a number of iterations are essential, without which, no learning algorithm can ensure robust performance Ross et al., (2011). Since learning an effective policy for an SPC task requires iterative testing and update, an efficient online learning technique that can reduce the number of learning iterations is desired. For autonomous driving, this becomes critical given that one iteration implies one incident (otherwise there is no need to update a policy by proceeding to the next iteration). Third, we need an effective architecture for the control policy. In the simplest case, a policy needs to classify road conditions and make corresponding maneuvers. The two basic road conditions are “safe”, e.g., no obstacle on the road—the vehicle can just follow the road, and “dangerous”, e.g., an obstacle appearing on the road—the vehicle needs to avoid it. Next, I will briefly explain and introduce my solutions to these challenges.

Obtaining recovery data of vehicles from dangerous situations in the physical world is impractical, due to the high expenditure of a vehicle and potential injuries to people both inside and outside a vehicle. In addition, even if such recovery data are collected, human experts are usually required to label them. This process could be inefficient and subject to judgmental errors Ross et al., (2013). These difficulties suggest the potential use of the virtual world, in which we can simulate various dangerous scenarios including accidents and then analyze the simulated scenarios to generate recovery data. I have developed a simulation platform for this purpose.

In order to develop an efficient online learning technique, my solution is to treat the principled simulation as the “expert”. This “expert” will plan alternative safe trajectories during the analysis of a simulated accident while taking the kinematic and dynamic constraints of a vehicle into account. As a result, not only can the number of “expert” trajectories be generated indefinitely, but the access to such an “expert” is instantaneous. These features will assist in reducing the number of learning iterations.

Lastly, I propose a hierarchical control policy using deep neural networks LeCun et al., (2015) and long short-term memory networks Hochreiter and Schmidhuber, (1997). My policy consists of three modules, the detection module, which monitors a road condition and categorizes it as “safe” or “dangerous”, the following module, which directs a vehicle to follow the road if the condition is considered “safe”, and the avoidance module, which steers a vehicle away from an obstacle when the road condition is considered “dangerous”. Chapter 4 is dedicated to explain the abovementioned solutions in details.

Although this dissertation addresses the macroscopic level and the microscopic level of traffic separately, the two levels are tightly coupled. Many applications can stem from their rich connection. For example, from macroscopic to microscopic, the reconstructed city-scale traffic can assist route planning of autonomous driving and enrich virtual environments for training purposes Chao et al., (2019); from microscopic to macroscopic, individual autonomous vehicles can be adopted to stabilize and regulate traffic flows Wu et al., (2018). These applications will be further discussed in Chapter 5. In the following, I will introduce the main results regarding city-scale traffic reconstruction and autonomous driving.

Main Results

City-Scale Traffic Reconstruction

1.3.1.1 Deterministic Estimation of Traffic Conditions

My first solution to the estimation of traffic conditions is an efficient deterministic approach Li et al., 2017a , for which details can be found in Chapter 2. My solution is based on two observations: traffic patterns exhibit weekly periodicity and traffic conditions are quasi-static. Using these observations, I treat one week as a traffic period and assume that traffic conditions are static within each hourly time interval of a weekly period. Then, in each time interval, I conduct map-matching by replacing the shortest distance criterion with the shortest travel-time criterion, since the former criterion can lead to mapping errors in a congested network.

In order to compute the shortest travel-time path between any two nodes in a network, we need the travel time information of the network. However, the initial condition of a road network usually lacks such information. My approach to alleviating this issue is to adapt a travel-time allocation method from Hellinga et al. Hellinga et al., (2008), which was developed based on empirical observations of the real-world traffic dynamics. Using this technique, the subsequent map-matching process can take the intermediate estimated traffic conditions into account, which helps improving the overall map-matching accuracy.

I have evaluated the effectiveness of my solution through comparison with a state-of-the-art technique from Lou et al. Lou et al., (2009) on a synthetic road network. I have established 30 traffic conditions corresponding to 30 congestion levels as the ground truth. For each congestion level, I test my algorithm by sampling different portions of the simulated traffic population, namely at levels 20%, 40%, 60%, 80%, and 100%. In principle, the more GPS traces are used in traffic reconstruction, the more accurate are the reconstruction results.

The first analysis, shown in Figure 1.3, is conducted using the aggregate travel time of the road network under each congestion level as the true travel time. Starting with 20% GPS traces, my technique shows close approximations to the true travel time at all congestion levels, while the shortest-distance based technique fails to achieve the same performance. Table 1.1 shows the “absolute error to ground truth” computed from all congestion levels.

The second analysis, shown in Figure 1.4, summarizes the relative improvements measured in mean squared error (MSE) of my method over Lou et al. Lou et al., (2009). As the congestion level increases or more GPS traces are used in recovery, my technique outperforms the other technique. In comparison, the improvements are less salient when the congestion level is low, (i.e., the first 10 congestion levels out of 30), but more salient when the congestion level is high (i.e., the rest 20 congestion levels out of 30).

1.3.1.2 Temporal Missing Data Completion

Traffic data can be scarce for certain time periods such as early-morning or late-night hours. In order to interpolate the data for these time periods, I exploit the fact that the traffic pattern is intrinsically periodic (as a result of periodic human behaviors). It thus has a sparse representation in the frequency domain. Based on this observation, I have developed a technique based on Compressed-Sensing Donoho, (2006); Candes et al., (2006) to fill in missing travel time information and robustly recover the traffic pattern of a road segment over an entire traffic period Li et al., 2017a . The details of this approach can be found in Chapter 2.

In order to test the effectiveness of my approach, I have adopted speed measurements from 38 loop detectors in the city of San Francisco. These data represent relatively complete and accurate measurements of traffic. The hourly average speed measurements of a single loop detector for a weekly period is termed the traffic signal (see an example in the Figure 1.5 top panel). To analyze each traffic signal, I have performed a spectral analysis, which results are shown in Figure 1.5. We can see from the middle panel that the most salient oscillation appears at 24 hours and from the bottom panel that most signal energy can be captured by keeping the 10 largest frequencies of the signal. These indicate that a traffic signal indeed has a sparse representation in the frequency domain, which is emphasized by the analysis of the decaying rates of frequency magnitude presented in the top panel of Figure 1.6.

According to the Nyquist-Shannon Theorem, we need at least 168 measurements to fully recover a traffic signal, which are lacking as a result of the temporal sparsity of GPS data. However, Compressed Sensing promises that if a signal has a sparse representation and randomly distributed frequency components, we can have a accurate recovery of the signal. Those features do appear in a traffic signal as shown in Figure 1.6. This indicates that we can recover a traffic signal using Compressed Sensing. An example is shown in Figure 1.7. The robustness analysis of my approach can be found in Figure 1.8.

By confirming the applicability of Compressed Sensing algorithm to recovering traffic signals, we can proceed to apply it to real-world GPS datasets. To demonstrate that my approach can recover features of a traffic signal, I have adopted the metric $\text{\emph{fluidity}}\in[0,1]$ Hofleitner et al., 2012c , computed as the ratio of the estimated travel speed to the free-flow speed of a road segment. In Figure 1.9, I show the recovered traffic pattern using actual GPS data in San Francisco, which shows clear periodicity at one cycle per 24 hours.

The similarity among different days in a week can also be used to show the effectiveness of a signal recovering method. I have computed the cosine distance between every pair of days for both the estimated traffic conditions and the actual traffic conditions derived from the loop-detector data from San Francisco. The results are shown in Figure 1.10. The left panel shows the distance scores while the right panel shows the qualitative results: the upper triangle indicates the estimated quantities and the lower triangle indicates the actual quantities. The symmetrical pattern illustrates that my technique can produce accurate results compared to the ground-truth values.

1.3.1.3 Iterative Estimation and Spatial Missing Data Completion

While the deterministic approach introduced in the above section for estimating traffic conditions is efficient and effective, in order to further improve the accuracy for interpolating spatial missing data, I have developed an iterative algorithm that embeds map-matching and travel-time estimation as its sub-routines Li et al., (2018).

To be specific, in areas with GPS data coverage, I first conduct a coarse inference of travel times at each hourly time interval by solving a convex optimization program inspired by Wardrop’s Principles Wardrop, (1952); Sheffi, (1985). Then, I refine the inferred results via executing map-matching and travel-time estimation iteratively. Next, I proceed to establish baseline estimation of traffic conditions in areas without GPS data coverage using a nested optimization procedure Yang et al., (1992) to ensure that certain traffic flow characteristics are met. The details can be found in Chapter 3.

I have evaluated my approach using a real-world road network, which contains 5407 nodes and 1612 road segments. Additionally, 34 ground-truth travel times are established and over 10 million synthetic GPS traces are sampled based on the established heuristic travel times. I have compared my approach with two state-of-the-art methods, Hunter et al. Hunter, (2014) and Rahmani et al. Rahmani et al., (2015). The results show that my algorithm offers the lowest error rate and up to 97% relative improvement in estimation accuracy (see Figure 1.11).

Currently, most existing methods interpolate spatial missing measurements statically Hellinga et al., (2008); Rahmani et al., (2015); Herring et al., (2010); Hunter, (2014); Tang et al., 2016b ; Hunter, (2014); Li et al., 2017a . In order to account for the dynamic nature of traffic, I have leveraged traffic simulation for the interpolation task and developed an algorithm to guarantee the consistency of traffic flows on the boundaries of areas with and without GPS data Li et al., 2017b .

An effective way to dynamically interpolate spatial missing data is to run traffic simulation in the “empty” areas. However, arbitrary simulations will not respect the boundary conditions from the previously estimated traffic conditions in data-rich areas. This indicates we need to fine tune the simulation with the objective of enforcing minimal discrepancy between simulated traffic flows and previously estimated traffic flows. For this goal, I allow the simulation algorithm to alter “turning ratios” at intersections—parameters that determine how traffic will distribute itself to downstream road segments at an intersection. Using this design perspective, the objective then becomes deriving the optimal “turning ratios” such that the simulated traffic flows will respect the estimated traffic flows (from GPS data) at the boundaries that join data-deficient and data-rich regions.

The altered objective leads me to use simulation-based optimization for finding the optimal solution. However, as our goal is to reconstruct city-scale traffic, an optimization program at that scale could be computationally cost prohibitive. To remedy this issue, I have adopted a metamodel-based simulation optimization Osorio et al., (2015). Compared to a stochastic microscopic traffic simulator, the metamodel is a deterministic function, thus is much more tractable and computationally efficient when running within an optimization routine. In order to make sure that the metamodel behaves similarly to the simulator, the metamodel is trained using simulations. This way we only need to run traffic simulation dozens of times (for training the metamodel) instead of hundreds or even thousands of times (for running simulation-based optimization).

I have compared my technique to a baseline approach that only uses a simulator (i.e., simulation-only approach). I set up the experiments using various origin-destination demands on the road network of San Francisco to establish ground-truth traffic conditions. Tested on these conditions, my approach maintains on average a 7% error rate while achieving up to 90 times speedup compared to the simulation-only approach. These statistics can be seen in Figure 1.12 and Figure 1.13, respectively.

After interpolating spatial missing data, I have fully reconstructed spatial-temporal traffic at the scale of a city. The reconstructed traffic can be visualized in many ways such as 2D flow map, 2D animation, and 3D animations. Some examples are shown in Figure 1.14, Figure 1.15, Figure 1.16, and Figure 1.17. These visual representations can be used to enable 1) analysis of traffic patterns at street level, region level, and the city level, and 2) virtual environment applications such as virtual tourism and the training of autonomous vehicles.

Autonomous Driving

I have developed the framework ADAPS for learning and testing the control policy of autonomous driving. ADAPS consists of two simulation platforms and a hierarchical control policy. The first simulation platform resides in a 3D virtual environment and is utilized to test a learned policy, simulate accidents, and generate labelled data. The second simulation platform operates in a 2D environment and serves as an “expert” to analyze and resolve an accident via planning alternative safe trajectories for a vehicle by considering its kinematic and dynamic constraints.

In addition, ADAPS represents a more efficient online learning mechanism compared to existing techniques such as DAGGER Ross et al., (2011). The reason for the improvement is due to a switch from a “reset modeling” approach, in which we can only sample observations by putting an agent in its initial state distribution, to “generative modeling” approach, in which we can put an agent in an arbitrary state and sample observations by executing any action in that state. The reason that I can make this switch is because of two reasons: 1) the generation of training examples is from simulations that take vehicle kinematic and dynamic constraints into account, rather than merely executing a policy; 2) the assumption that we have access to all agent states in a simulation is because the simulation is conducted in retrospect of an accident, thus viable.

In order to understand the theoretical results, I will briefly introduce the notation and definitions used in the analysis. The problem we consider is a $T$-step control task. Given the observation $\phi=\phi(s)$ of a state $s$ at each step $t\in[\![1,T]\!]$, the goal of a learner is to find a policy $\pi\in\Pi$ such that its produced action $a=\pi(\phi)$ will lead to the minimal cost:

[TABLE]

where $C\left(s,a\right)$ is the expected immediate cost of performing $a$ in $s$. For many tasks such as driving, we may not know the true value of $C$. Instead, the observed surrogate loss $l(\phi,\pi,a^{*})$ is commonly minimized. This loss is assumed to upper bound $C$, based on the approximation of the learner’s action $a=\pi(\phi)$ to the expert’s action $a^{*}=\pi^{*}(\phi)$. I denote the distribution of observations at $t$ as $d_{\pi}^{t}$, which is the result of executing $\pi$ from timestep $1$ to $t-1$. Consequently, $d_{\pi}=\frac{1}{T}\sum_{t=1}^{T}d_{\pi}^{t}$ is the average distribution of observations by executing $\pi$ for $T$ steps. The goal of solving an SPC task is to obtain $\hat{\pi}$—a policy that can minimize the observed surrogate loss under its own induced observations with respect to an expert’s actions for those observations:

[TABLE]

I further denote $\epsilon=\mathbb{E}_{\phi\sim d_{\pi^{*}},a^{*}\sim\pi^{*}(\phi)}\left[l\left(\phi,\pi,a^{*}\right)\right]$ as the expected loss under the training distribution induced by the expert’s policy $\pi^{*}$, and the cost-to-go over $T$ steps of $\hat{\pi}$ as $J\left(\hat{\pi}\right)$ and of $\pi^{*}$ as $J\left(\pi^{*}\right)$.

By simply treating expert demonstrations as i.i.d. samples, the discrepancy between $J\left(\hat{\pi}\right)$ and $J\left(\pi^{*}\right)$ is $\mathcal{O}(T^{2}\epsilon)$ Syed and Schapire, (2010); Ross et al., (2011). Given the error of a typical supervised learning is $\mathcal{O}\left(T\epsilon\right)$, this demonstrates the additional cost due to covariate shift when solving an SPC task via standard supervised learning.

DAGGER Ross et al., (2011) has been used to solve an SPC task by keeping the $\mathcal{O}\left(T\epsilon\right)$ error. To illustrate its result, I introduce more definitions: the best policy at the $i$th iteration (trained using all observations from the previous $i-1$ iterations) is denoted as $\pi_{i}$; for any policy $\pi\in\Pi$, we have its expected loss under the observation distribution induced by $\pi_{i}$ as $l_{i}\left(\pi\right)=\mathbb{E}_{\phi\sim d_{\pi_{i}},a^{*}\sim\pi^{*}(\phi)}\left[l_{i}\left(\phi,\pi,a^{*}\right)\right],l_{i}\in\left[0,l_{max}\right]$; the minimal loss in hindsight after $N\geq i$ iterations is denoted as $\epsilon_{min}=\min_{\pi\in\Pi}\frac{1}{N}\sum_{i=1}^{N}l_{i}(\pi)$ (i.e., the training loss after $N$ iterations); the average regret is $\epsilon_{regret}=\frac{1}{N}\sum_{i=1}^{N}l_{i}(\pi_{i})-\epsilon_{min}$. Then, the accumulated error after $T$-step using DAGGER Ross et al., (2011) is bounded by the summation of three terms:

[TABLE]

where $f\left(\cdot\right)$ is the function of fixed $T$ and $l_{max}$. The second term tends to [math] if a no-regret algorithm is used Hazan et al., (2007). The third term tends to [math] if $N\rightarrow\infty$.

1.3.2.1 Theoretical Results

With the above-introduced background, I can introduce the following theoretical results.

Theorem 1.1**.**

If the surrogate loss $l$ upper bounds the true cost $C$, by collecting $K$ trajectories using ADAPS at each iteration, with probability at least $1-\mu$, $\mu\in(0,1)$, ADAPS offers the following guarantee:

[TABLE]

This theorem bounds the expected cost-to-go of the best policy $\hat{\pi}$ based on the empirical loss of the best policy in $\Pi$ (i.e., $\hat{\epsilon}_{min}$) and the empirical average regret of the learner (i.e., $\hat{\epsilon}_{regret}$). Similar to DAGGER Ross et al., (2011), the second term can be eliminated if a no-regret algorithm such as Follow-the-Leader Hazan et al., (2007) is used. The third term implies the number of training examples $KN$ needs to be $\mathcal{O}\left(T^{2}l^{2}_{max}\log{\frac{1}{\mu}}\right)$ in order to become negligible. We can achieve $\mathcal{O}\left(T^{2}l^{2}_{max}\log{\frac{1}{\mu}}\right)$ samples easily as ADAPS uses principled simulations for generating them. With these changes, this theorem can lead to the following Corollary.

Corollary 1.

If $l$ is convex in $\pi$ for any $s$ and it upper bounds $C$, and Follow-the-Leader is used to select the learned policy, then for any $\epsilon>0$, after collecting $\mathcal{O}\left(\frac{T^{2}l^{2}_{max}\log{\frac{1}{\mu}}}{\epsilon^{2}}\right)$ training examples, with probability at least $1-\mu$, $\mu\in(0,1)$, ADAPS offers the following guarantee:

[TABLE]

Now, we only need the training error $\hat{\epsilon}_{min}$ to be minimal, which is achievable via standard supervised learning.

1.3.2.2 Experimental Results

I have tested my method empirically in three simulated scenarios, namely a straight road (representing a linear geometry), a curved road (representing a non-linear geometry), and an open ground. The straight and curved roads represent an on-road scenarios with a static obstacle in the form of a traffic cone. The open ground represents an off-road scenario with a dynamic obstacle in the form a vehicle.

I have compared my policy to the technique from Bojarski et al. Bojarski et al., (2016), as it is one of the representative approaches for end-to-end autonomous driving. Usually, this type of approach is limited to single-lane following Chen et al., (2015); Xu et al., (2017); Zhang and Cho, (2017); Codevilla et al., (2017) or off-road collision avoidance LeCun et al., (2005) behaviors.

For the two on-road scenarios, I collect training datasets from straight road with or without an obstacle and curved road with or without an obstacle. This separation enables six policies for testing the effectiveness of learning from accidents:

• —

My policy: trained with just the lane-following data $O_{follow}$; $O_{follow}$ additionally trained after analyzing an accident on the straight road $O_{straight}$; and $O_{straight}$ additionally trained after analyzing an accident on the curved road $O_{full}$.

• —

Similarly, for the policy from Bojarski et al. Bojarski et al., (2016): $B_{follow}$, $B_{straight}$, and $B_{full}$.

Being evaluated first are $O_{follow}$ and $B_{follow}$. The evaluation strategy is to run these two policies on both the straight and curved roads, and count how many laps (out of 50) can be safely finished. The results showing both policies can finish all 50 laps safely. Then, I add an obstacle to the straight road and test both policies again. Since both policies have not trained on the accident data yet, they both run into the obstacle and cause an accident.

After analyzing the occurred accident and incorporating the accident data into training, I obtain two new policies $O_{straight}$ and $B_{straight}$. As expected, $O_{straight}$ avoids the obstacle, while $B_{straight}$ continues to cause collision.

I proceed to add an obstacle to the curved road, after a similar process, I obtain $O_{full}$ and $B_{full}$. Again, as expected, $O_{full}$ manages to perform both lane-following and collision avoidance behaviors. $B_{full}$, in contrast, leads the vehicle to an accident. These two cases are illustrated in Figure 1.18 LEFT and CENTER.

In order to test the generalization of my policy, I uniformly sampled 50 positions to place the obstacle on a $3$ meters line segment that is perpendicular to the direction of a road and in the same lane as the vehicle. The success rate (i.e., avoid an obstacle and resume normal driving) are documented in Table 1.2.

For the off-road scenario, I first train the AV to head towards a green sphere as its target. Then, I scripted an adversary vehicles to collide with the AV on its default course. By having and addressing the accident, my policy can steer the vehicle away from the adversary vehicle and resume its direction to the target. This is illustrated in Figure 1.18 RIGHT.

The key to rapid improvements of a policy is the generation of sufficient and heterogeneous training data. In Figure 1.19, I show the visualization results of images collected via my method and DAGGER Ross et al., (2011) in one learning iteration. By progressively increasing the number of sampled trajectories, my method results in much more heterogeneous training data, which, when produced in a large quantity, can greatly facilitate the update of a policy.

Thesis Statement

My thesis statement is as follows:

Urban mobility can benefit from better designed intelligent transportation systems which can be further improved with 1) accurate and efficient reconstruction of city-scale traffic at the macroscopic level and 2) enhanced control and learning mechanisms for individual vehicles at the microscopic level.

To support this thesis, at the macroscopic level, I have developed methods to efficiently estimate traffic conditions, accurately interpolate spatial-temporal missing traffic data, dynamically reconstruct traffic flows, and produce visual analytics in various forms. At the microscopic level, I have developed a framework to simulate and analyze various driving scenarios while automatically producing labeled training data, which, when combined with an efficient online learning mechanism and an effective policy architecture, can lead to robust control policies for autonomous driving.

Organization

The remainder of this dissertation is organized as follows. In Chapter 2, I describe my approach to deterministically estimate traffic conditions and interpolate temporal missing traffic data. In Chapter 3, I describe my approach to iteratively estimate traffic conditions and interpolate spatial missing traffic data. In Chapter 4, I present ADAPS, a framework for obtaining robust control polices for autonomous driving. Finally in Chapter 5, I conclude this dissertation with discussion of potential future research directions.

CHAPITRE 2:  CITYWIDE ESTIMATION OF TRAFFIC DYNAMICS VIA SPARSE GPS TRACES

Introduction

Traffic is ubiquitous in modern cities, impacting their social, economic, and environmental developments. However, ever-present gridlock and congestion keep challenging transportation researchers and urban planners. According to the 2015 Urban Mobility Scorecard Schrank et al., (2015), traffic congestion causes an extra 6.9 billion travel hours and 3.1 billion gallons of fuel consumption annually in the United States, which costs are approximately $160 billion. As an increasing number of metropolitan areas experience severe traffic conditions and the overall cost is estimated more than one trillion U.S. dollars worldwide, the ability to analyze and understand traffic dynamics is becoming crucial.

In order to understand traffic congestion, first, we need to obtain its measurements. Traditionally, traffic measurements are collected via in-road sensors such as loop detectors and video cameras Leduc, (2008). While these sensors produce relatively accurate records, the high expenditures for installation and maintenance prevent them from being deployed over an entire city, rather than major roads and highways. Consequently, the lack of sensing infrastructure for arterial streets—which comprise the majority of a city—has made the large-scale traffic measuring task difficult.

Mobile data such as GPS traces, in contrast, are more promising sources for understanding citywide traffic dynamics due to their much boarder coverage. However, such data are limited in two aspects: 1) inevitable errors in measurement and transmission often yield reported locations off the road, and 2) due to energy and privacy concerns, GPS data commonly have a low sampling rate, meaning that the time difference between consecutive points can be large (e.g., greater than 60 seconds), and a low penetration rate, meaning that only a small percentage of traffic population is willing to send location reports.

Together, these limitations of GPS data bring a large degree of uncertainty into using them for studying and estimating traffic dynamics, for which purpose, several processing steps are required. First, off-the-road GPS points need to be mapped onto the road network, and the true traversed paths of vehicles need to be inferred. This process is called map-matching. Second, the time taken to travel each road segment must be accurately estimated. Because of the low sampling rate, the inferred path between two consecutive GPS points is likely to consist of multiple road segments, but only the aggregate travel time (i.e., the difference between the GPS timestamps) is known. The aggregate travel time needs to be distributed to individual road segments, which process is named travel time allocation. Third, in order to understand the full traffic dynamics of a city, traffic data are needed for an entire traffic period for each road segment of a city’s road network. However, GPS data often do not provide complete temporal coverage as they are commonly scarce in late night and early morning hours. The process for interpolating the missing temporal information is named missing value completion.

Many efforts have been made towards improving the effectiveness of the abovementioned three processing steps. To be specific, the low sampling rate introduces issues: two consecutive GPS points are likely far apart from each other and multiple paths can exist for connecting them (especially in a complex urban environment). Thus, inferring the true traversed path between them is challenging. Many state-of-the-art map-matching approaches use the shortest-distance criterion Lou et al., (2009); Yuan et al., (2010); Miwa et al., (2012); Hunter et al., (2014); Chen et al., (2014); Quddus and Washington, (2015) to infer the traversed path. While this criterion is viable when a road network is under or close to a free-flow condition, it can introduce errors in a congested environment where other paths (not the shortest-distance path) can be traveled with less time and will be preferred by GPS devices and most drivers. If we have a wrongly inferred path, the timestamp difference (i.e., aggregate travel time) will be distributed to a wrong set of road segments. In other words, the introduced errors will be carried over to the subsequent step travel time allocation, causing the overall estimation of traffic conditions deteriorated.

I have developed a novel framework to estimate citywide traffic dynamics using GPS data. My framework is developed based on two observations: 1) traffic patterns exhibit weekly periodicity Hofleitner et al., 2012c , and 2) traffic conditions are quasi-static Hunter et al., (2014). Based on these observations, I treat one week as a traffic period and assume the traffic conditions within each hourly time interval of a weekly period as static. Computationally, my framework consists of two phases. The first phase is conducted on individual time intervals and the second phase is performed over all time intervals. In the first phase, based on Wardrop’s Principles Wardrop, (1952), I use the shortest travel time criterion instead of the shortest distance criterion to perform map-matching. Along with a travel time allocation technique adapted from Hellinga et al. Hellinga et al., (2008), a novel computation scheme is designed to reconstruct traffic dynamics of a road network. In the second phase, exploiting the sparsity embedded in traffic patterns, I have developed a novel method based on Compressed Sensing Donoho, (2006); Candes et al., (2006) to interpolate missing travel information over an entire traffic period. The overview of my framework pipeline is shown in Figure 2.1.

I have extensively evaluated and tested the effectiveness of my approach and compared it to the method developed by Lou et al. Lou et al., (2009), using GIS data from a synthetic road network and the city of San Francisco Piorkowski et al., (2009). The results demonstrate major improvements over the existing method Lou et al., (2009) in various steps during the estimation process. In summary, the contributions of this work are the following:

• —

A novel perspective (switching from the shortest-distance criterion to the shortest travel-time criterion) in addressing sparse GPS traces during map-matching;

• —

An improved travel time allocation technique which incorporates estimated intermediate traffic conditions into computation;

• —

An efficient and robust method for interpolating missing traffic measurements in traffic patterns.

The rest of this chapter is organized as follows. In Section 2.2, I discuss the related work and highlight the differences between my work and existing studies. In Section 2.3, I take a holistic view of map matching and travel time allocation, detailing the process of reconstructing the velocity field of a road network. In Section 2.4, I explain how to interpolate missing travel information. In Section 2.5, I show the estimated traffic dynamics of San Francisco and the analysis of the key features of the recovered traffic patterns. Finally, I conclude with a discussion of future work in Section 2.6.

Related Work

Over the last few decades, the estimation of traffic conditions has gained increasing scholarly attention Celikoglu, (2007); Celikoglu et al., (2009); Gao, (2012); Abadi et al., (2015); Kachroo and Sastry, (2016); Agarwal et al., (2016). Early studies on traffic estimation have focused on traffic states on highways using accurate measurements from stationary sensors such as loop detectors and video cameras Leduc, (2008). Recent advancements have shifted to combining multiple data sources and traffic simulation models for achieving better estimation results Work et al., (2010); Sun and Work, (2014); Gning et al., (2011); Li et al., (2014); Celikoglu and Silgu, (2016); Hajiahmadi et al., (2016). However, the scenarios of interest in these studies were limited to road segments with lengths of a few kilometers.

The increasing availability of GPS data provides new means for conducting large-scale estimation of traffic conditions. However, as GPS data are inherently noisy, the estimated traffic conditions usually do not satisfy the flow conservation requirement assumed by many simulation models Phan and Ferrie, (2011); Kong et al., (2013); Zhang et al., (2013). As a result, new studies that consist of several steps for the estimation task, are emerging.

The first step, map-matching addresses the problem of mapping off-the-road GPS points onto a road network and identifies the true traversed path between consecutive GPS points. However, GPS data could contain a low sampling rate, which causes points to be far away from each other and making the selection among multiple paths connecting the points difficult. In order to determine the “actual” traversed path of a vehicle, a common approach is to use the shortest-distance criterion to connect two GPS points on a road network Lou et al., (2009); Yuan et al., (2010); Miwa et al., (2012); Hunter et al., (2014); Chen et al., (2014); Quddus and Washington, (2015). Nonetheless, the shortest-distance assumption can lead to errors in a congested environment, in which alternative paths can be traveled faster than the shortest-distance path and preferred by GPS devices and drivers. Essentially, the shortest-distance criterion only uses spatial information (i.e., longitude and latitude of GPS points, and the geometry of a road network), while ignoring the temporal information (i.e., timestamps) recorded in GPS reports. This happens primarily due to the travel times of a road network are largely unknown, causing the temporal information has nothing to be compared with Tang et al., 2016a ; Rahmani and Koutsopoulos, (2013).

After map-matching, travel time of individual road segments need to be estimated. To provide a few examples, Hellinga et al. Hellinga et al., (2008) have developed an analytical solution to estimate travel times of road segments using intuitive and empirical observations of traffic patterns in real life. Rahmani et al. Rahmani et al., (2015) take a non-parametric approach, performing an estimation using a kernel-based method. Probabilistic frameworks have also been developed to conduct an estimation of traffic conditions Khosravi et al., (2011); Westgate et al., (2013); Herring et al., (2010); Hofleitner et al., 2012a ; Kuhi et al., (2015). While significant improvements have been achieved, these methods all perform the estimation steps sequentially, causing the limitations of map-matching will be carried over to its subsequent steps and eventually deteriorate the overall estimation accuracy.

Researchers have also proposed solutions to missing value completion. For example, tensor-based approaches Wang et al., 2014a ; Asif et al., (2016), which explore correlations among nearby road segments, have been developed. From Zhu et al. Zhu et al., (2013) and Mitrovic et al. Mitrovic et al., (2015), algorithms based on Compressed Sensing have been proposed by taking an entire road network as the study subject. Interpolating missing values has also been addressed in an online setting Anava et al., (2015). Nevertheless, the abovementioned methods were not designed to tackle the problem of estimating full traffic dynamics of individual road segments over an entire city—a subject for which little progress has been made Hofleitner et al., 2012c .

Traffic Velocity Field Reconstruction

I take a holistic view of map-matching and travel time allocation, and propose a noval method to reconstruct the velocity field of a road network. Starting with some definitions in this section, I then discuss methodologies and implementation details of my approach. My algorithm is evaluated and validated using a synthetic road network with microscopic traffic simulation.

Notation and Definitions

A road network is defined as a directed graph $G=(V,E)$ in which edges $E$ denote road segments and nodes $V$ represent intersections and terminal points. Each road segment $e\in E$ contains several attributes: the length $e.len$, the maximum/free-flow travel speed $e.v_{max}$, the minimum/free-flow travel time $e.t_{min}=\frac{e.len}{e.v_{max}}$, and the maximum/jam density $e.k_{max}$.

A path from node $g$ to node $h$ on a network g\overset{\text{p}}{\leadsto}h is a collection of road segments $p=\{e_{1},e_{2},\dots,e_{n}\}$, where $g$ is the starting node of $e_{1}$ and $h$ is the ending node of $e_{n}$. A trace is a sequence of GPS points $S=\{s_{1},s_{2},\dots,s_{n}\}$ in which each point is a tuple $s_{i}=<s_{i}.x,s_{i}.y,s_{i}.t>$ containing longitude, latitude, and a timestamp.

Velocity Field Estimation

Given the periodicity of traffic patterns over a week Hofleitner et al., 2012c , I study traffic dynamics over the region of interest in a weekly period. I discretize one week into hourly time intervals and assume that traffic conditions remain the same within one hour on a road segment. For simplicity, I restrict my discussion of estimating the velocity field to one time interval (i.e., one hour). The process can be trivially extended to cover other time intervals of an entire traffic period.

Ideally, if the actual traversed path of a vehicle is known and the generated GPS points are exactly on the road, I can derive the average travel speed of a path $p$ that connects GPS points $s_{i}$ and $s_{i+1}$ as $p.t=\frac{\sum_{e\in p}e.len}{s_{i+1}.t-s_{i}.t}$. However, GPS points are often off-the-road due to inherent measurement and sensing errors, and the underlying path of a vehicle is also unknown. To address these issues, a number of candidate nodes of the network are considered for mapping a GPS point, based on their distances to the point. Then, one of the paths connecting a pair of candidate nodes of two consecutive GPS points is selected to represent the actual path. As mentioned earlier, one common approach for choosing such a path is taking the shortest distance criterion Lou et al., (2009); Yuan et al., (2010); Miwa et al., (2012); Hunter et al., (2014); Chen et al., (2014); Quddus and Washington, (2015), which can produce errors in a congested environment (an illustrative example is shown in Figure 2.2).

According to Wardrop’s Principles Wardrop, (1952), the traffic in congested networks would move in a way such that no vehicle can reduce its travel time by switching routes. This state is called user equilibrium, and is a result of every vehicle non-collaboratively attempting to minimize its traveling cost—which commonly appears to be the travel time. Under such an equilibrium state, the average travel time is balanced (i.e., the same) for all users of the network.

Assumption 1. Based on Wardrop’s Principles Wardrop, (1952) and the observation that modern GPS devices largely adopt the fastest route, I assume all GPS traces are planned using the shortest travel time criterion.

Denoting the network with ground-truth traffic conditions as $G_{true}$, according to Assumption 1, all GPS traces represent the fastest routes planned on $G_{true}$. However, since $G_{true}$ is unknown, my goal is to use available GPS traces and the initial road network $G_{est}$, with all road segments set to their speed limits, to estimate $G_{true}$, i.e., reconstruct the velocity field of $G_{true}$.

Corollary 2.

A GPS trace with travel time $t$ matching a path g\overset{\text{p}}{\leadsto}h implies that no path in $G_{true}$ from $g$ to $h$ has a travel time smaller than $t$.

Démonstration.

According to Assumption 1, the traffic along every traveled route between $g$ and $h$ in $G_{true}$ is in user equilibrium, meaning all routes between $g$ and $h$ have the equivalent lower-bound travel time. ∎

Corollary 3.

A pair of GPS points from a trace matched to locations $g$ and $h$ are sufficient to bound the travel time for all paths connecting $g$ and $h$.

Démonstration.

According to Assumption 1, the path indicated by a GPS trace is a time-optimal path (i.e., the shortest travel-time path). Therefore, it has an optimal substructure, in which a subset of two is time-optimal and provides a bound on the travel time for paths from $g$ to $h$. ∎

As Collorary 3 implies, during the reconstruction of a velocity field, I can inspect two consecutive GPS points at a time (due to the optimal substructure). Consider two arbitrary GPS points $s_{i}$ and $s_{i+1}$, the true path s_{i}\overset{\text{p_{true}}}{\leadsto}s_{i+1} is the fastest path between $s_{i}$ and $s_{i+1}$ on $G_{true}$. Based on Collorary 2, the travel time $p_{true}.t=s_{i+1}.t-s_{i}.t$ is the lower bound of all travel times between the two points. This means if a path $p_{est}$ has a higher travel time, the speed of the road segments on $p_{est}$ should be decreased utill $p_{est}.t\geq p_{true}.t$. I refer to such $p_{est}$ as an overestimated path. In practice, $p_{est}$ connects candidate nodes of $s_{i}$ and $s_{i+1}$ rather than $s_{i}$ and $s_{i+1}$ themselves, and there exists a set of paths $\{p_{est}\}_{all}$ for $s_{i}$ and $s_{i+1}$, in which one is the “closest” to $p_{true}$. Denoting an arbitrary element in $\{p_{est}\}_{all}$ as $p_{est}$, if $p_{est}$ and $p_{true}$ happen to be the same path (i.e., containing the same set of road segments), $p_{est}.t$ should be equal to $p_{true}.t$, otherwise $p_{est}.t$ should be larger than $p_{true}.t$. Since there is no further information for me to estimate the excessive time of $p_{est}.t$ over $p_{true}.t$, I conservatively set $p_{est}.t=p_{true}.t$. I refer to this process of addressing overestimated paths in $\{p_{est}\}_{all}$ as relaxation.

The relaxation will make all paths in $\{p_{est}\}_{all}$ having the same travel time (i.e., $p_{true}.t$). However, it is difficult to deterministically derive the “closest” path to $p_{true}.t$ using a single GPS trace. To remedy this issue, I rely on the “collective intelligence” of multiple GPS traces that share a partial or full set of road segments. These segments will become gradually updated during relaxation of each GPS trace and eventually assist in differentiating the paths that include them from other paths in terms of the travel time. An illustration of this process is shown in Figure 2.3. The relaxation is essentially a fulfillment of Wardrop’s Principles, and is conducted in a greedy fashion: I repeatedly extract the fastest path in $\{p_{est}\}_{all}$ and relax it, until no path in $\{p_{est}\}_{all}$ has its travel time smaller than $p_{true}.t$. Given that there may exist multiple paths connecting two nodes in a network, causing the number of elements in $\{p_{est}\}_{all}$ to be large, a sub-network with a specified radius is extracted from the entire network. This sub-network encompasses $s_{i}$, $s_{i+1}$, and their mapping candidates but no more. Using this approach, the number of paths in $\{p_{est}\}_{all}$ is reduced. The rationale behind this choice is through empirical observations that a vehicle rarely takes an opposite direction or arbitrary long detour from a GPS point to the next one.

Theorem 2.1**.**

The speed of a road segment is monotonically decreasing during relaxation.

Démonstration.

I prove this theorem by contradiction. Assume two overestimated paths $p_{est}^{1}$ and $p_{est}^{2}$ share a road segment $e$. Without loss of generality, consider the relaxation of $p_{est}^{1}$, the road segment $e$ has its speed $e.v$ decreased to $e.v^{\prime}$ and travel time $e.t$ increased to $e.t^{\prime}$ so that $p_{est}^{1}.t=p_{true}^{1}.t$. During relaxation of $p_{est}^{2}$, $e.v^{\prime}$ and $e.t^{\prime}$ are subject to change. If instead of monotonically decreasing, $e.v^{\prime}$ gets increased and $e.t^{\prime}$ gets decreased, then $p_{est}^{1}.t<p_{true}^{1}.t$, which invalidates the previous relaxation process and further contradicts Collorary 2. ∎

Taking advantage of Theorem 2.1, further reduction in computation can be achieved by retaining reduced speed of each path in $\{p_{est}\}_{all}$. To be specific, as $\{p_{est}\}_{all}$ is generated for $s_{i}$ and $s_{i+1}$ in a sub-network, many paths in $\{p_{est}\}_{all}$ will have shared road segments. Therefore, the speed reduction in these road segments will make multiple paths in $\{p_{est}\}_{all}$ to have increased travel times. As a result, the greedy process of relaxation is much more efficient than the brute-force enumeration.

The overall process is described in Algorithm 1. Subroutines RelaxNetwork and Relaxation are specified in Algorithm 2 and Algorithm 3, respectively. In particular, two types of paths are considered as outliers, which are excluded from the computation: one has a travel time shorter than its free-flow travel time and one has a travel time longer than the travel time under a jam density. The procedure TravelTimeAllocation in Line 10 of Algorithm 3 is discussed in the next section.

Travel Time Allocation

During relaxation, each overestimated path $p_{est}$ between $s_{i}$ and $s_{i+1}$ needs to be addressed in order to achieve $p_{est}.t=p_{true}.t=s_{i+1}.t-s_{i}.t$. Due to low sampling rate, $p_{est}$ often consists of several road segments. The aggregate travel time $p_{true}.t$ needs to be appropriately allocated to individual road segments of $p_{est}$. To address this issue with respect to traffic flow analysis, I adapt the solution proposed by Hellinga et al. Hellinga et al., (2008).

According to Hellinga et al. Hellinga et al., (2008), the travel time of a road segment $e$ can be decomposed into three components: free-flow travel time $\tau_{e,f}$, congestion time $\tau_{e,c}$, and stopping time $\tau_{e,s}$. For an overestimated path $p_{est}$, I denote its total free-flow travel time as $T_{f}=\sum_{e\in p_{est}}\tau_{e,f}$, total congestion time as $T_{c}=\sum_{e\in p_{est}}\tau_{e,c}$, total stopping time as $T_{s}=\sum_{e\in p_{est}}\tau_{e,s}$, and the allocation time as $T_{a}=p_{true}.t$. To validate $p_{est}$, the following relationship needs to be ensured: $T_{f}+T_{c}+T_{s}=T_{a}$. While $T_{f}$ can be derived trivially as $T_{f}=\sum_{e\in p_{est}}\tau_{e,f}=\sum_{e\in p_{est}}\frac{e.len}{e.v_{max}}$, the computations of $T_{c}$ and $T_{s}$ require additional considerations.

By assuming nearby road segments have similar traffic conditions, the path congestion level is defined as $w=\frac{T_{c}}{T_{c}+T_{f}}$. The minimum value $w_{min}=0$ is reached when a path can be traveled using its free-flow speed, and the maximum value $w_{max}=\frac{T_{c,max}}{T_{c,max}+T_{f}}$ where $T_{c,max}=T_{a}-T_{f}$ is reached when a path is congested and $T_{s}=0$. Using a specific path congestion level $w_{min}\leq w\leq w_{max}$, the probability that congestion occurred on $p_{est}$ is computed as follows:

[TABLE]

where $T_{c,max}^{prev}$ and $T_{a}^{prev}$ represent the maximum congestion time and the allocation time of path s_{j}\overset{\text{p_{est}^{prev}}}{\leadsto}s_{j+1},\,j+1\leq i. From Hellinga et al. Hellinga et al., (2008), $p_{est}^{prev}$ denotes the path that has an allocation time longer than the jam-density travel time with maximum possible $j$. As I have excluded the path outliers, $p_{est}^{prev}$ indicates the path connecting $s_{i-1}$ and $s_{i}$, and is the directly preceding path of $p_{est}$. $P_{c}(w)$ is defined under assumptions: when all variables are fixed, the probability of a specific level of congestion occurring increases as $T_{c,max}$ increases; given a particular $T_{c,max}$, higher level congestion is less likely to occur than lower level congestion.

Next, the stopping likelihood function is defined for computing the probability of stopping. Since the original formula does not take estimated traffic conditions into account, I alter it to the following:

[TABLE]

where $\beta$ is the weighting factor (set to 0.5), and $e.k$ is the estimated density as a result of possible previous relaxation (otherwise $e.k=0$). Equation 2.2 computes the likelihood by leveraging the path congestion level $w$ and the road-segment congestion level $\frac{e.k}{e.k_{max}}$. Intuitively, the road-segment congestion level becomes higher when the estimated density $e.k$ approaches the jam density $e.k_{max}$. In order to derive $e.k$ from the estimated speed $e.v$, I utilize the Greenshield’s model Greenshields et al., (1935):

[TABLE]

By having the stopping likelihood function, the probability that a vehicle stopped on a particular road segment is stipulated by assuming the vehicle stops at most once on $p_{est}$:

[TABLE]

Using Equation 2.1 and Equation 2.4, the congestion time on a single road segment is calculated by integrating all path congestion levels together:

[TABLE]

where $Q_{s}=\int_{0}^{w_{max}}P_{c}(w)\sum_{e}P_{s,e}(w)dw$ is the normalizing factor. After summing the congestion time of all road segments $T_{c}=\sum_{e\in p_{est}}\tau_{e,c}$, the total stopping time can be derived easily: $T_{s}=T_{a}-T_{f}-T_{c}$. Finally, I can calculate the stopping time on each road segment using the following formula:

[TABLE]

To this point, I have solved the travel time allocation problem by having $\tau_{e,f}$, $\tau_{e,c}$, and $\tau_{e,s}$ for all road segments of $p_{est}$ and fulfilled $T_{f}+T_{c}+T_{s}=T_{a}$ (i.e., $p_{est}.t=p_{true}.t$).

Evaluation on A Synthetic Network Using Traffic Simulation

In order to evaluate my technique on the effectiveness of estimating a velocity field, I use a synthetic road network and an agent-based traffic simulator SUMO Krajzewicz et al., 2012b . The road network is modeled as a grid with $5\times 5$ intersections. By treating one hour as a time interval for a specific congestion level, a set of vehicles is routed in the network. The average travel times of all road segments are taken as the ground truth for this congestion level. All traces are simulated by randomly selecting nodes of the network as sources and targets using the fastest route strategy. As my method operates on size two subsets of GPS traces, I emit pairs of points at the source and target for each simulated trace to resemble the low sampling rate feature of real-world GPS data. This design choice enables the testing of the travel-time allocation algorithm developed in the previous section.

All road segments share the same setting: length of $150\leavevmode\nobreak\ m$, maximum speed at $17.88\leavevmode\nobreak\ m/s$, and a maximum density of $0.15\leavevmode\nobreak\ vehicles/meter$ . In total, 30 congestion levels are created by simulating 50 to 1500 vehicles with an increment of 50 vehicles per time interval. In addition, for each time interval, five levels of the vehicle population, namely at 20%, 40%, 60%, 80%, and 100%, are sampled to generate GPS reports.

The first analysis is conducted by treating the network as a whole: the aggregate travel time over the entire network is used for evaluation. The comparison between the estimated values computed using my technique and Lou et al. Lou et al., (2009), respectively to the ground-truth values are shown in Figure 2.4. Starting with 20% of vehicles, my technique demonstrates close approximations to the ground truth at all congestion levels, while the other technique fails to achieve the same performance. Shown in Table 2.1, I examine the influence of different traffic population percentages in estimation by computing the absolute error to the ground truth across all congestion levels. The smallest error (mean = $3.4\leavevmode\nobreak\ s$, std = $3.3\leavevmode\nobreak\ s$) is achieved using 80% of GPS traces from the simulated traffic. The slight increase in errors when using 100% GPS traces is mainly due to the stochastic aspect of the travel-time allocation method that I adapted.

The absolute errors in the recovered travel time computed using my technique vs. Lou et al. (Lou et al., 2009) by using GPS traces from various percentages of the traffic population. My technique results in much smaller errors as of Lou et al. (Lou et al., 2009).

The second analysis computes the relative improvements measured in mean squared error (MSE) as $\frac{1}{n}\sum_{i=1}^{n}(\hat{e.t}-e.t)^{2}$ of my method over Lou et al. Lou et al., (2009). The results are shown in Figure 2.5. As the congestion level increases or a higher percentage of GPS traces becomes available in estimation, my technique outperforms the other method. The improvements are less clear when a congestion level is low (i.e., $<10$), but apparent when a congestion level is high (i.e., $\geq 10$).

Missing Value Completion

The temporal sparsity of GPS data indicates that traffic data can be missing for certain time intervals within a weekly period. The missing data inhibit us to accurately estimate full traffic dynamics. To address this issue, I explore sparsity embedded in traffic patterns and propose a novel technique based on the Compressed Sensing algorithm Donoho, (2006); Candes et al., (2006) for traffic reconstruction.

I have adopted loop-detector data111The loop-detector data are obtained from Caltrans Performance Measurement System (PeMS): http://pems.dot.ca.gov/., which represent complete and accurate measurements of traffic, to explore features of traffic patterns. I use speed measurements from 38 loop detectors installed in San Francisco for this purpose. The time range of these data is the same as that of the Cabspotting dataset Piorkowski et al., (2009) (GPS data) in San Francisco. I refer to the average speed measurements over a weekly period from a loop detector as a traffic signal. An example is shown in the top panel of Figure 2.6, which exhibits clear periodicity.

I performed a spectral analysis on a traffic signal by setting the frequencies as the reciprocal of the signal length (i.e., 1/168, an hourly interval within a weekly period) and subtracting the signal from its mean to make the oscillations easier to observe. The results, shown in the middle panel of Figure 2.6, reveal that the period of the most salient oscillation is 24 hours (i.e., one cycle per day). In addition, the signal is sparse in the frequency domain, which is reflected as over 95% energy is preserved by retaining its 10 largest frequencies (Figure 2.6, BOTTOM).

According to the specification of a traffic signal, the highest frequency being supported is one cycle per two hours. To recover a traffic signal from its samples, the classical Nyquist-Shannon Theorem requires at least 168 measurements, which we do not have due to the temporal sparsity of GPS data. However, the Compressed Sensing algorithm Candes et al., (2006); Donoho, (2006) suggests that a signal can be recovered with a small set of samples if the signal has a sparse representation. This feature is manifested in Figure 2.6 and the top panel of Figure 2.7, which shows the rapid decaying of the sorted frequency magnitudes of traffic signals. On average, the decaying rate reaches 45.8% with the most prominent frequency and 78.9% with the 10 most prominent frequencies. After the $15$th frequency, the decaying rates become ineligible.

Another merit of Compressed Sensing is that it does not require any prior knowledge of the sparse structure of a signal, such as the locations of large frequency components. This generality is ideal for recovering the traffic condition of a road segment, given that traffic is intrinsically stochastic and the sparse structure of a traffic signal varies from one road segment to another. In the bottom panel of Figure 2.7, I plot locations and amplitudes of the frequency components of all obtained traffic signals in my analysis: besides the appearance of the prominent oscillations at 24, 48, and 72 hours, other oscillations are spread out along the frequency axis. Together, these observations and features confirm the applicability of using Compressed Sensing on traffic signal recovery.

Given a signal $f\in\mathbb{R}^{n}$ and its measurements $b\in\mathbb{R}^{m}$, I consider the undersampled case in which the number of measurements $m$ is smaller than the signal’s dimension $n$. The goal is to derive an estimated signal $\hat{f}\in\mathbb{R}^{n}$ from $b\in\mathbb{R}^{m}$ such that the error $\lVert f-\hat{f}\rVert_{\mathcal{L}_{2}}$ is minimized. In general, the better the desired reconstruction quality, the more measurements are needed. In order to achieve a predefined accuracy level, signal reconstruction requires a minimum number of measurements $m_{min}$. According to Candés and Wakin Candès and Wakin, (2008), $m_{min}$ is on the order of $\mu^{2}S\log(n)$, where $\mu$ is the coherence between a measurement basis and a representation basis, $S$ is the signal’s sparsity level, and $n$ is the signal’s dimension (in this chapter $n=168$). I estimate $S$ by averaging the number of frequencies in preserving 95% energy of all traffic signals, which results in $S=17.63$. The minimum coherence value $\mu=1$ is obtained by performing a discrete cosine transform (DCT) on a traffic signal $f$:

[TABLE]

where $f_{dct}$ is the representation of $f$ in the DCT domain and ${\bf\Phi}_{n\times n}$ is the DCT matrix. With these estimated values, the minimum number of samples required to recover a traffic signal can be computed: $m_{min}=\mu^{2}S\log(n)=1^{2}\cdot 17.63\cdot\log(168)\approx 90$.

I test the performance of my method by first obtaining random measurements via sampling: $b_{m\times 1}={\bf\Psi}_{m\times n}f_{n\times 1}$, where ${\bf\Psi}$ is the sampling matrix constructed by randomly permuting rows of the identity matrix. Then, I derive the recovered signal in the DCT domain $\hat{f}_{dct}$ from $b$ by solving the following linear system:

[TABLE]

Equation 2.8 represents an underdetermined system, which has infinitely many solutions $\hat{f}_{dct}$. Among all solution candidates, the desired $\hat{f}_{dct}$ should exhibit sparsity as observed in $f_{dct}$, which can be computed as follows:

[TABLE]

An example solution to Equation 2.9 is shown in Figure 2.8, where the actual solution elements of $f_{dct}$ and the recovered solution elements of $\hat{f}_{dct}$ both demonstrate sparsity. The final recovered signal $\hat{f}$ is acquired by performing an inverse DCT on $\hat{f}_{dct}$. Figure 2.9 gives an example in which the recovered signal exhibits high similarity to the original signal. A more thorough analysis of the recovering performance can be found in Figure 2.10. As a result, as the number of measurements used in recovery increases, both the standard deviation and the expectation of the $\mathcal{L}2$ loss decrease (the average error of using $90$ measurements is $1.4\leavevmode\nobreak\ m/s$). This analysis also demonstrates the robustness of my approach as the error only increases linearly when the number of samples used in recovery decreases.

To apply Compressed Sensing on GPS data, the measurements will be obtained from travel time estimation rather than from a sampling operation performed on a traffic signal. In this case, ${\bf\Phi}$ is set to $DCT\left(diag\left(1,\dots,1\right)\right)$, and ${\bf A}={\bf\Psi}{\bf D}$ is taken to solve Equation 2.9. Since I have established that $m_{min}=90$, it is worth mentioning I only address road segments that have measurements in more than 90 time intervals. Compared to techniques from Herring et al. Herring, (2010) and Hofleitner et al. Hofleitner et al., 2012c , I have reduced the minimum number of measurements required to recover a traffic signal from 168 to 90 by 46.4%.

Estimating Traffic Dynamics Via GPS Data

One of the hallmarks of traffic dynamics is the periodicity Hofleitner et al., 2012c : traffic patterns show a clear trend over the course of a day and collectively over the course of a week. In this section, I first demonstrate that this phenomenon can be recovered using my technique. Then, I will analyze and discuss features revealed in the reconstructed traffic patterns of San Francisco using the GPS dataset Cabspotting Piorkowski et al., (2009).

To assist visualization and analysis, the metric $\text{\emph{fluidity}}\in[0,1]$ Hofleitner et al., 2012c , computed as the ratio of the estimated travel speed to the free-flow speed, is adopted for each road segment. In Figure 2.11, I show the estimated traffic dynamics using the Cabspotting dataset Piorkowski et al., (2009), denoted by the average fluidity, across the road network of downtown San Francisco. From the demonstration, it is clear that my technique recovers the periodicity of the traffic pattern through the dominant frequency showing at one cycle per day. This characteristic resembles the one observed in loop-detector data from the same area (see Figure 2.6 and 2.9), thus proving the effectiveness of my technique.

The affinity between different days in a week can also be used to illustrate the quality of a recovery method. I have computed the correlation of every pair of days using the cosine distance for both the estimated traffic conditions using GPS data and the actual traffic conditions derived from loop-detector data. In the left panel of Figure 2.12, I provide all distance scores: the upper triangular matrix is derived using the estimated values using GPS data, and the lower triangular matrix is computed using loop-detector data. In the right panel of Figure 2.12, I provide the qualitative result for visual inspection. The symmetrical pattern across the diagonal line indicates that the estimated traffic states largely agree with the loop-detector data.

Based on the distance scores, a hierarchical clustering is performed to reveal the similarity between various day pairs. The closest pair is Wednesday and Thursday, followed by Friday and Saturday, and Monday and Tuesday. In the second level of the hierarchy, Sunday joins Monday and Tuesday. These three day-pairs suggest that a typical week of San Francisco can be roughly divided into three stages: beginning of the week (Sunday, Monday, and Tuesday), middle of the week (Wednesday and Thursday), and end of the week (Friday and Saturday).

Summary and Future Work

I have presented a novel computational scheme for estimating travel times, traversed paths, and missing values over a large-scale road network using spatially and temporally sparse GPS traces. Specifically, an approach based on the shortest travel time is performed to reconstruct the velocity field of a road network. In addition, an algorithm based on the Compressed Sensing algorithm has been developed to interpolate missing travel information over an entire traffic period. Lastly, I have extensively evaluated my approach and compared my technique to a state-of-the-art technique. My technique demonstrates consistent improvements over the previous technique in various traffic scenarios.

There are several possible future directions. To start with, as abundant GPS data are becoming available, processing them will be computationally expensive. Besides using the power of distributed computing, it is promising to explore sparsity and periodicity of traffic patterns to further reduce the amount of data needed for various Intelligent Transportation System (ITS) applications Lin et al., (2019). Similarly, correlation in traffic patterns due to proximity/spatial coherence can also be examined. Lastly, it might be beneficial to integrate data mining of historical data, real-time traffic reconstruction from current data, and predictive traffic simulation to achieve more comprehensive and accurate estimations of travel conditions over a metropolitan area.

While in this chapter, I have introduced a deterministic approach to estimate traffic conditions in areas with GPS data coverage and a novel algorithm based on Compressed Sensing to interpolate temporal missing values, an effective method to address the spatial sparsity, i.e., estimating traffic conditions in areas without GPS data coverage, is still needed, given our ultimate goal is to reconstruct city-scale traffic. One natural way to interpolate spatial missing traffic data is to run traffic simulation in the “empty” areas and adopt the simulation results. However, the simulated traffic flows, when propagate to areas with GPS data coverage, need to respect the estimated traffic flows in those areas. In order to minimize the discrepancy between estimated and simulated traffic flows, we need to not only ensure the estimated flows are as accurate as possible but also be able to dynamically tune traffic simulation so that the resulting simulated flows will respect the estimated flows. These are the topics of the next chapter.

CHAPITRE 3:  CITY-SCALE TRAFFIC ANIMATION USING STATISTICAL LEARNING AND METAMODEL-BASED OPTIMIZATION

Introduction

Many efforts have been invested in digitalizing and visualizing urban environments, for example, software tools like Google Maps and Virtual Earth. As new technologies like VR systems Wang et al., (2005); MIT, (2011); TAC, (2017) and autonomous vehicles emerge, there is an increasing demand to incorporate realistic traffic flows into virtualized cities. The potential applications range from virtual tourism, networked gaming, navigation services, urban design, to training of autonomous driving. The ability to reconstruct city-scale traffic from mobile sensor data can enable the visualization and animation of realistic real-world traffic conditions, thus contributes to those applications.

Traditional traffic data collection methods (e.g., in-road sensors such as loop detectors and video cameras) are costly; new and cheaper data sources such as GPS devices are becoming increasingly ubiquitous. Especially, taxicabs and shared ride services (e.g., Uber and Lyft) are systematically equipping their car fleets with these devices. As a result, GPS traces are part of the most promising data sources to estimate citywide traffic conditions, attributing to their broad coverage. However, GPS data usually contain a low-sampling rate, i.e., the time difference between two consecutive points is large (e.g., greater than 60 seconds). This can cause difficulty to the inference of the actual traversed path of a vehicle, since there are may be multiple paths for connecting the two far-apart points in a complex urban environment. In addition, GPS data exhibit spatial-temporal sparsity, i.e., data can be scarce in certain areas (e.g., suburbs) and time periods (e.g., early-morning hours), which makes city-scale traffic reconstruction challenging.

While it is already challenging to reconstruct traffic conditions in areas with GPS data coverage, in order to reconstruct city-scale traffic, we need to reconstruct traffic conditions in areas without GPS data coverage as well. In theory, the local traffic in these areas can be approximated using traffic simulation. However, it is critical to ensure the consistency of traffic flows on the boundaries of areas with and without GPS data. This requires that traffic simulation must be dynamically tuned to ensure the matching flows with the reconstructed traffic in regions with heavy and complete GPS data coverage.

In general, reconstructing city-scale traffic dynamics using GPS data presents a number of challenges: 1) processing the available data, 2) coping with insufficient data coverage, and 3) reconstructing local traffic flows that are consistent with the global traffic dynamics at large spatial-temporal scale. In order to address these challenges, I propose a systematic approach which takes a digital map (processed using the technique from Wilkie et al. Wilkie et al., (2012)) and GPS data (collected from taxicabs in San Francisco Piorkowski et al., (2009)) as input, and reconstructs the city-scale traffic using a two-phase process. In the first phase, I reconstruct and progressively refine estiamted traffic flows on individual road segments from the sparse GPS data using statistical learning, optimization, map-matching Quddus and Washington, (2015), and travel-time estimation Li et al., 2017a techniques. In the second phase, I use a metamodel-based simulation optimization to efficiently propagate the reconstructed results from the previous phase, along with a microscopic simulator Krajzewicz et al., 2012a to dynamically interpolate missing traffic data. To ensure that the reconstructed traffic is correct, I fine tune the simulation flows with respect to city-wide boundary (traffic) constraints and the estimated flows from the first phase, which objective is enforced through an error approximation of the traffic flow computed using my novel metamodel-based formulation. In summary, I address the problem of learning-based traffic animation and visualization using GPS data with the following contributions:

• —

Accurate estimation of traffic conditions in areas with GPS data coverage using statistical learning;

• —

Dynamic interpolation of traffic conditions in areas without GPS data coverage using metamodel-based simulation optimization;

• —

City-scale traffic reconstruction for traffic animation and visualization.

The rest of this chapter is structured as follows. I survey related work in traffic reconstruction and simulation in Section 3.2. A general overview of my framework pipeline is provided in Section 3.3. I detail the two phases of my approach in Sections 3.4 and Section 3.5, respectively. I present results for reconstruction and simulation, as well as system validation and application demonstration in Section 3.6. Finally, I conclude and discuss future work in Section 3.7.

Related Work

The modeling of urban environments has received considerable attention in recent years. Many systems have been developed to describe a variety of aspects including layout, vehicle traffic, and pedestrian motion Thomas and Donikian, (2000); Willemsen et al., (2006); Musialski et al., (2013); Li et al., (2016); Garcia-Dorado et al., (2017). Regarding vehicle motion, a number of driving simulators and vehicle behavioral models have been proposed Wang et al., (2005); MIT, (2011); TAC, (2017), with several designed for virtual reality systems Kuhl et al., (1995); Bayarri et al., (1996); Wang et al., (2005). Many advancements have also been made in visualizing traffic Andrienko and Andrienko, (2006); Ferreira et al., (2013); Wang et al., 2014b , and developing open-world video games (e.g., Grand Theft Auto, Burnout Paradise, and Watch Dogs).

Traffic simulation has received renewed interest in the past decade, for example, with the introduced concept of “virtualized traffic” van den Berg et al., (2009); Sewall et al., 2011a , and the increasing efforts towards modeling realistic traffic flows Wilkie et al., (2015). Some recent developments include an extension of macroscopic models to generate detailed animations of traffic flows Sewall et al., (2010), and their integration with existing microscopic models to produce traffic animation on urban road networks Sewall et al., 2011b . To provide some other examples, Shen and Jin Shen and Jin, (2012) and Mao et al. Mao et al., (2015) have improved existing microscopic models to produce believable traffic animations. A technique from texture synthesis has been proposed to enhance the visual quality of traffic flows Chao et al., (2018). Characterization of heterogeneous vehicle types Lin et al., (2016) and driver personalities Lu et al., (2014) have also been explored.

Real-world data have been used to calibrate simulated traffic flows. Recent studies, including van den Berg et al. van den Berg et al., (2009), Sewall et al. Sewall et al., 2011a , and Wilkie et al. Wilkie et al., (2013), have explored in-road sensors to reconstruct the traffic flow; Chao et al. Chao et al., (2013) acquired individual vehicle characteristics from video cameras; Bi et al. Bi et al., (2016) adopted a data-driven method to enrich the lane-changing behaviors of traffic simulations. Finally, Garcia-Dorado et al. Garcia-Dorado et al., (2014) endowed users the flexibility to assign desired vehicular behaviors to a road network.

Data-driven modeling has been studied by many researchers in the context of multi-agent simulation Lerner et al., (2007); Lee et al., (2007); Ju et al., (2010); Charalambous and Chrysanthou, (2014); Li et al., (2015). There are however two main distinctions between these studies and our work. The first distinction is the scale: while data-driven crowds are often limited to a few hundreds of individuals, city-scale traffic reconstruction concerns tens of thousands of vehicles. The second distinction is the nature of the data: in data-driven crowds, at least several positions and joint angles are recorded per second for each agent, while mobile traffic data is often very sparse and individual trajectories are not known and cannot be assumed.

Traffic reconstruction has drawn much attention in the field of transportation engineering Kachroo and Sastry, (2016). In order to achieve high reconstruction accuracy, multiple data sources and traffic simulation models have been investigated Work et al., (2010); Li et al., (2014); Perttunen et al., (2015). While significant results have been achieved, these methods along with other projects such as Mobile Century Herrera et al., (2010) are largely restricted to highway segments with lengths of a few kilometers. In order to expand reconstructions to arterial roads and surface streets, recent studies Kong et al., (2013); Castro et al., (2012); Zhang et al., (2013) have adopted GPS data for the reconstruction task. However, due to the uncertainty and sparsity embedded in GPS data, several processing steps are required and are usually executed sequentially.

The first processing step is map-matching, which addresses off-the-road GPS points and infers the truly traversed path of a vehicle. One of the commonly used techniques for map-matching is the shortest-distance criterion Hunter et al., (2014); Quddus and Washington, (2015). Unfortunately, this criterion can introduce errors in congested environments, where the shortest-distance path differs from the shortest travel-time path, and the latter is preferred by GPS devices and most drivers Tang et al., 2016b ; Hunter, (2014); Li et al., 2017a . Although the shortest travel-time criterion is a more reasonable assumption, finding the shortest travel-time path requires accurate estimations of traffic conditions, which are difficult to obtain due the lack of traffic monitoring sensors on arterial roads.

The second processing step, travel-time estimation, tries to distribute the time difference between consecutive GPS points to the road segments of a map-matched path. To list a few examples, Hellinga et al. Hellinga et al., (2008) developed an analytical solution based on empirical observations of real-world traffic patterns. Rahmani et al. Rahmani et al., (2015) took a non-parametric approach and adopted a kernel-based estimator. Other approaches were developed based on probability theory Herring et al., (2010); Hunter, (2014). While promising, these methods suffer from the inherent limitation of a sequential pipeline, in which errors from map-matching will be carried over to its following procedures. Especially, in a congested network, if we use the shortest-distance criterion for map-matching, we will likely to have a wrong map-matched path causing the aggregate travel time to be distributed to a wrong set of road segments. The main difference between my approach for addressing GPS data and the previous approaches Tang et al., 2016b ; Hunter, (2014); Li et al., 2017a is that I take an iterative perspective combining map-matching and its follow-on process travel-time estimation such that the errors generated from either step will get gradually attenuated.

My entire framework differs from previous efforts mainly due to the challenges brought by the traffic reconstruction task on arterial roads (which constitute the majority of a city) than that on highways and major roads. On highways and major roads, in-road sensors are commonly found, which can provide accurate and complete traffic measurements. The modeling of traffic flows on major roads is also easier to suffice macroscopic traffic features because of the limited branching and merging of the major roads. Due to these features, many state-of-the-art simulation techniques are developed base on accurate traffic data and macroscopic traffic assumptions van den Berg et al., (2009); Wilkie et al., (2013). However, these features do not appear on arterial roads: only noisy GPS data are available and macroscopic traffic assumptions are difficulty to maintain Kong et al., (2013).

My framework addresses these issues by first conducting statistical learning on GPS data to reconstruct traffic in areas with rich GPS data coverage, then dynamically completing the reconstruction in areas where GPS data are insufficient or missing. Although efficient and large-scale traffic simulation techniques have been developed Sewall et al., (2010); Sewall et al., 2011b , they are not designed to ensure simulation fidelity and flow consistency across the areas with and without GPS data coverage. My approach, in contrast, can satisfy city-wide boundary constraints and derive consistent traffic reconstruction at a city scale.

Overview

Here, I provide an overview of my approach and define the notation used in this chapter.

System

My system enables city-scale traffic visualization and animation through two phases: 1) an initial traffic reconstruction phase (center-left column in Figure 3.1, and Section 3.4), which estimates initial traffic flows on all road segments of a road network, and 2) a dynamic data completion phase (center-right column in Figure 3.1, and Section 3.5), which yields a dynamic and much more accurate reconstructed traffic at the city scale.

For the Initial Traffic Reconstruction (Section 3.4) phase, a main objective is to reconstruct road-segment flows from the input data. As mentioned before, this problem presents two challenges, which I will address in order.

The first one is the sparseness of GPS data, in which consecutive data-points are on average 60 seconds apart. To cope with this issue, I embed map-matching Quddus and Washington, (2015) and travel-time estimation Li et al., 2017a into an iterative process (top and bottom boxes, center-left column in Figure 3.1, and Section 3.4.2), where the output of one is treated as the input of the other and vice-versa. This process progressively refines the estimation of travel times on road segments and is initialized with a starting estimation obtained through a “naïve” optimization (left box, center-left column in Figure 3.1, and Section 3.4.1). The results of this iterative process are estimated traffic flows for all road segments in areas with GPS data coverage.

The second challenge is the uneven coverage of GPS data. In order to reconstruct traffic flows in areas with little to no traffic data, I perform a bilevel optimization (right box, center-left column in Figure 3.1, and Section 3.4.3). The main principle of this phase is to use existing data (either outdated traffic data or estimated results) that is very coarse, and update it using new information (i.e., the reconstruction from the previous phase). The results of this phase are estimated traffic flows for road segments in areas without GPS data coverage. Note however that in these areas, flows are only accurate at a large scale but not at the scale of individual road segments. Additionally, the reconstruction is static and does not account for interactions among individual vehicles.

The Dynamic Data Completion (Section 3.5) phase addresses the abovementioned issues by leveraging traffic simulation for data-deficient areas. However, the simulation traffic may not respect boundary conditions and previously reconstructed traffic in data-rich regions during propagation. To address this issue, I allow the simulation algorithm to change the “turning ratios” at intersections, i.e., deciding how traffic would distribute itself to the downstream road segments at an intersection. The objective is then finding a set of optimal turning ratios such that the discrepancy between simulated flows and estimated flows is minimized.

One way to derive the optimal turning ratios is to run simulation-based optimization, which can be cost prohibitive due to the use of the microscopic simulator in optimization. I accelerate this procedure by approximating the cost function with a metamodel Osorio et al., (2015) (top box, center-right column in Figure 3.1), which can make the optimization problem more tractable. The optimization scheme is initialized randomly (left box, center-right column in Figure 3.1, and Section 3.5.2). The resulting turning ratios of the optimization are adopted lastly for obtaining various visualization results (right box, center-right column in Figure 3.1, and Section 3.5.2).

Notation

A road network is represented as a directed graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$, in which the nodes $\mathcal{V}$ and edges $\mathcal{E}$ represent intersections (or terminal points) and road segments, respectively. A path on the road network is denoted as a set of road segments $k\in\mathcal{K}$ where $\mathcal{K}$ represents the set of all paths.

Geographically, a city can be divided into many traffic analysis zones (TAZs) based on socio-economic data. The centroids of these TAZs are considered traffic-flow origins and destinations. The sets of origins and destinations are respectively denoted as $\mathcal{O}\subseteq\mathcal{V}$ and $\mathcal{D}\subseteq\mathcal{V}$. The traffic flow is considered to take place between origin-destination pairs (OD pairs), and the average flow from $o\in\mathcal{O}$ to $d\in\mathcal{D}$ during a certain time interval is noted $u_{od}$ (in accordance with transportation engineering literature, this flow is the average number of vehicles). With $\mathcal{K}_{od}$ as the set of paths connecting $o$ and $d$, the flow on path $k\in\mathcal{K}_{od}$ is $u(k)$. The flow inside an OD pair is a summation of flows on all paths in this pair:

[TABLE]

The flow $f(e)$ on a road segment $e\in\mathcal{E}$ is defined as the summation of all path flows traversing $e$:

[TABLE]

where $\delta_{e}^{k}$ is $1$ if path $k$ contains road $e$, and [math] otherwise. This means that $\forall e\in\mathcal{E},\forall k\in\mathcal{K}_{od}$, if $e\in k$, then $e$ contributes a certain portion $p_{e,od}$ of $u_{od}$. By arranging the flows of all OD pairs in a vector $\mathbf{u}=[u_{od}]^{\intercal}_{(o,d)\in\mathcal{O}\times\mathcal{D}}$, and the contributed flow portions of a road segment $e$ as $\mathbf{P}_{e}=[p_{e,od}]_{(o,d)\in\mathcal{O}\times\mathcal{D}}$, the following relation can be derived:

[TABLE]

With all road-segment flows in a network $\mathbf{f}_{\mathcal{E}}=[f(e)]_{e\in\mathcal{E}}^{\intercal}$ and the portions of all road segments $\mathbf{P}_{\mathcal{E}}=[\mathbf{P}_{e}]_{e\in\mathcal{E}}^{\intercal}$, the general relation between flows in road segments and OD pairs can be obtained:

[TABLE]

with $\mathbf{P}_{\mathcal{E}}$ termed assignment matrix.

I list other notation used in this chapter as follows:

• —

$\mathcal{S}$ denotes the GPS data, then $\forall s\in\mathcal{S}$ contains a longitude, a latitude, and a timestamp; a pair of successive GPS data points is noted $(s_{1},s_{2})\in\mathcal{S}_{p}$, where $\mathcal{S}_{p}$ represents the set of all pairs of consecutive GPS data points; $\mathcal{K}_{s_{1},s_{2}}$ denotes the set of all paths connecting $s_{1}$ to $s_{2}$.

• —

$t$ denotes travel time; for instance, $t(e)$ is the travel time of a road segment $e\in\mathcal{E}$, $t(k)=\sum_{e\in k}t(e)$ is the travel time of a path $k\in\mathcal{K}$, and $t(s_{1},s_{2})$ is the travel time between a pair of two consecutive GPS data points $(s_{1},s_{2})\in\mathcal{S}_{p}$ given by their timestamps.

City-Scale Traffic Reconstruction

There are three steps in the initial traffic reconstruction phase: 1) initial estimation of travel times on road segments, 2) iterative refinement of these travel times, and 3) bilevel optimization for filling data-lacking areas.

Initial Estimation

For the initial estimation of travel times, I use Wardrop’s principle Wardrop, (1952), which states that traffic will arrange itself in congested networks such that no vehicle can reduce its travel cost by switching routes. This state is termed user equilibrium and is a result of every user non-collaboratively attempting to minimize their travel times. While the actual traffic may not form an exact user equilibrium, this state can serve as an approximation to real-world traffic Hato et al., (1999) and motivate the generation of optimization constraints.

Following this principle, for a pair of consecutive GPS data points $(s_{1},s_{2})\in\mathcal{S}_{p}$, the travel time between them $t(s_{1},t_{2})$ is the minimum travel time of all paths connecting the two points: $\forall k\in\mathcal{K}_{s_{1},s_{2}},t(k)\geq t(s_{1},s_{2})$. Thus, the travel times of the road segments should satisfy the following “Wardrop” constraints $\mathcal{W}$:

[TABLE]

Additionally, the travel time of a road segment $e\in\mathcal{E}$ is bounded by $t_{min}(e)$ and $t_{max}(e)$, which represent free-flow travel time (at $120\%$ of the road segment’s speed limit111The threshold is set to be $120\%$ because of two reasons: 1) it regulated the range of traffic flows, and 2) by examining map-matched GPS data, traffic flow speed that exceeds $120\%$ is rare.) and travel time at jam density (at $0.5\leavevmode\nobreak\ m.s^{-1}$), respectively.

In order to derive a solution with respect to typical traffic patterns, I pose a regularization term, $\mathcal{R}$, on $\mathbf{t}_{\mathcal{E}}=[t(e)]_{e\in\mathcal{E}}^{\intercal}$ to model the correlation in traffic patterns of road segments in close proximity Sheffi, (1985); Zheng et al., (2011); Wang et al., (2016). $\mathcal{R}$ is formed as a 2-dimensional fused Lasso penalty Tibshirani et al., (2005), in which each row represents a pair of road segments connecting at a node. The entries corresponding to the pair of road segments are set to -1 and 1, accordingly. As an example, the $\mathcal{R}$ of a three-way intersection with road segments $\{e_{1},e_{2},e_{3}\}$ takes the form: $\mathcal{R}=\{[1,1,0]^{\intercal},[-1,0,1]^{\intercal},[0,-1,-1]^{\intercal}\}$. $\mathcal{R}$ is set via enumerating all pairs of road segments at each $v\in\mathcal{V}$, which results in $\mathcal{R}\in\mathbb{R}^{m\times n}$, where $m=\sum_{v\in\mathcal{V}}{deg(v)\choose 2}$, $n=|\mathcal{E}|$, and $deg(v)$ is the total degree of $v$. The initial estimation $\bar{\mathbf{t}}_{\mathcal{E}}$ of travel times is formed as follows:

[TABLE]

Iterative Estimation

I refine $\bar{\mathbf{t}}_{\mathcal{E}}$ with an iterative process that alternates between map-matching and travel-time estimation. This design is based on the observation that many approaches take a sequential perspective Kong et al., (2013); Zhang et al., (2013); Hunter, (2014); Li et al., 2017a . As a consequence, the bias generated during map-matching (especially under the shortest-distance criterion) will get cascaded into travel-time estimation, thus affecting the overall estimation accuracy. By resorting to an iterative process, as newly estimated travel times get more accurate, so does the map-matching, and vice-versa.

3.4.2.1 Map Matching

I adopt a simple strategy to compute the “true” path $\bar{k}_{s_{1},s_{2}}$ between a pair of successive GPS data points $(s_{1},s_{2})\in\mathcal{S}_{p}$. Assuming $\mathcal{Q}_{1}$ and $\mathcal{Q}_{2}$ to denote the sets of candidate positions on the map for $s_{1}$ and $s_{2}$, respectively. This step returns $\mathcal{N}=\{\bar{k}_{s_{1},s_{2}}\}_{\forall(s_{1},s_{2})\in\mathcal{S}_{p}}$, where:

[TABLE]

3.4.2.2 Travel Time Estimation

Given the paths $\mathcal{N}$ assigned to each pair of successive GPS data points by the map-matching step, the next step is to estimate a more accurate set of travel times $\bar{\mathbf{t}}_{\mathcal{E}}$ on individual road segments.

The travel time of a road segment is modeled to follow a probability distribution $\bar{t}(e)\sim\pi_{e},\forall e\in\mathcal{E}$, which is parameterized by $\boldsymbol{\theta}=\{\theta_{e}\}_{e\in\mathcal{E}}$ learned through maximum likelihood estimation (MLE):

[TABLE]

where $\mathcal{L}$ is the likelihood function. Following the methodology from Hofleitner et al. Hofleitner et al., 2012b , I assume that the travel time of a road segment can be modeled by a univariate distribution, and the travel-time distributions of road segments are pairwise independent. Using these assumptions, Equation 3.8 is solved via expectation maximization (EM). The maximization step within the M-step is conducted for each road segment:

[TABLE]

where $\pi_{e}$, $\theta_{e}$, $\Omega$, $t_{\omega}$, and $w_{\omega}$ are as follows:

$\bullet$ $\pi_{e}$, $\theta_{e}$:

Following Hunter et al. Hunter, (2014), $\pi$ is taken to be the Gamma distribution $\Gamma$, which has a positive domain and is robust to long-tail observations. These features make $\Gamma$ suitable for modeling the travel time of a road segment. Consequently, $\theta_{e}=(\alpha_{e},\beta_{e})$ where $\alpha_{e}$ is the shape and $\beta_{e}$ is the scale.

$\bullet$ $\Omega$:

As a road segment can be part of multiple paths, which link different pairs of successive GPS data points, samples regrading each pair must be included in Equation 3.9. In this work, I compute 100 samples for each pair: $\Omega=\{(s_{1},s_{2},i)\}_{\forall(s_{1},s_{2})\in\mathcal{S}_{p},\forall i\in[\![1,100]\!]}$.

$\bullet$ $t_{\omega}$:

Using the previous choices for $\pi_{e}$, $\theta_{e}$ and $\Omega$, $t_{\omega}$ is computed as follows according to Hunter et al. Hunter, (2014):

[TABLE]

$\bullet$ $w_{\omega}$:

A weight $w_{\omega}(e)$ is then computed as the distance between $t_{\omega}(e)$ and $A_{\omega}(e)$.

After solving Equation 3.9, a new estimation of $\bar{\mathbf{t}}_{\mathcal{E}}$ is computed, where for each road segment $e\in\mathcal{E}$, $\bar{t}(e)$ is computed as the mean of $\pi_{e}$. From here, the iterative estimation loops back to the map-matching step. The entire process stops after 10 iterations (determined empirically), with the refined $\bar{\mathbf{t}}_{\mathcal{E}}$ as the output.

Bilevel Optimization

The previous steps have estimated travel times $\bar{\mathbf{t}}_{\mathcal{E}}$ of road segments with GPS data coverage. The objective of this step is to compute travel times of all road segments (including those without data coverage).

To proceed, $\bar{\mathbf{t}}_{\mathcal{E}}=[\bar{t}(e)]_{e\in\mathcal{E}}^{\intercal}$ is converted to $\bar{\mathbf{f}}_{\mathcal{E}}=[\bar{f}(e)]_{e\in\mathcal{E}}^{\intercal}$ by inverting the road-segment performance function proposed by the U.S. Bureau of Public Roads:

[TABLE]

where $c(e)$ is the capacity computed as follows (the formula is accessible at http://www.fhwa.dot.gov):

[TABLE]

Next, the target OD pairs $\bar{\mathbf{u}}$ are derived based on the ratio of estimated flows to the loop-detector measurements on the same road segment Yang et al., (2017). Having $\bar{\mathbf{f}}_{\mathcal{E}}$ from the previous steps, traffic flows of all road segments $\hat{\mathbf{f}}_{\mathcal{E}}$ and the corresponding flows between OD pairs $\hat{\mathbf{u}}$ can be computed through the following minimization Cascetta and Nguyen, (1988):

[TABLE]

where $\mathcal{F}_{1}$ and $\mathcal{F}_{2}$ are generalized distance functions. $\mathcal{M}$ is the assignment map which determines $\mathbf{P}_{\mathcal{E}}$. If $\mathcal{M}$ follows Wardrop’s principle Wardrop, (1952), Equation 3.13 becomes a bilevel optimization problem Yang et al., (1992): the upper level minimizes the distances of estimated OD pairs and traffic flows to their corresponding measurements, while the lower level satisfies the user equilibrium. For $\mathcal{F}_{1}$ and $\mathcal{F}_{2}$, I select the generalized least squares (GLS) estimator Bera and Rao, (2011), as it permits different weighting schemes of $\bar{\mathbf{u}}$ and $\bar{\mathbf{f}}_{\mathcal{E}}$. I further assume that $\bar{\mathbf{u}}$ and $\bar{\mathbf{f}}_{\mathcal{E}}$ are results from the following stochastic system of equations:

[TABLE]

Using these choices, Equation 3.13 can be explicitly written as:

[TABLE]

where $U$ and $V$ are variance-covariance matrices of $\mathbf{\epsilon}_{1}$ and $\mathbf{\epsilon}_{2}$, respectively; $\mathbb{E}(\mathbf{\epsilon}_{1})=0$ and $\mathbb{E}(\mathbf{\epsilon}_{2})=0$ are derived from experiments by Cascetta and Nguyen Cascetta and Nguyen, (1988); $\eta\in[0,1]$ is the weighting factor. When $\eta=1$, $\bar{\mathbf{f}}_{\mathcal{E}}$ is ignored and the estimation is solely based on $\bar{\mathbf{u}}$; when $\eta=0$, the estimation is solely based on $\bar{\mathbf{f}}_{\mathcal{E}}$. Equation 3.15 can be solved iteratively: in the upper level the GLS estimator is solved using quadratic programming; in the lower level user equilibrium is approximated using the one-shot function within SUMO Krajzewicz et al., 2012a . As a result of this procedure, I can use the newly computed $\hat{\mathbf{f}}_{\mathcal{E}}$ on road segments where GPS data are unavailable.

Dynamic Data Completion

For detailed traffic simulation, I adopt a microscopic simulation algorithm. To ensure that the simulation for any given area would respect flows at the boundaries from the previously reconstructed traffic, I allow the algorithm to alter the turning ratios at intersections (which decide how traffic would distribute itself to the downstream road segments at an intersection). I choose the turning ratios as optimization variables because of the following reasons: 1) turning ratios implicitly encode traffic light logic, 2) driving behaviors such as lane changing are limited at intersections, and 3) detailed road information is difficult to obtain. In most microscopic traffic simulators, specifying turning ratios is one of the main mechanisms to start a simulation. Furthermore, the split flows can be used to form a metamodel and compute estimation errors from the previous reconstructed result. My algorithm can also cope with multivariable optimization if the information of other simulation parameters is provided.

I denote the vector of all turning ratios by $\mathbf{x}=[x_{v,e}]_{\forall e\in v,\forall v\in\mathcal{V}}$, and name it a decision point. In order to systematically derive a decision point $\mathbf{x}^{*}$ in a simulation region, which not only meets OD demands but also conforms the previous estimated traffic conditions, I rely on the following optimization task:

[TABLE]

where $\bar{F}$ is the objective function, and $F$ is a stochastic network performance measure. The distribution of $F$ depends on the decision point $\mathbf{x}$ and exogenous parameters $\rho$, which record a network topology and road-segment metrics. Every simulation run with $\mathbf{x}$ is a realization of $F$, which involves sampling many other distributions that account for the stochastic nature of traffic (e.g., driver differences). Assuming $r$ independent simulations with a given $\mathbf{x}$ are executed, the objective function can be approximated:

[TABLE]

However, using a simulation algorithm in optimization can be very costly, especially consider the details a microscopic traffic simulator models and the scale it is applied to. This motivates me to adopt metamodel-based simulation optimization for efficiency.

Metamodel-Based Simulation Optimization

A metamodel can simplify simulation-based optimization, as it is typically a deterministic function rather than a stochastic simulator. Therefore, one way to circumvent the issues of using a microscopic traffic simulator in a simulation-optimization loop is to develop a deterministic metamodel to replace the stochastic simulation response. The metamodel is usually less realistic in terms of the modeling capability, but much cheaper to evaluate.

The most common metamodels are functional metamodels, which are general-purpose functions and can be used to approximate arbitrary objective functions. Often, they are results of a linear combination of basis functions such as low-order polynomials, spline models, and radial basis functions Conn et al., (2009). However, they require a large number of decision points to be fitted, since the structure of an underlying problem is not considered. This means that I need to run the simulator many times on various decision points in order to fit a well-performing metamodel. This procedure is expensive and to a certain degree defeats the purpose of using a metamodel. Instead, I use a metamodel that contains not only a functional component but also a physical component which encodes the underlying problem for achieving high efficiency.

I build this physical component based on the classical flow conversion equation, also known as the traffic equation Osorio, (2010):

[TABLE]

where $\mathcal{C}$ represents the set of road segments in the simulation region, $\gamma(e_{1})$ is the external flow injected into road segment $e$, and $p(e_{1},e_{2})$ is the transition probability from road segment $e_{1}$ to road segment $e_{2}$. The exogenous parameters in Equation 3.18 are external flows and transition probabilities. Having them, both the traffic equation and the traffic simulator can be executed to obtain the flows of all road segments in a simulation region. Denoting the subset of road segments with estimated traffic flows from GPS data as $\mathcal{A}\in\mathcal{C}$, the estimated flows as $\mathbf{f}^{E}=\{f(e)^{E}\}_{e\in\mathcal{A}}$, the propagated flows using the traffic equation as $\mathbf{f}^{T}=\{f(e)^{T}\}_{e\in\mathcal{A}}$, and the simulated flows using a traffic simulator as $\mathbf{f}^{S}=\{f(e)^{S}\}_{e\in\mathcal{A}}$, the approximation of the objective function derived from the simulator with one simulation run ($r=1$ in Equation 3.17) is defined as follows:

[TABLE]

The approximation of the objective function derived from the traffic equation is defined as follows:

[TABLE]

The metamodel is then constructed as a combination of the physical component $T$ and a functional component $\Phi$:

[TABLE]

where $\alpha$ (initially set to 0.5) and $\boldsymbol{\beta}$ (initially set to $\boldsymbol{1}$) are parameters of the metamodel. The functional component, $\Phi$, is chosen to be a quadratic polynomial Osorio and Bierlaire, (2013):

[TABLE]

where $|\mathbf{x}|$ is the dimension of $\mathbf{x}$; $x_{i}$ and $\beta_{i}$ are the $i$th elements of $\mathbf{x}$ and $\boldsymbol{\beta}$, respectively. The quadratic polynomial provides Taylor-type bounds, serves as a general term within a metamodel formulation, and ensures global convergences Conn et al., (2009). In order to fit the metamodel, I rely on the decision points that have been evaluated using both the traffic equation and the traffic simulator. Denoting $\mathcal{X}$ as the pool of the decision points, the metamodel can be fit (i.e., compute $\alpha$ and $\boldsymbol{\beta}$) by solving:

[TABLE]

where $w_{0}$ is a fixed constant. $w_{i}$ is the weight associating each $\mathbf{x}_{i}$ and a new decision point $\mathbf{x}_{new}$ at each iteration during the optimization, computed as $w_{i}=1/(1+\lVert\mathbf{x}_{new}-\mathbf{x}_{i}\rVert_{2})$, representing the inverse distance Atkeson et al., (1997). The first term in Equation 3.23 represents the weighted distance between simulated results and estimated results. The remaining terms in Equation 3.23 guarantee the least square matrix to have a full rank.

Algorithmic Steps

Following the framework proposed by Conn et al. Conn et al., (2009) and its adaptation in Osorio and Bierlaire Osorio and Bierlaire, (2013), I combine the metamodel with the derivative-free trust-region algorithm to solve the optimization problem in Equation 3.16, which can now be expressed at any given iteration as follows:

[TABLE]

where $\mathbf{x}^{*}$ is the best decision point so far and $\Delta$ is the current trust-region radius.

The specific algorithmic steps are the following:

• —

Step 1: Initialize $\mathcal{X}$ to contain 5 randomly sampled decision points, evaluate each of them using both the traffic equation and simulator, arbitrarily set $\mathbf{x}^{*}$ as a any element of $\mathcal{X}$, and compute $\alpha$ and $\boldsymbol{\beta}$ (Equation 3.23).

• —

Step 2: Use Equation 3.24 to compute $\mathbf{x}_{new}$.

• —

Step 3: Compute $\hat{F}(\mathbf{x}_{new};\rho)$ (simulator run). Compute the relative improvement $\tau=\frac{\hat{F}\left(\mathbf{x}_{new}\right)-\hat{F}\left(\mathbf{x}^{*}\right)}{M_{i}\left(\mathbf{x}_{new}\right)-M_{i}\left(\mathbf{x}^{*}\right)}$. If $\tau\geq 1e-3$, accept $\mathbf{x}_{new}$ and set $\mathbf{x}^{*}:=\mathbf{x}_{new}$, otherwise reject $\mathbf{x}_{new}$. In any case, add $\mathbf{x}_{new}$ to $\mathcal{X}$, and compute $\alpha$ and $\boldsymbol{\beta}$ (Equation 3.23).

• —

Step 4: If $\alpha$ and $\boldsymbol{\beta}$ have not changed much in step 3, i.e., $\frac{\lVert(\alpha_{new},{\boldsymbol{\beta}}_{new})-(\alpha_{old},{\boldsymbol{\beta}}_{old})\rVert}{\lVert(\alpha_{old},{\boldsymbol{\beta}}_{old})\rVert}\leq 0.1$, add a new randomly sampled decision point to $\mathcal{X}$, evaluate it using both the traffic equation and simulator, and compute $\alpha$ and $\boldsymbol{\beta}$ (Equation 3.23).

• —

Step 5: Update the trust-region radius:

[TABLE]

• —

Step 6: Exit the loop when the maximum number of allowed simulator runs ($20$) is reached; otherwise go to step 2.

When the algorithm stops, $\mathbf{x}^{*}$ can be taken to generate a simulation, thus ending dynamic data completion.

In practice, instead of running this procedure on the entire road network, I adopt a decomposition approach and separate the network into sub-networks, which are going to be modeled independently. For these sub-networks, I consider nodes with no predecessors as artificial origins and nodes without successors as artificial destinations (i.e., locations where vehicles respectively enter and exit a sub-network). Additionally, observing that vehicles rarely navigate in loops, I extract directed acyclic graphs (DAG) from these sub-networks to operate.

Results

In this section, I present evaluations of Initial Traffic Reconstruction and Dynamic Data Completion, and demonstrate the visualization and animation results.

Evaluation of Initial Traffic Reconstruction

I have generated abundant synthetic data for evaluating my approach. These synthetic datasets are produced via traffic models that have been extensively validated using real-world datasets in transportation engineering. Many newly proposed traffic models are evaluated using these models and real-world datasets. I have conducted my experiments in a similar vein. In this section, I provide details on the generation of the synthetic dataset.

3.6.1.1 Road Network and GPS Dataset

The road network (obtained from http://openstreetmap.org/) used in testing is from downtown San Francisco (Figure 3.2), which contains 5407 nodes, 1612 road segments, and 296 TAZs (obtained from https://data.sfgov.org/). The GPS dataset is obtained from the Cabspotting project Piorkowski et al., (2009) (Figure 3.2 LEFT), in which the low-sampling-rate is reflected as the average timestamp difference between consecutive points is approximately 60 seconds.

3.6.1.2 Traffic Conditions via System Optimal Model

I establish the first set of heuristic network travel times by solving the system optimal (SO) model Sheffi, (1985). The SO model addresses the traffic assignment problem by minimizing the entire travel time of a road network:

[TABLE]

The solution to Equation 3.26 is a set of flows and travel times of all road segments of a network. The key input is the OD pairs estimated by first setting $\mathcal{O}=\mathcal{V}$ and then recording the number of in-and-out GPS traces for each TAZ. As GPS traces are usually sampled from a small percentage of the entire traffic population, thus representing a partial network flow, I multiply the estimated OD pairs by 10 constants and solve Equation 3.26 accordingly. As a result, I have constructed 10 network travel times in which the corresponding congestion levels, measured by volume over capacity (VOC) (computed as $\sum_{e\in\mathcal{E}}\frac{f(e)}{c(e)}$), range uniformly from 0.19 to 1.85.

3.6.1.3 Traffic Conditions via Timestamp Model

Heuristic network travel times can also be generated based on GPS timestamps. Using the Cabspotting dataset Piorkowski et al., (2009), I equally distribute the time difference between two consecutive GPS points to all paths that connect them. For road segments that are covered by multiple GPS traces, the average travel times are adopted. Using this approach, I have produced 24 network travel times representing 24 hours in a typical weekday. An example can be seen at Figure 3.2 RIGHT. I refer to this method of generating network travel times as the Timestamp model.

3.6.1.4 Synthetic GPS Traces

Using established network travel times, I can generate synthetic GPS traces in which the true traversed paths and other information are encoded. In order to study the effect of the number of traces used in reconstruction on the estimation accuracy, I have randomly simulated 20 batches of synthetic traces from 50 to 1000 in increments of 50. Each batch contains 30 sets of GPS traces and all set contain 50 traces. As a result, I have generated $315\leavevmode\nobreak\ 000$ traces for each traffic condition and over $10$ million traces in total. A synthetic trace is created by selecting a random source and a target in the network and routing with the shortest travel-time strategy. To mimic features of a real-world GPS dataset, the sampling rate is set to 60 seconds, and all coordinates are perturbed by the Gaussian noise $(0,20)$ in meters Yuan et al., (2010).

3.6.1.5 Evaluation and Comparison

I compare my technique with two state-of-the-art methods, namely Hunter et al. Hunter, (2014) and Rahmani et al. Rahmani et al., (2015). The first method also uses EM algorithm as of the inner loop of my travel-time estimation process. In their work, the number of EM iterations is set to 5 and the number of random allocations per aggregate measurement is set to 100. These settings are reported to produce the highest estimation accuracy by Hunter et al. Hunter, (2014). The second method takes a non-parametric principle, using a kernel-based technique to estimate travel times. The weights used to allocate travel times to individual road segments are set to be the ratio of free-flow travel times among road segments Hellinga et al., (2008).

I set parameters of my nested iterative process as follows: retaining the same settings for the inner loop as from Hunter et al. Hunter, (2014), I empirically set the number of iterations for the outer loop to 10. This setting is based on the results shown in Figure 3.3, where the relationship between the normalized convergence rate and the number of iterations for both types of network travel times is shown. Each datum in the plot is the average value computed using all network travel times across all sets of synthetic GPS traces using either the SO model (6000 trials) or the Timestamp model (14400 trials). The measurement of each trial is the mean square error (MSE) between a recovered and a ground-truth traffic condition (i.e., $\frac{\sum_{e}(t_{e}-\hat{t}_{e})^{2}}{|\mathcal{E}|}$). As can be seen, the convergence rate decreases quadratically as the number of iterations increases and tends to flatten after 10 iterations.

I evaluate my technique using three metrics. The first metric is the error rate of the aggregate travel time across the entire network, computed as $\frac{|\sum_{e}\hat{t}_{e}-\sum_{e}t_{e}|}{\sum_{e}t_{e}},\forall e\in\mathcal{E}$ where $\hat{t}_{e}$ represents an estimated travel time and $t_{e}$ represents a ground-truth travel time. The left diagram of Figure 3.4 TOP shows the results by averaging the experimental outcomes of all network travel times via the SO model. The minimum error rate of my technique is 18%, of Rahmani et al. Rahmani et al., (2015) is 34%, and of Hunter et al. Hunter, (2014) is 48%. Experimenting on network travel times generated via the Timestamp model, the corresponding minimum error rates are 8%, 28%, and 37%, respectively, which are shown in the right diagram of Figure 3.4 TOP. As the number of synthetic GPS traces used in estimation increases, my technique demonstrates consistent advantages in performance over the other two methods.

The second metric is the relative improvement of my technique over the existing methods on travel times of all road segments. I compute this metric based on $MSE=\frac{\sum_{e}(t_{e}-\hat{t}_{e})^{2}}{|\mathcal{E}|}$ as follows:

[TABLE]

where $MSE_{my}$ represents the error between a recovered traffic condition using my technique and the ground-truth traffic condition, and $MSE_{existing}$ represents the error computed using an existing method with the same ground truth. The maximum relative improvements over Hunter et al. Hunter, (2014) and Rahmani et al. Rahmani et al., (2015) under the SO model (shown in the left diagram of Figure 3.4 MIDDLE) are 78% and 97%, and under the Timestamp model (shown in the right diagram of Figure 3.4 MIDDLE) are 54% and 49%. In general, with more synthetic GPS traces used in estimation, better relative improvements are achieved. Such effects are more apparent on the SO model than the Timestamp model.

The third metric evaluates the map-matching accuracy. For one trace, I calculate the success rate as follows:

[TABLE]

I sum all success rates generated using my method and an existing approach, and derive the relative improvement as follows:

[TABLE]

The maximum relative improvements of my method over Hunter et al. Hunter, (2014) and Rahmani et al. Rahmani et al., (2015) under the SO model are 28% and 34%, and under the Timestamp model are 19% and 25%. These results are shown in Figure 3.4 BOTTOM. Again, as the number of GPS traces used in recovering network travel times increases, gains in the improvements are observed.

3.6.1.6 Analysis of Bilevel Optimization

I have shown that my approach outperforms the existing methods on estimating travel times of a road network. In turn, by inverting Equation 3.11, I can obtain better estimations of target flows, which serve as the input to the bilevel optimization program.

The factors affecting the program are the weighting factor $\eta$, the noisy level of target OD pairs $\bar{\mathbf{u}}$, and the noisy level of target road-segment flows $\bar{\mathbf{f}}$. The noises of $\bar{\mathbf{u}}$ and $\bar{\mathbf{f}}$ are assumed to have zero means and diagonal variance-covariance matrices Cascetta and Nguyen, (1988). In practice, the noisy level of $\bar{\mathbf{u}}$ is difficult to assess because not only $\bar{\mathbf{u}}$ usually comes from existing data but also the true values of $\bar{\mathbf{u}}$ are usually unknown. Due to these reasons, in the analysis of $\eta$ and the noisy level of $\bar{\mathbf{f}}$, I set the normalized noisy level of $\bar{\mathbf{u}}$ to $50\%$. Subsequently, the normalized noisy levels of $\bar{\mathbf{f}}$ (%) computed based on MSE of estimated travel times to ground truth are shown in Figure 3.5 TOP and MIDDLE. In general, my technique produces lower noisy levels of $\bar{\mathbf{f}}$ than the other two techniques, especially under the SO model which is considered to be a better approximation to real-world traffic than the Timestamp model Sheffi, (1985).

In order to evaluate how $\eta$ and the noisy level of $\bar{\mathbf{f}}$ affect the estimation accuracy of $\hat{\mathbf{u}}$ (i.e., the estimated OD pairs), I compute the normalized MSE of $\hat{\mathbf{u}}$ (%) under different $\eta$ and various noisy levels of $\bar{\mathbf{f}}$ (%). The results are shown in Figure 3.5 BOTTOM. When $\eta$ takes a small value (e.g., 0.1), the impact of $\bar{\mathbf{u}}$ is restricted, thus the MSE of $\hat{\mathbf{u}}$ reacts actively to perturbations on $\bar{\mathbf{f}}$. As I gradually increase the value of $\eta$, the impact of $\bar{\mathbf{f}}$ attenuates. Nevertheless, the MSE of $\hat{\mathbf{u}}$ increases as the noisy level of $\bar{\mathbf{f}}$ progresses.

Evaluation of Dynamic Data Completion

I compare my technique to the approach that only uses traffic simulation on various OD demands and road networks. The experiments are conducted by starting at the center of the road network of San Francisco and gradually increase the radius to retrieve networks with $20$ to $5\leavevmode\nobreak\ 000$ road segments. This step results in $48$ road networks. For all DAGs constructed in all networks, the OD demand varies from $1000\leavevmode\nobreak\ vehicle/hour$ to $10000\leavevmode\nobreak\ vehicle/hour$ with an increment of $1000\leavevmode\nobreak\ vehicle/hour$.

In Figure 3.6, I show the accuracy of my technique compared to the approach using only the traffic simulation (i.e., the simulation-only approach). For each road network in the experiment, I randomly select an intersection and assign turning ratios for this intersection. The assigned turning ratios (i.e., a decision point) are treated as the ground truth for the two approaches to recover. Since the most accurate method for recovering a traffic condition is the microscopic simulator, I use the error level of the microscopic simulator to the ground truth as the baseline. For each decision point, I first compute the difference between the recovered value using my technique to the ground truth, as well as the difference between the recovered value by the simulator to the ground truth. Then, I subtract these two error differences and represent the result as a gray cross in Figure 3.6. The mean, minimum, and maximum values of the average deviations to the simulator (indicated by the solid line in Figure 3.6), are respectively 7.8%, 0%, and 13%. In many cases, my technique even outperforms the simulation-only approach with a smaller difference to the ground truth, as indicated by a negative value in Figure 3.6. 3

In the second study, I analyze the performance speedup achieved by my technique over the simulation-only approach. The results are shown in Figure 3.7. Each gray cross indicates a speedup measure for a particular road network and a specific OD demand. The highest speedup is over $90x$. On average, the maximum speedup is about $35.6x$ and the minimum speedup is about $16.1x$. There also exist several cases where my technique demonstrates rather negligible speedups (i.e., values close to 0). This is because the initial random guess of a particular decision point is close to the ground truth, in which circumstances either method can achieve a quick convergence. In reality, I expect these situations happen rarely. Finally, in nearly all experiments, my technique converges in the first five iterations.

Traffic Visualization and Animation

To demonstrate the effectiveness of my approach, I use the Cabspotting dataset Piorkowski et al., (2009) (GPS data) in San Francisco to perform traffic visualization and animation. This dataset recorded driving history from 536 taxicabs, which includes over 11 million GPS traces in total.

In Figure 3.9 (top row), I show the visualization results over four time intervals: Sunday 9AM, Tuesday 9AM, Thursday Noon, and Friday 7PM, which exemplify weekend morning traffic, weekday morning traffic, weekday mid-day traffic, and weekday evening traffic, respectively. In San Francisco, first, the congestion level of Sunday 9AM is low compared to the rest of the time intervals across all areas. Second, the congestion of Tuesday 9AM is more severe in the north-west, central-west, and central-east areas (residential regions) than the same areas in other time intervals. Lastly, the north-east region (downtown commercial and financial districts) of Tuesday 9AM and Friday 7PM appear to have more severe congestion than other time intervals.

In addition to the qualitative results, I have quantitatively compared my reconstructed results to the loop-detector data (obtained from http://pems.dot.ca.gov/) extracted from the same location and time periods as the Cabspotting dataset. The loop-detector data represent relatively accurate measurements and are often regarded as the benchmark for evaluating GPS-based estimations Work et al., (2010).

I have additionally used a filtering process based on the observation that traffic exhibits periodic patterns, which I introduced in Chapter 2 and are shown in Figure 3.9. One way to construct such a filter is to transform all traffic signals to the frequency domain and take the average of all frequency components. While this approach can expect to reduce the embedded white noise, it does not consider other types of noise in the data. This is illustrated in the top panel of Figure 3.8(c): the most significant frequency component is captured while other harmonics are reduced to various degrees.

Considering time-domain only methods, a naïve approach is to get the average signal, transform it to the frequency domain, and analyze its frequency components. Though this is a straightforward way in dealing with univariate and multivariate data, the resulting signal could be a poor summary of original signals both in the time domain (Figure 3.8(a)) and in the frequency domain (Figure 3.8(c) MIDDLE). The reason is that this approach does not take the phase variability into account. To address this, I align signals in the time domain, calculate the average signal, transform it to the frequency domain, and extract its major frequencies (see Figure 3.8(b) and Figure 3.8(c) bottom panel). Using this approach, I obtain important frequency components and their corresponding magnitudes in the right ratio. The frequency-domain version of this signal then serves as my filter.

After applying this filtering process, my reconstruction can approximate the accurate loop-detector readings with small losses (around $1\leavevmode\nobreak\ m/s$) in the speed measurement. This validation result can be found in the bottom row of Figure 3.9.

As a result of the dynamic data completion, I obtain turning ratios that can lead to simulations that respect the estimated traffic conditions in areas with GPS data coverage. The results can be illustrated using both 2D and 3D traffic animation for various virtual-world applications. In Figure 3.10, I provide 2D traffic animation of four regions in San Francisco222The number of lanes of a road segment is decided by the digital map. using the reconstructed traffic on Friday at 7PM. This 2D animation can be used to study dynamic traffic patterns at a metropolitan scale. The downtown area is further modeled in a 3D Virtual San Francisco to showcase the potential of embedding real-world traffic in a virtual world for immersive VR experiences and virtual tourism applications (see Figure 3.11). It is worth mentioning that although I have used the GPS data from one city throughout my experiments, my approach is independent from data sources, as the effectiveness is demonstrated using both synthetic datasets (Figure 3.4) and real-world datasets (Figure 3.9).

Summary and Future Work

I have presented an effective algorithm to reconstruct city-scale traffic from GPS data using statistical learning. To address the issues with incomplete and/or sparse data, a metamodel-based simulation optimization is proposed to dynamically bridge the “gap” between the reconstructed traffic learned from GPS data and the simulated traffic in areas with incomplete or missing data. My approach is able to perform visualization of city-scale traffic, as well as data-driven 2D and 3D traffic animation in a virtual environment. With more GPS datasets being made available and released to public, e.g., Mobile Century Herrera et al., (2010), T-Drive Microsoft, (2010), GeoLife Microsoft, (2009), and Uber Movement Uber, (2017), I believe future research would be abundant.

Although the proposed approach is specialized for traffic reconstruction, similar approaches can be developed to reconstruct aggregate dynamics of other multi-agent systems using spatial-temporal data, such as schools of fishes, flocks of birds, and swarms of insects. More importantly, the idea of learning from mobile sensor data as well as the concept of using metamodel-based optimization to refine the simulation parameters and to accelerate local fitting for large-scale motion reconstruction are generalizable to many other topics in computer graphics and beyond.

The main limitation of this work is that the reconstruction accuracy is limited by the available data. While my approach can maintain the accuracy down to the road-segment level, high frequency vehicle positions can not be modeled precisely, as such information is largely missing from the data. Another limitation of this work is that while my technique can satisfy the flow conservation in each sub-network used in computation, such a relationship is difficult to maintain at a city scale. This issue may be alleviated if more in-road sensors are installed on arterial roads to provide broad and accurate traffic measurements.

There are a number of future directions. Algorithmically, one extension is to combine other data sources (in-road sensors, video streams, or surveying) with GPS traces to further improve the reconstruction accuracy. Another possibility is to incorporate a macroscopic traffic simulator, so that we can dynamically switch between simulators of varying fidelities to further reduce the computational cost. Application-wise, a virtual tourism system, a route planning Wilkie et al., (2011) and navigation system Wilkie et al., (2014), or an autonomous vehicle training system can immediately benefit from my framework through incorporating and visualizing the reconstructed traffic.

With Chapter 2 and Chapter 3, I have finished introducing my efforts on city-scale traffic reconstruction. Traffic is an aggregate form and its dynamics can only be studied collectively. Examining traffic at the scale of a city is necessary if we want to analyze congestion causes, identify network bottlenecks, and experiment with novel transport policies. Furthermore, traffic is form by individual vehicles. If we can improve the safety and efficiency of individual vehicles by converting them from human-driven to autonomous (given over 90% crashes are due to some kind of human errors), together, these autonomous vehicles have the potential to alleviate the severe traffic condition we are facing. Next chapter introduces my efforts on improving autonomous driving with a focus on enabling the autonomous vehicle to navigate in dangerous situations including accidents.

CHAPITRE 4:  ADAPS: AUTONOMOUS DRIVING VIA PRINCIPLED SIMULATIONS

Introduction

Autonomous driving is a complex task, consider the dynamics of an environment and the lack of accurate definitions of various driving behaviors. These characteristics lead to conventional control methods to suffer subpar performance on the task Ratliff, (2009); Silver, (2010). Nevertheless, driving can be easily demonstrated by human experts. This observation has inspired imitation learning, which leverages expert demonstrations to synthesize a controller.

While there are many advantages of using imitation learning, it also has drawbacks. For autonomous driving, the most critical one is covariate shift, meaning the training and test distributions are different. This could lead autonomous vehicles (AVs) to accidents since a learned policy may fail to respond to unseen scenarios including those dangerous situations that do not occur often.

In order to mitigate this issue, the training dataset needs to be augmented to cover a wide spectrum of driving scenarios with expert demonstrations—especially ones of significant safety threats to the passengers—so that a policy can learn how to recover from its own mistakes. This is emphasized by Pomerleau Pomerleau, (1989), who synthesized a neural network based controller for AVs: “the network must not solely be shown examples of accurate driving, but also how to recover (i.e. return to the road center) once a mistake has been made.”

However, obtaining recovery data from accidents in the physical world is impractical, due to the high cost of a vehicle and potential injuries to both passengers and pedestrians. In addition, even one managed to collect accident data, if we need human experts to label them, the process can be inefficient and subject to judgmental errors Ross et al., (2013).

These difficulties naturally lead us to the virtual world, where accidents can be simulated and analyzed much freely Chao et al., (2019). I have developed ADAPS (Autonomous Driving Via Principled Simulations) to achieve this goal. ADAPS consists of two simulation platforms and a memory-enabled hierarchical control policy. The first simulation platform, referred to as SimLearner, runs in a 3D environment and is used to test a learned policy, simulate accidents, and collect training data. The second simulation platform, referred to as SimExpert, acts in a 2D environment and serves as the “expert” to analyze and resolve an accident via principled simulations, which can plan alternative safe trajectories for a vehicle by taking its kinematic and dynamic constraints into account.

Furthermore, ADAPS represents a more efficient online learning mechanism than existing methods such as DAGGER Ross et al., (2011). This is useful consider learning to drive requires iterative testing and update of a control policy. Ideally, we want to obtain a robust policy using minimal online iterations, as one iteration corresponds to one incident (otherwise there is no need to update a policy by going into the next iteration). This would require the generation of training data at each iteration to be accurate, efficient, and sufficient so that a policy can gain a large improvement after this iteration. ADAPS can be of assistance to achieve this goal.

The main contributions of this chapter are specifically. (1) The accidents generated in SimLearner will be analyzed by SimExpert to produce alternative safe trajectories. (2) These trajectories will be automatically processed to generate a large number of annotated and segmented training data. Because SimExpert is parameterized and has taken the kinematic and dynamic constraints of a vehicle into account (i.e., principled), the resulting training data are not only more heterogeneous than the data collected from running a learned policy multiple times, but also more effective than the data collected through random sampling. (3) Both theoretical and experimental results are provided to show that ADAPS is an efficient online learning mechanism.

Related Work

I sample previous studies that are related to each aspect of my framework and discuss the differences within.

Autonomous Driving. Among various methods to plan and control an AV Schwarting et al., (2018), I focus on end-to-end imitation learning as it can avoid manually designed features and lead to a more compact policy compared to conventional mediation perception approaches Chen et al., (2015). The early studies done by Pomerleau Pomerleau, (1989) and LeCun et al. LeCun et al., (2005) have shown that neural networks can be used for an AV to achieve lane-following and off-road obstacle avoidance. Due to the advancements of deep neural networks (DNNs), a number of studies have emerged Bojarski et al., (2016); Xu et al., (2017); Pan et al., (2018); Codevilla et al., (2017). While significant improvements have been made, these results mainly restrict a vehicle to either the lane-following behavior Codevilla et al., (2017) or the off-road collision avoidance LeCun et al., (2005). My policy, in contrast, enables an AV to learn from accidents and allows it to achieve on-road collision avoidance with both static and dynamic obstacles.

Hierarchical Control Policy. There have been many efforts in constructing a hierarchical policy to control an agent at different stages of a task Barto and Mahadevan, (2003). Example studies include the options framework Sutton et al., (1999) and transferable motor skills Konidaris et al., (2012). When combined with DNNs, the hierarchical approach has been adopted for virtual characters to learn locomotion tasks Levine and Koltun, (2013). In these studies, the goal is to discover a hierarchical relationship from complex sensorimotor behaviors. I apply a hierarchical and memory-enabled policy to autonomous driving based on multiple DNNs. My policy enables an AV to continuously categorize the road condition as safe or dangerous, and execute corresponding control commands to achieve accident-free driving.

Generative Policy Learning. Using principled simulations to assist learning can be considered as taking a generative model approach. Several studies have adopted the same principle for deriving the (near-)optimal policy, examples including Function Approximations Gordon, (1995), Sparse Sampling Kearns et al., (2002), and Fitted Value Iteration Szepesvári and Munos, (2005). These studies leverage a generative model to stochastically generate training samples. The emphasize is to simulate the feedback from an environment assuming the reward function is known. My system, on the other hand, does not assume any reward function of a driving behavior but models the kinematic and dynamic constraints of a vehicle, and uses simulation to plan its trajectories with respect to environment characteristics. In essence, my method acquires a control policy through learning from simulated expert demonstrations rather than from an agent’s self-exploration Lin, (1992) as of the previous studies.

Preliminaries

Autonomous driving is a sequential prediction and controlled (SPC) task, for which a system must predict a sequence of control commands based on inputs that depend on past predicted control commands. Because the control and prediction processes are intertwined, SPC tasks often encounter covariate shift, meaning the training and test distributions vary. Next, I will introduce notation and definitions to formulate an SPC task, and briefly discuss its existing solutions. Note that, I use “state” and ”observation” interchangeably for proofs in this and later sections.

The problem being consider is a $T$-step control task. Given the observation $\phi=\phi(s)$ of a state $s$ at each step $t\in[\![1,T]\!]$, the goal of the learner is to find a policy $\pi\in\Pi$ such that its produced action $a=\pi(\phi)$ will lead to the minimal cost:

[TABLE]

where $C\left(s,a\right)$ is the expected immediate cost of performing $a$ in $s$. For many tasks such as driving, we may not know the true value of $C$. Instead, the observed surrogate loss $l(\phi,\pi,a^{*})$ is commonly minimized, which is assumed to upper bound $C$, based on the approximation of the learner’s action $a=\pi(\phi)$ to the expert’s action $a^{*}=\pi^{*}(\phi)$. The distribution of observations at $t$ is denoted as $d_{\pi}^{t}$, which is the result of executing $\pi$ from $1$ to $t-1$. $d_{\pi}=\frac{1}{T}\sum_{t=1}^{T}d_{\pi}^{t}$ is then the average distribution of observations by executing $\pi$ for $T$ steps. My goal is to solve an SPC task by obtaining $\hat{\pi}$—a policy that can minimize the observed surrogate loss under its own induced observations with respect to expert’s actions for those observations:

[TABLE]

I further denote $\epsilon=\mathbb{E}_{\phi\sim d_{\pi^{*}},a^{*}\sim\pi^{*}(\phi)}\left[l\left(\phi,\pi,a^{*}\right)\right]$ as the expected loss under the training distribution induced by the expert’s policy $\pi^{*}$, and the cost-to-go over $T$ steps of $\hat{\pi}$ as $J\left(\hat{\pi}\right)$ and of $\pi^{*}$ as $J\left(\pi^{*}\right)$.

By simply treating expert demonstrations as independently and identically distributed (i.i.d.) samples, the discrepancy between $J\left(\hat{\pi}\right)$ and $J\left(\pi^{*}\right)$ is $\mathcal{O}(T^{2}\epsilon)$ Syed and Schapire, (2010); Ross et al., (2011). Given the error of a typical supervised learning is $\mathcal{O}\left(T\epsilon\right)$, this demonstrates the additional cost due to covariate shift when solving an SPC task via standard supervised learning. I adapt and simplify the proof from Ross et al. Ross et al., (2011) to show that solving an SPC task using standard supervised learning will lead to $\mathcal{O}(T^{2}\epsilon)$ error. This proof will better prepare readers for understanding the guarantees of ADAPS presented in Section 4.4.

Theorem 4.1**.**

Ross et al., (2011) Consider a $T$-step control task. Let $\epsilon=\mathbb{E}_{\phi\sim d_{\pi^{*}},a^{*}\sim\pi^{*}(\phi)}\left[l\left(\phi,\pi,a^{*}\right)\right]$ be the observed surrogate loss under the training distribution induced by the expert’s policy $\pi^{*}$. I assume $C\in\left[0,C_{max}\right]$ and $l$ upper bounds the 0-1 loss. $J\left(\pi\right)$ and $J\left(\pi^{*}\right)$ denote the cost-to-go over $T$ steps of executing $\pi$ and $\pi^{*}$, respectively. Then, I have the following result:

[TABLE]

Démonstration.

In order to prove this theorem, I introduce the following notation and definitions:

• —

$d_{t,c}^{\pi}$: the state distribution at $t$ as a result of the following event: $\pi$ is executed and has been choosing the same actions as $\pi^{*}$ from time $1$ to $t-1$.

• —

$p_{t-1}\in\left[0,1\right]$: the probability that the above-mentioned event holds true.

• —

$d_{t,e}^{\pi}$: the state distribution at $t$ as a result of the following event: $\pi$ is executed and has chosen at least one different action than $\pi^{*}$ from time $1$ to $t-1$.

• —

$(1-p_{t-1})\in\left[0,1\right]$: the probability that the above-mentioned event holds true.

• —

$d_{t}^{\pi}=p_{t-1}d_{t,c}^{\pi}+(1-p_{t-1})d_{t,e}^{\pi}$: the state distribution at $t$.

• —

$\epsilon_{t,c}$: the probability that $\pi$ chooses a different action than $\pi^{*}$ in $d_{t,c}^{\pi}$.

• —

$\epsilon_{t,e}$: the probability that $\pi$ chooses a different action than $\pi^{*}$ in $d_{t,e}^{\pi}$.

• —

$\epsilon_{t}=p_{t-1}\epsilon_{t,c}+(1-p_{t-1})\epsilon_{t,e}$: the probability that $\pi$ chooses a different action than $\pi^{*}$ in $d_{t}^{\pi}$.

• —

$C_{t,c}$: the expected immediate cost of executing $\pi$ in $d_{t,c}^{\pi}$.

• —

$C_{t,e}$: the expected immediate cost of executing $\pi$ in $d_{t,e}^{\pi}$.

• —

$C_{t}=p_{t-1}C_{t,c}+(1-p_{t-1})C_{t,e}$: the expected immediate cost of executing $\pi$ in $d_{t}^{\pi}$.

• —

$C_{t,c}^{*}$: the expected immediate cost of executing $\pi^{*}$ in $d_{t,c}^{\pi}$.

• —

$C_{max}$: the upper bound of an expected immediate cost.

• —

$J\left(\pi\right)=\sum_{t=1}^{T}C_{t}$: the cost-to-go of executing $\pi$ for $T$ steps.

• —

$J\left(\pi^{*}\right)=\sum_{t=1}^{T}C_{t,c}^{*}$: the cost-to-go of executing $\pi^{*}$ for $T$ steps.

The probability that the learner chooses at least one different action than the expert in the first $t$ steps is:

[TABLE]

This gives $\left(1-p_{t}\right)\leq(1-p_{t-1})+\epsilon_{t}$ since $p_{t-1}\in\left[0,1\right]$. Solving this recurrence gives us:

[TABLE]

Now consider in state distribution $d_{t,c}^{\pi}$, if $\pi$ chooses a different action than $\pi^{*}$ with probability $\epsilon_{t,c}$, $\pi$ will incur a cost at most $C_{max}$ more than $\pi^{*}$. This can be represented as:

[TABLE]

Thus, we have:

[TABLE]

Summing the above result over $T$ steps and using the fact $\frac{1}{T}\sum_{t=1}^{T}\epsilon_{t}\leq\epsilon$:

[TABLE]

∎

Several approaches have been proposed to solve SPC tasks using supervised learning while keeping the error growing linearly instead of quadratically with $T$ Syed and Schapire, (2010); Ross et al., (2011); Daumé et al., (2009). Essentially, these methods reduce an SPC task to online learning. By further leveraging interactions with experts and no-regret algorithms that have strong guarantees on convex loss functions Kakade and Tewari, (2009), at each iteration, these methods train one or multiple policies using standard supervised learning and improve the trained policies as the iteration proceeds.

To illustrate, I denote the best policy at $i$th iteration (trained using all observations from the previous $i-1$ iterations) as $\pi_{i}$ and for any policy $\pi\in\Pi$ we have its expected loss under the observation distribution induced by $\pi_{i}$ as $l_{i}\left(\pi\right)=\mathbb{E}_{\phi\sim d_{\pi_{i}},a^{*}\sim\pi^{*}(\phi)}\left[l_{i}\left(\phi,\pi,a^{*}\right)\right],l_{i}\in\left[0,l_{max}\right]$111In online learning, the surrogate loss $l$ can be seen as chosen by some adversary which varies at each iteration.. In addition, I denote the minimal loss in hindsight after $N\geq i$ iterations as $\epsilon_{min}=\min_{\pi\in\Pi}\frac{1}{N}\sum_{i=1}^{N}l_{i}(\pi)$ (i.e., the training loss after using all observations from $N$ iterations). Then, the average regret of this online learning program can be represented as $\epsilon_{regret}=\frac{1}{N}\sum_{i=1}^{N}l_{i}(\pi_{i})-\epsilon_{min}$. Using DAGGER Ross et al., (2011) as an example method, the accumulated error difference becomes the summation of three terms:

[TABLE]

where $f\left(\cdot\right)$ is the function of fixed $T$ and $l_{max}$. As $N\rightarrow\infty$, the third term tends to [math] so as the second term if a no-regret algorithm such as the Follow-the-Leader Hazan et al., (2007) is used.

To further prepare readers for understanding ADPAS, I adapt the proof of DAGGER from Ross et al. Ross et al., (2011) and include it here for completeness. Note that for Theorem 4.4, I have arrived at the different third term as of Ross et al. Ross et al., (2011).

Lemma 4.1**.**

Ross et al., (2011) Let $P$ and $Q$ be any two distributions over elements $x\in\mathcal{X}$ and $f:\mathcal{X}\rightarrow\mathbb{R}$, any bounded function such that $f(x)\in\left[a,b\right]$ for all $x\in\mathcal{X}$. Let the range $r=b-a$. Then $|\mathbb{E}_{x\sim P}\left[f(x)\right]-\mathbb{E}_{x\sim Q}\left[f(x)\right]|\leq\frac{r}{2}\lVert P-Q\rVert_{1}$.

Démonstration.

[TABLE]

Taking $c=a+\frac{r}{2}$ leads to $\max_{x}|f(x)-c|\leq\frac{r}{2}$, thus proves the lemma. ∎

Lemma 4.2**.**

Ross et al., (2011) Let $\hat{\pi}_{i}$ be the learned policy, $\pi^{*}$ be the expert’s policy, and $\pi_{i}$ be the policy used to collect training data with probability $\beta_{i}$ executing $\pi^{*}$ and probability $1-\beta_{i}$ executing $\hat{\pi}_{i}$ over $T$ steps. Then, I have $\lVert d_{\pi_{i}}-d_{\hat{\pi}_{i}}\rVert_{1}\leq 2\min(1,T\beta_{i})$.

Démonstration.

In contrast to $d_{\hat{\pi}_{i}}$ which is the state distribution as the result of solely executing $\hat{\pi}_{i}$, I denote $d$ as the state distribution as the result of $\pi_{i}$ executing $\pi^{*}$ at least once over $T$ steps. This gives $d_{\pi_{i}}=(1-\beta_{i})^{T}d_{\hat{\pi}_{i}}+\left(1-(1-\beta_{i})^{T}\right)d$. I also have the facts that for any two distributions $P$ and $Q$, $\lVert P-Q\rVert_{1}\leq 2$ and $(1-\beta)^{T}\geq 1-\beta T,\forall\beta\in\left[0,1\right]$. Then, I have $\lVert d_{\pi_{i}}-d_{\hat{\pi}_{i}}\rVert_{1}\leq 2$ and can further show the following:

[TABLE]

∎

Theorem 4.2**.**

Ross et al., (2011) If the surrogate loss $l\in\left[0,l_{max}\right]$ is the same as the cost function $C$ or upper bounds it, then after $N$ iterations of DAGGER:

[TABLE]

Démonstration.

Let $l_{i}\left(\pi\right)=\mathbb{E}_{\phi\sim d_{\pi_{i}},a^{*}\sim\pi^{*}(\phi)}\left[l\left(\phi,\pi,a^{*}\right)\right]]$ be the expected loss of any policy $\pi\in\Pi$ under the state distribution induced by the learned policy $\pi_{i}$ at the $i$th iteration and $\epsilon_{min}=\min_{\pi\in\Pi}\frac{1}{N}\sum_{i=1}^{N}l_{i}(\pi)$ be the minimal loss in hindsight after $N\geq i$ iterations. Then, $\epsilon_{regret}=\frac{1}{N}\sum_{i=1}^{N}l_{i}(\pi_{i})-\epsilon_{min}$ is the average regret of this online learning program. In addition, the expected loss of any policy $\pi\in\Pi$ under its own induced state distribution is denoted as $L\left(\pi\right)=\mathbb{E}_{\phi\sim d_{\pi},a^{*}\sim\pi^{*}(\phi)}\left[l\left(\phi,\pi,a^{*}\right)\right]]$ and consider $\bar{\pi}$ as the mixed policy that samples the policies $\{\hat{\pi}_{i}\}_{i=1}^{N}$ uniformly at the beginning of each trajectory. Using Lemma 4.1 and Lemma 4.2, we can show:

[TABLE]

By further assuming $\beta_{i}$ is monotonically decreasing and $n_{\beta}=\operatorname*{arg\,max}_{n}(\beta_{n}>\frac{1}{T}),n\leq N$, we have the following:

[TABLE]

Summing over $T$ gives us:

[TABLE]

Define $\beta_{i}=(1-\alpha)^{i-1}$, in order to have $\beta_{i}\leq\frac{1}{T}$, we need $(1-\alpha)^{i-1}\leq\frac{1}{T}$ which leads $i\leq 1+\frac{\log{\frac{1}{T}}}{\log{(1-\alpha)}}$. In addition, note now $i=n_{\beta}$ and $\sum_{i=n_{\beta}+1}^{N}\beta_{i}=\frac{(1-\alpha)^{n\beta}-(1-\alpha)^{N}}{\alpha}\leq\frac{1}{T\alpha}$, continuing the above derivation, we have:

[TABLE]

Given the fact $J\left(\hat{\pi}\right)=\min_{i\in 1:N}J(\hat{\pi}_{i})\leq J(\bar{\pi})$ and representing the third term as $\mathcal{O}(\frac{f(T,l_{max})}{N})$, we have proved the theorem. ∎

DAGGER offers a practical way to solve SPC tasks. However, it may require many iterations to obtain a robust policy. In addition, usually human experts or pre-defined controllers are needed for labeling the generated training data, which could be inefficient and difficult to generalize. For autonomous driving, we want the iteration number to be minimal since it directly corresponds to the number of accidents. This requires the generation of training data being accurate, efficient, and sufficient.

ADAPS

In the following, I present theoretical analysis of ADAPS and introduce its pipeline.

Theoretical Analysis

I have evaluated my approach against existing learning mechanisms such as DAGGER Ross et al., (2011), with my method’s results proving to be more effective. Specifically, DAGGER Ross et al., (2011) assumes that an underlying learning algorithm has access to a reset model. So, the training examples can be obtained only online by resetting an agent to its initial state distribution and executing a learned policy, thus achieving “small changes” at each iteration Ross et al., (2011); Daumé et al., (2009); Kakade and Langford, (2002); Bagnell et al., (2004). In comparison, my method allows a learning algorithm to access a generative model so that the training examples can be acquired offline by putting an agent to arbitrary states during the analysis of an accident and letting a generative model simulate its behavior. This approach can result in massive training data, thus has the potential to assist a policy achieving “large improvements” in one iteration.

Additionally, existing techniques such as DAGGER Ross et al., (2011) usually incorporate the demonstrations of a few (human) experts into training. Because of the reset model assumption and the lack of a diversity requirement on experts, these demonstrations can be homogeneous. In contrast, using my parameterized model to retrace and analyze each accident, the number of recovery actions obtained can be multiple orders of magnitude higher. More importantly, I can treat the generated trajectories and the additional data generated based on them (described in Section 4.6.2) as running a learned policy to sample independent expert trajectories at different states, since 1) the generated trajectories are results from a principled simulation algorithm which samples its own parameter distributions independently for each simulation run; 2) my model provides near-exhaustive coverage of the configuration space of a vehicle within the boundaries of a road. With these assumptions, I derive the following theorem.

Theorem 4.3**.**

If the surrogate loss $l$ upper bounds the true cost $C$, by collecting $K$ trajectories using ADAPS at each iteration, with probability at least $1-\mu$, $\mu\in(0,1)$, ADAPS offers the following guarantee:

[TABLE]

Démonstration.

Assuming at the $i$th iteration, my model generates $K$ trajectories. These trajectories are independent from each other since they are generated using different parameters and at different states during the analysis of an accident. For the $k$th trajectory, $k\in[\![1,K]\!]$, I can construct an estimate $\hat{l}_{ik}(\hat{\pi}_{i})=\frac{1}{T}\sum_{t=1}^{T}l_{i}\left(\phi_{ikt},\hat{\pi}_{i},a^{*}_{ikt}\right)$, where $\hat{\pi}_{i}$ is the learned policy from data gathered in previous $i-1$ iterations. Then, the approximated expected loss $\hat{l}_{i}$ is the average of these $K$ estimates: $\hat{l}_{i}(\hat{\pi}_{i})=\frac{1}{K}\sum_{k=1}^{K}\hat{l}_{ik}(\hat{\pi}_{i})$. I denote $\hat{\epsilon}_{min}=\min_{\pi\in\Pi}\frac{1}{N}\sum_{i=1}^{N}\hat{l}_{i}(\pi)$ as the approximated minimal loss in hindsight after $N$ iterations, then $\hat{\epsilon}_{regret}=\frac{1}{N}\sum_{i=1}^{N}\hat{l}_{i}(\hat{\pi}_{i})-\hat{\epsilon}_{min}$ is the approximated average regret.

Let $Y_{i,k}=l_{i}(\hat{\pi}_{i})-\hat{l}_{ik}(\hat{\pi}_{i})$ and define random variables $X_{nK+m}=\sum_{i=1}^{n}\sum_{k=1}^{K}Y_{i,k}+\sum_{k=1}^{m}Y_{n+1,k}$, for $n\in[\![0,N-1]\!]$ and $m\in[\![1,K]\!]$. Consequently, $\{X_{i}\}_{i=1}^{NK}$ form a martingale and $|X_{i+1}-X_{i}|\leq l_{max}$. By Azuma-Hoeffding’s inequality, with probability at least $1-\mu$, I have $\frac{1}{KN}X_{KN}\leq l_{max}\sqrt{\frac{2\log{\frac{1}{\mu}}}{KN}}$.

Next, I denote the expected loss of any policy $\pi\in\Pi$ under its own induced state distribution as $L\left(\pi\right)=\mathbb{E}_{\phi\sim d_{\pi},a^{*}\sim\pi^{*}(\phi)}\left[l\left(\phi,\pi,a^{*}\right)\right]]$ and consider $\bar{\pi}$ as the mixed policy that samples the policies $\{\hat{\pi}_{i}\}_{i=1}^{N}$ uniformly at the beginning of each trajectory. During the data collection in each iteration, I only execute the learned policy instead of mix it with the expert’s policy, which leads to $L(\hat{\pi}_{i})=l(\hat{\pi}_{i})$. Finally, I can show:

[TABLE]

Summing over $T$ proves the theorem. ∎

Theorem 4.3 provides a bound for the expected cost-to-go of the best learned policy $\hat{\pi}$ based on the empirical error of the best policy in $\Pi$ (i.e., $\hat{\epsilon}_{min}$) and the empirical average regret of the learner (i.e., $\hat{\epsilon}_{regret}$). The second term can be eliminated if a no-regret algorithm such as Follow-the-Leader Hazan et al., (2007) is used and the third term suggests that I need the number of training examples $KN$ to be $\mathcal{O}\left(T^{2}l^{2}_{max}\log{\frac{1}{\mu}}\right)$ in order to have a negligible generalization error, which is easily achievable using ADAPS. Summarizing these changes, I derive the following Corollary.

Corollary 4.

If $l$ is convex in $\pi$ for any $s$ and it upper bounds $C$, and Follow-the-Leader is used to select the learned policy, then for any $\epsilon>0$, after collecting $\mathcal{O}\left(\frac{T^{2}l^{2}_{max}\log{\frac{1}{\mu}}}{\epsilon^{2}}\right)$ training examples, with probability at least $1-\mu$, $\mu\in(0,1)$, ADAPS offers the following guarantee:

[TABLE]

Démonstration.

Following Theorem 4.3 and the aforementioned deduction. ∎

Now I only need the best policy to have a small training error $\hat{\epsilon}_{min}$, which can be achieved via standard supervised learning.

Framework Pipeline

The pipeline of my framework is the following. First, in SimLearner, I test a learned policy by letting it control an AV. During the testing, an accident may occur, in which case the trajectory of the vehicle and the full specifications of the situation (e.g., positions of obstacles, road configuration, etc.) are known. Next, I switch to SimExpert and replicate the specifications of the accident in order to “solve” is (i.e., find alternative safe trajectories and dangerous zones). After obtaining the solutions, I then use them to generate additional training data in SimLearner, which will be combined with previously generated data to update a policy. Next, I test the updated policy again. This process continues until the policy has reached a pre-specified threshold for the imitation error.

Policy Learning

In this section, I will detail ADAPS’s control policy by first explaining my design rationale then formulating the problem and introducing the training data collection.

Driving is a hierarchical decision process. In its simplest form, a driver needs to constantly monitor the road condition, decide it is “safe” or “dangerous”, and make corresponding maneuvers. When designing a control policy for AVs, we need to consider this hierarchical aspect. In addition, driving is a temporal behavior. Drivers need reaction time to respond to various road situations Johansson and Rumar, (1971); McGehee et al., (2000). A Markovian-based control policy will not model this aspect and instead likely to give a vehicle jerky motions. Consider these factors, I propose a hierarchical and memory-enabled control policy.

The task I consider is autonomous driving via a single front-facing camera. My control policy consists of three modules: Detection, Following, and Avoidance. The Detection module keeps monitoring road conditions and activates either Following or Avoidance to produce a steering command. All these modules are trained via end-to-end imitation learning and share the same network architecture, which combines Long Short-Term Memory (LSTM) Hochreiter and Schmidhuber, (1997) and Convolutional Neural Networks (CNNs) LeCun et al., (2015). Images from the front-facing camera will first go through a CNN and then a LSTM. The number of images of a training sample going into the LSTM is empirically set to 5. I use the many-to-many mode of LSTM and set the number of hidden units of the LSTM to 100. The output is the average value of the output sequence.

My CNN architecture consists of eight layers. The first five are convolutional layers and the last three are dense layers. The kernel size is $5\times 5$ in the first three convolutional layers and $3\times 3$ in the other two convolutional layers. The first three convolutional layers have a stride of 2 while the last two convolutional layers are non-strided. The filters for the five convolutional layers are 24, 36, 48, 64, 64, respectively. All convolutional layers use VALID padding. The three dense layers have 100, 50, and 10 units, respectively. I use ELU as the activation function and $\mathcal{L}2$ as the kernel regularizer, which is set to 0.001 for all layers.

I train my model using Adam Kingma and Ba, (2015) with initial learning rate set to 0.0001. The batch size is 128 and the number of epochs is 500. For training Detection (a classification task), I use Softmax for generating the output and categorical cross entropy as the loss function. For training Following and Avoidance (regression tasks), I use mean squared error (MSE) as the loss function. I have also adopted cross-validation with 90/10 split. The input image data have $220\times 66$ resolution in RGB channels.

End-to-end Imitation Learning

The objective of imitation learning is to train a model that behaves or makes decisions like an expert through demonstrations. The model could be a classifier or a regresser $\pi$ parameterized by $\mathbf{\theta}_{\pi}$:

[TABLE]

where $\mathcal{F}$ is a distance function.

The end-to-end aspect denotes the mapping from raw observations to decision/control commands. For my policy, I need one decision module $\pi_{Detection}$ and two control modules $\pi_{Following}$ and $\pi_{Avoidance}$. The input to $\pi_{Detection}$ is a sequence of annotated images while the outputs are binary labels, indicating whether a road condition is dangerous or safe. The inputs to $\pi_{Following}$ and $\pi_{Avoidance}$ are both sequences of annotated images while the outputs are steering angles. Together, these learned policies form a hierarchical control mechanism enabling an AV to drive safely on roads and avoid obstacles when needed.

Training Data Collection

For training Following, inspired by the technique from Bojarski et al. Bojarski et al., (2016), I collect images from three front-facing cameras behind the main windshield: one at the center, one at the left side, and one at the right side. The image from the center camera is labeled with the exact steering angle while the images from the other two cameras are labeled with adjusted steering angles. However, once Following is learned, it only needs images from the center camera to operate.

For training Avoidance, I rely on SimExpert, which can generate numerous intermediate collision-free trajectories between the first moment and the last moment of a potential accident (see Section 4.6.1). By positioning an AV on these trajectories, I collect images from the center front-facing camera along with corresponding steering angles. The training of Detection requires a more sophisticated mechanism and is the subject of the next section.

Learning from Accidents

I explain how I analyze an accident in SimExpert and use the generated data to train the Avoidance and Detection modules of my policy. SimExpert is built based on the multi-agent simulator WarpDriver Wolinski et al., (2016).

Solving Accidents

When an accident occurs, we know the trajectory of the tested vehicle for the latest $K$ frames, which I note as a collection of states $\mathcal{S}=\bigcup_{k\in[\![1,K]\!]}\mathbf{s}_{k}$, where each state $\mathbf{s}_{k}\in\mathbb{R}^{4}$ contains the 2-dimensional position and velocity vectors of the vehicle. Then, there are three notable states on this trajectory that worth tracking. The first is the earliest state where the vehicle involved in an accident (is in a collision) $\mathbf{s}_{k_{a}}$ (at frame $k_{a}$). The second is the last state $\mathbf{s}_{k_{l}}$ (at frame $k_{l}$) where the expert algorithm can still avoid a collision. The final one is the first state $\mathbf{s}_{k_{f}}$ (at frame $k_{f}$) where the expert algorithm perceives the interaction leading to the accident with the other involved agent, before that accident.

In order to compute these notable states, I briefly recall the high-level components of WarpDriver Wolinski et al., (2016). This collision-avoidance algorithm consists of two parts. The first is a function $p$, which given the current state of an agent $\mathbf{s}_{k}$ and any prediction point $\mathbf{x}\in\mathbb{R}^{3}$ in 2-dimensional space and time (in this agent’s referential), gives the probability of that agent’s colliding with any neighbor $p(\mathbf{s}_{k},\mathbf{x})\in[0,\>1]$. The second part is the solver, which based on this function, computes the agent’s probability of colliding with neighbors along its future trajectory starting from a state $\mathbf{s}_{k}$ (i.e., computed for $\mathbf{x}$ spanning the future predicted trajectory of the agent, I denote this probability $P\left(\mathbf{s}_{k}\right)$), and then proposes a new velocity to lower this probability. Subsequently, I can initialize an agent in this algorithm to any state $\mathbf{s}_{k}\in\mathcal{S}$ and compute a new trajectory consisting of $\hat{K}$ new states $\mathcal{\hat{S}}_{k}=\bigcup_{\hat{k}\in[\![1,\hat{K}]\!]}\mathbf{\hat{s}}_{\hat{k}}$, where $\mathbf{\hat{s}}_{1}=\mathbf{s}_{k}$.

Additionally, since $\mathbf{x}=(0,\>0,\>0)$ in space and time in an agent’s referential represents the agent’s position at the current time (I can use this point $\mathbf{x}$ with function $p$ to determine if the agent is currently in a collision with another agent), I find $\mathbf{s}_{k_{a}}$ where $k_{a}=\min(k)$ subject to $k\in[\![1,K]\!]$ and $p(\mathbf{s}_{k},(0,\>0,\>0))>0$. I note that a trajectory $\mathcal{\hat{S}}_{k}$ produced by the expert algorithm could contain collisions (accounting for vehicle dynamics) depending on the state $\mathbf{s}_{k}$ that it was initialized from. I denote the set of colliding states along this trajectory as $coll(\mathcal{\hat{S}}_{k})=\{\mathbf{\hat{s}}_{\hat{k}}\in\mathcal{\hat{S}}_{k}\>|\>p(\mathbf{\hat{s}}_{\hat{k}},(0,\>0,\>0))>0\}$. Then, I can compute $\mathbf{s}_{k_{l}}$ where $k_{l}=max(k)$ subject to $k\in[\![1,k_{a}]\!]$ and $coll(\mathcal{\hat{S}}_{k})=\emptyset$. Finally, I can compute $\mathbf{s}_{k_{f}}$ with $k_{f}=1+max(k)$ subject to $k\in[\![1,k_{l}]\!]$ and $P(\mathbf{s}_{k})=0$.

Knowing these notable states, I can solve the accident situation by computing the set of collision-free trajectories $solve(\mathcal{S})=\{\mathcal{\hat{S}}_{k}\>|\>k\in[\![k_{f},k_{l}]\!]\}$. An example is shown in Figure 4.1, in which a set of generated trajectories for a situation where the vehicle had collided with a static obstacle in front of it after driving on a straight road. As expected, the trajectories feature sharper turns (red trajectories) as the starting state tends towards the last moment that the vehicle can still avoid the obstacle. These trajectories can then be used to generate training examples in SimLearner in order to train the Avoidance module.

Additional Data Coverage

The previous step generates collision-free trajectories $solve(\mathcal{S})$ between $\mathbf{s}_{k_{f}}$ and $\mathbf{s}_{k_{l}}$. It is possible to build on these trajectories if the tested steering algorithm has particular data/training requirements. Here I detail the data I derived for training the Detection module. where the task is to determine if a situation is dangerous and tell Avoidance to address it.

To proceed, I essentially generate a number of trajectories parallel to $\{\mathbf{s}_{k_{f}},...,\mathbf{s}_{k_{a}}\}$, and for each position on them, generate several images for various orientations of the vehicle. These images are then labeled based on under-steering/over-steering as compared to the “ideal” trajectories in $solve(\mathcal{S})$. This way, I scan the region of the road before the accident locus, generating several images (different vehicle orientations) for each point in that region. Next, I will explain this process in details.

For the following paragraph, I abusively note $\mathbf{s}_{k}.x$, $\mathbf{s}_{k}.y$ the position coordinates at state $\mathbf{s}_{k}$, and $\mathbf{s}_{k}.vx$, $\mathbf{s}_{k}.vy$ the velocity vector coordinates at state $\mathbf{s}_{k}$. Then, for any state $\mathbf{s}_{k}\in\{\mathbf{s}_{k_{f}},...,\mathbf{s}_{k_{a}}\}$ I can define a line $L(\mathbf{s}_{k})=\{\mathbf{l}_{u}=(\mathbf{s}_{k}.x,\>\mathbf{s}_{k}.y)+u\times(-\mathbf{s}_{k}.vy,\>\mathbf{s}_{k}.vx)\>|\>u\in\mathbb{R}\}$. On this line, I note $\mathbf{l}_{i}$ the furthest point on $L(\mathbf{s}_{k})$ from $(\mathbf{s}_{k}.x,\>\mathbf{s}_{k}.y)$ which is at an intersection between $L(\mathbf{s}_{k})$ and a collision-free trajectory from $solve(\mathcal{S})$. This point determines how far the vehicle can be expected to stray from the original trajectory $\mathcal{S}$ before the accident, if it followed an arbitrary trajectory from $solve(\mathcal{S})$. I also note $\mathbf{l}_{rl}$ and $\mathbf{l}_{rr}$ the two intersections between $L(\mathbf{s}_{k})$ and the road edges ($\mathbf{l}_{rl}$ is on the “left” with $rl>0$, and $\mathbf{l}_{rr}$ is on the “right” with $rr<0$). These two points delimit how far from the original trajectory the vehicle could be. Finally, I define a user-set margin $g$ as outlined below (I set $g=0.5\leavevmode\nobreak\ m$).

Altogether, these points and margin are the limits of the region along the original trajectory wherein I generate images for training: a point $\mathbf{l}_{u}\in L(\mathbf{s}_{k})$ is inside the region if it is between the original trajectory and the furthest collision-free trajectory plus a margin $g$ (if $\mathbf{l}_{u}$ and $\mathbf{l}_{i}$ are on the same side, i.e., $sign(u)=sign(i)$), or if it is between the original trajectory and either road boundary plus a margin $g$ (if $\mathbf{l}_{u}$ and $\mathbf{l}_{i}$ are not on the same side, i.e., $sign(u)\neq sign(i)$).

In addition, if a point $\mathbf{l}_{u}\in L(\mathbf{s}_{k})$ is positioned between two collision-free trajectories $\mathcal{\hat{S}}_{k_{1}},\mathcal{\hat{S}}_{k_{2}}\in solve(\mathcal{S})$, I consider the two closest states on $\mathcal{\hat{S}}_{k_{1}}$, and the two closest states from $\mathcal{\hat{S}}_{k_{1}}$, and bi-linearly interpolate these four states’ velocity vectors, resulting in an approximate velocity vector $\mathbf{v}_{bilin}(\mathbf{l}_{u})$ at $\mathbf{l}_{u}$. Similarly, if a point $\mathbf{l}_{u}\in L(\mathbf{s}_{k})$ is not positioned between two collision-free trajectories, I consider the two closest states on the single closest collision-free trajectory $\mathcal{\hat{S}}_{k_{1}}\in solve(\mathcal{S})$, and linearly interpolate their velocity vectors, resulting in an approximate velocity vector $\mathbf{v}_{lin}(\mathbf{l}_{u})$ at $\mathbf{l}_{u}$.

From here, I can construct images at various points $\mathbf{l}_{u}$ along $L(\mathbf{s}_{k})$ (increasing $u$ by steps of $0.1\leavevmode\nobreak\ m$), with various orientation vectors (noted $\mathbf{v}_{u}$ and within 2.5 degrees of $(\mathbf{s}_{k}.vx,\>\mathbf{s}_{k}.vy)$), and label them using the following scheme (also illustrated in Figure 4.2). If the expert algorithm steers the vehicle to avoid obstacles ($\mathbf{l}_{i}$ with $i>0$), there are four cases to consider when building a point $\mathbf{l}_{u}$:

• —

$u<i+g$ and $u>i$: $\mathbf{l}_{u}$ is outside of the computed collision-free trajectories $solve(\mathcal{S})$, on the outside of the steering computed by the expert algorithm. The label is SAFE if $det(\mathbf{v}_{lin}(\mathbf{l}_{u}),\mathbf{v}_{u})\geq 0$, and DANGER otherwise.

• —

$u<i$ and $u>0$: $\mathbf{l}_{u}$ is inside the computed collision-free trajectories $solve(\mathcal{S})$. The label is SAFE if $det(\mathbf{v}_{bilin}(\mathbf{l}_{u}),\mathbf{v}_{u})\geq 0$ (over-steering), and DANGER otherwise (under-steering).

• —

$u<0$ and $u>rr$: $\mathbf{l}_{u}$ is outside the computed collision-free trajectories $solve(\mathcal{S})$ on the inside of the steering computed by the expert algorithm. The label is always DANGER.

• —

$u<rr$ and $u>rr-g$: $\mathbf{l}_{u}$ is in an unattainable region, but I include it to prevent false reactions to similar (but safe) future situations. The label is DANGER if $det(\mathbf{v}_{lin}(\mathbf{l}_{u}),\mathbf{v}_{u})>0$, SAFE otherwise.

Here, the function $det(\cdot,\cdot)$ computes the determinant of two vectors from $\mathbb{R}^{2}$. Conversely, if the expert algorithm made the vehicle avoid obstacles by steering right ($\mathbf{l}_{i}$ with $i<0$), there are four cases to consider when building a point $\mathbf{l}_{u}$:

• —

$u>i-g$ and $u<i$: the label is SAFE if $det(\mathbf{v}_{lin}(\mathbf{l}_{u}),\mathbf{v}_{u})\leq 0$, and DANGER otherwise.

• —

$u>i$ and $u<0$: the label is SAFE if $det(\mathbf{v}_{bilin}(\mathbf{l}_{u}),\mathbf{v}_{u})\leq 0$, and DANGER otherwise.

• —

$u>0$ and $u<rl$: the label is always DANGER.

• —

$u>rl$ and $u<rl+g$: the label is DANGER if $det(\mathbf{v}_{lin}(\mathbf{l}_{u}),\mathbf{v}_{u})<0$, SAFE otherwise.

I then generate images from these (position, orientation, label) triplets which are used to further train the Detection module of my policy.

Experiments

In this section, I will first detail my experiment setup then show my evaluation results.

Experiment Setup

4.7.1.1 Scenarios

I have tested my method in three scenarios. The first is a straight road representing a linear geometry, the second is a curved road representing a non-linear geometry, and the third is an open ground. The first two represent on-road situations with a static obstacle while the last represents an off-road situation with a dynamic obstacle.

Both the straight and curved roads consist of two lanes. The width of each lane is $3.75\leavevmode\nobreak\ m$ and there is a $3\leavevmode\nobreak\ m$ shoulder on each side of the road. The curved road is half circular with radius at $50\leavevmode\nobreak\ m$ and is attached to two straight roads at each end. The open scenario is a $1000\leavevmode\nobreak\ m$ $\times$ $1000\leavevmode\nobreak\ m$ ground, which has a green sphere treated as the target for the Following module to steer the AV.

4.7.1.2 Vehicle Specifications

The vehicle’s speed is set to $20\leavevmode\nobreak\ m/s$, which value is used to compute the throttle value in the simulator. Due to factors such as the rendering complexity and the delay of the communication module, the actual running speed is in the range of $20\pm 1\leavevmode\nobreak\ m/s$. The length and width of the vehicle are $4.5\leavevmode\nobreak\ m$ and $2.5\leavevmode\nobreak\ m$, respectively. The distance between the rear axis and the rear of the vehicle is $0.75\leavevmode\nobreak\ m$. The front wheels can turn up to 25 degrees in either direction. I have three front-facing cameras set behind the main windshield, which are at $1.2\leavevmode\nobreak\ m$ height and $1\leavevmode\nobreak\ m$ front to the center of the vehicle. The two side cameras (one at left and one at right) are set to be $0.8\leavevmode\nobreak\ m$ away from the vehicle’s center axis. These two cameras are only used to capture data for training Following. During runtime, my control policy only requires images from the center camera to operate.

4.7.1.3 Obstacles

For the on-road scenarios, I use a scaled version of a virtual traffic cone as the obstacle on both the straight and curved roads. This scaling operation is meant to preserve the obstacle’s visibility, since at distances greater than $30\leavevmode\nobreak\ m$ a normal-sized obstacle is quickly reduced to just a few pixels. This is an intrinsic limitation of the single-camera setup (and its resolution), but in reality I can emulate this “scaling” using the camera’s zoom function for instance. For the off-road scenario, I use a vehicle with the same specifications as of the AV as the dynamic obstacle. This vehicle is scripted to collide into the AV on its default course when no avoidance behavior is applied by the AV.

4.7.1.4 Training Data

In order to train Following, I have built a waypoint system on the straight road and curved road for AVs to follow, respectively. By running the vehicle for roughly equal distances on both roads, I have gathered in total 65 061 images (33 642 images for the straight road and 31 419 images for the curved road). On the open ground, I have sampled 30 000 positions and computed the angle difference between the direction towards the sphere target and the forward direction. This gives me 30 000 training examples.

In order to train Avoidance, on the straight road, I rewind the accident by 74 frames starting from the frame that the accident takes place, by re-planning the vehicle at each state, I have obtained 74 safe trajectories. Similarly, on the curved road, I rewind the accident by 40 frames, which results in 40 safe trajectories. On the open ground, I rewind the accident by 46 frames, which results in 46 safe trajectories. By positioning the vehicle on these trajectories and capturing the image from the front-facing camera, I have collected 34 516 images for the straight road, 33 624 images for the curved road, and 33 741 images for the open ground.

For the training of Detection, using the mechanism explained in Subsection 4.6.2, I have collected 32 538 images for the straight road, 71 859 for the curved road, and 67 102 images for the open ground. These statistics are summarized in Table 4.1.

Evaluation Results

For evaluation, I compare my policy to the “flat policy” that essentially consists of a single DNN  Chen et al., (2015); Xu et al., (2017); Zhang and Cho, (2017); Codevilla et al., (2017). Usually, this type of policy contains a few convolutional layers followed by a few dense layers. Although the specifications may vary, without human intervention, they are mainly limited to single-lane following Codevilla et al., (2017) or off-road collision avoidance LeCun et al., (2005). I select Bojarski et al. Bojarski et al., (2016) as an example network, as it is one of the most tested control policies. In the following, I will first demonstrate the effectiveness of my policy and then qualitatively illustrate the efficiency of my framework.

4.7.2.1 On-road Scenarios

I derive my training datasets from straight road with or without an obstacle and curved road with or without an obstacle. This separation allows me to train multiple policies and test the effect of learning from accidents using my policy compared to Bojarski et al. Bojarski et al., (2016). By progressively increasing the training datasets, I obtain six policies for evaluation:

• —

My policy: trained with only lane-following data $O_{follow}$; $O_{follow}$ additionally trained after analyzing one accident on the straight road $O_{straight}$; and $O_{straight}$ additionally trained after producing one accident on the curved road $O_{full}$.

• —

Similarly, for the policy from Bojarski et al. Bojarski et al., (2016): $B_{follow}$, $B_{straight}$, and $B_{full}$.

I first evaluate $B_{follow}$ and $O_{follow}$ using both the straight and curved roads by counting how many laps (out of 50) the AV can finish. As a result, both policies managed to finish all laps while keeping the vehicle in the lane. I then test these two policies on the straight road with a static obstacle added. Both policies result in the vehicle collides into the obstacle, which is expected since no accident data were used during the training.

Having the occurred accident, I can now use SimExpert to generate additional training data to obtain $B_{straight}$222The accident data are used only to perform a regression task as the policy by Bojarski et al. Bojarski et al., (2016) does not have a classification module. and $O_{straight}$. As a result, $B_{straight}$ continues to cause collision while $O_{straight}$ avoids the obstacle. Nevertheless, when testing $O_{straight}$ on the curved road with an obstacle, accident still occurs because of the corresponding accident data are not yet included in training.

By further including the accident data from the curved road in training, I obtain $B_{full}$ and $O_{full}$. $O_{full}$ manages to perform both lane-following and collision avoidance in all runs. $B_{full}$, on the other hand, leads the vehicle to drift away from the road.

For the studies involved an obstacle, I uniformly sampled 50 obstacle positions on a $3\leavevmode\nobreak\ m$ line segment that is perpendicular to the direction of a road and in the same lane as the vehicle. I compute the success rate as how many times a policy can avoid the obstacle (while stay in the lane) and resume lane-following afterwards. The results are shown in Table 4.2 and example trajectories are shown in Figure 4.3 LEFT and CENTER.

4.7.2.2 Off-road Scenario

I further test my method on an open ground which involves a dynamic obstacle. The AV is trained heading towards a green sphere while an adversary vehicle is scripted to collide with the AV on its default course. The result showing my policy can steer the AV away from the adversary vehicle and resume its direction to the sphere target. This can be seen in Figure 4.3 RIGHT.

4.7.2.3 Data Heterogeneity

The key to rapid policy improvement is to generate training data accurately, efficiently, and sufficiently. Using principled simulations covers the first two criteria, now I demonstrate the third. Compared to the average number of training data collected by DAGGER Ross et al., (2011) at one iteration, my method can achieve over 200 times more training examples for one iteration333The result is computed via dividing the total number of training images via my method by the average number of training data collected using the safe trajectories in each scenario.. This is shown in Table 4.1.

In Figure 4.4, I show the visualization results of images collected via my method and DAGGER Ross et al., (2011) within one iteration via progressively increasing the number of sampled trajectories. My method generates much more heterogeneous training data, which, when produced in a large quantity can facilitate the update of a control policy.

Summary and Future Work

In this chapter, I have proposed ADAPS, a framework that consists of two simulation platforms and a control policy. Using ADAPS, one can easily simulate accidents. Then, ADAPS can retrace each accident, analyze it, and plan alternative safe trajectories. With an additional training data generation technique, my method can produce a large number of heterogeneous training examples compared to existing methods such as DAGGER Ross et al., (2011), thus representing a more efficient learning mechanism. My hierarchical and memory-enabled policy offers robust collision avoidance that previous policies fail to achieve. I have evaluated my method using multiple simulated scenarios, showing a variety of benefits.

There are many future directions. First of all, I would like to combine long-range vision into ADAPS so that an AV can plan ahead in time. Secondly, the generation of accidents can be parameterized using knowledge from traffic engineering studies. Thirdly, I would like to combine more sensors and fuse their inputs so that an AV can navigate in more complicated traffic scenarios. Lastly, in order to improve the safety of the AV, it is critical to model the interactions between the AV and pedestrians. There has been a number of studies making virtual humans more intelligent and life-like Li and Allbeck, (2011); Li et al., (2012); Li and Allbeck, 2012a ; Li and Allbeck, 2012b ; Li et al., (2013). These results can be incorporated into the virtual environment for improving the training of the AV either directly or indirectly.

To this end, I have finished introducing my efforts on advancing autonomous driving in terms of enabling an AV to navigate safely in dangerous situations including accidents. In the next chapter, I will conclude my dissertation with a discussion of future research directions.

CHAPITRE 5:  CONCLUSION

In this dissertation, I have studied traffic at both the macroscopic level and the microscopic level. Macroscopically, I have developed methods to accurately and efficiently estimate and reconstruct city-scale traffic using mobile sensor data while generating visual analytics in various forms. Microscopically, I have developed a framework to simulate, analyze, and produce traffic accident data, and have proposed an efficient online learning mechanism for learning control policies for autonomous driving. The proposed techniques can enable many Intelligent Transportation System (ITS) applications including travel time estimation, route planning, visualization, traffic optimization, traffic management, and safety and control of autonomous vehicles (AVs).

Summary of Results

I described a deterministic approach to estimate traffic conditions based on Wardrop’s Principles Wardrop, (1952) and the shortest travel-time criterion. In order to compute the shortest travel-time path (for realizing the shortest travel-time criterion), the travel time of a road network is required, which information is usually lacking. This issue is addressed by adapting a travel-time allocation method from Hellinga et al. Hellinga et al., (2008), which was derived from the observations of real-world traffic conditions. Compared to other state-of-the-art approaches that use shortest distance criterion, my approach results in less estimation bias in congested environments for map-matching and overall better estimation accuracy on various synthetic GIS data.

I presented an approach to interpolate temporal missing measurements by exploring sparsity embedded in traffic patterns. To be specific, observing the sparse representation in the frequency domain of traffic patterns, I have proposed a method based on the Compressed Sensing algorithm Donoho, (2006); Candes et al., (2006) for recovering traffic patterns using GPS traces. My approach can provide accurate recovery when tested using the ground-truth traffic data from loop detectors, and is among few that have exploited the effectiveness of Compressed Sensing on traffic pattern processing Lin et al., (2019).

In order to further improve the estimation accuracy of traffic conditions for spatial data interpolation, I proposed an iterative approach, which embeds map-matching and travel-time estimation as its sub-routines. This process is further improved by the statistical modeling and learning of traffic conditions of a road segment. The results of my approach are accurate estimations of traffic conditions in areas with GPS data coverage—achieving up to 97% relative improvement in estimation accuracy over previous techniques—and coarse estimations of traffic conditions in areas without GPS data coverage (of a city).

For achieving accurate estimations of traffic conditions in data-deficient areas, using the abovementioned results, I presented a method to dynamically interpolate spatial missing traffic data. In particular, I have leveraged traffic simulation to ensure the consistency of traffic flows on the boundaries of areas with and without GPS data coverage. A metamodel-based simulation optimization is further developed to save the computational cost of using traffic simulation in optimization. Compared to the simulation-only approach, my technique has achieved on average a 7% error rate and up to 90 times speedup. My approach is the first dynamical and efficient method for interpolating large-scale traffic data while ensuring the flow consistency on city-scale boundaries.

After fully reconstructing spatial-temporal traffic at a city scale, I visualized the reconstructed traffic in various forms such as 2D flow map, 2D animation, and 3D animations. These visual representations can be adopted to improve many ITS applications including the analysis of traffic patterns at street level, region level, and the city level, and enrich virtual environment applications such as virtual tourism and the training of general driving behaviors or autonomous driving.

Regarding autonomous driving, I presented ADAPS, a framework that consists of two simulation platforms and a hierarchical control policy. ADAPS can be used to simulate, analyze various traffic scenarios, especially accidents, and automatically produce labeled training data. In addition, ADAPS represents a more efficient online learning mechanism compared to previous techniques, attributing to the switch from the reset modeling approach to the generative modeling approach. Using the hierarchical control policy and the efficient online learning mechanism of ADAPS, robust control policies for autonomous driving can be learned and applied to obtain normal driving and safe navigation in dangerous situations including accidents.

Future Work

There are many future development and research directions can stem from this dissertation, at the macroscopic level of traffic (i.e., city-scale traffic), the microscopic level of traffic (i.e., autonomous driving), the connection between the two levels, and beyond. I will discuss a few of them in each category in the following.

Macroscopic Level

On the “city-scale traffic” side, first of all, it would be useful to develop an interactive simulation platform. Using the platform, policy makers, city planners, and other users can easily edit a road network and alter a transport policy in order to test the effectiveness of these changes via observing the response of simulated traffic flows propagating in a city. Building such a platform would require several elements: a 3D virtual environment with a user interface, a road network construction mechanism Wilkie et al., (2012); Musialski et al., (2013), a road network editing mechanism Chen et al., (2008), and a real-time traffic simulation technique Sewall et al., 2011b ; Wilkie et al., (2013); Garcia-Dorado et al., (2014). The 3D virtual environment can be built using a game engine such as Unity (https://unity.com/) or Unreal (https://www.unrealengine.com). The rest of the elements have been explored to various degrees in existing studies. Unifying these elements would be an interesting topic.

Secondly, simulating city-scale traffic, depending on the levels of detail, can be computationally prohibitive. A scalable approach that can combine modern machine learning techniques and traffic flow models is highly desirable. Such a approach can especially benefit applications with highly interactive and real-time demands such as the simulation platform mentioned above. The metamodel-based simulation optimization presented in Chapter 3 is an example work in this direction. However, the functional component of the metamodel is currently chosen to be quadratic polynomial, which offers limited expressiveness. With the emergence of deep learning LeCun et al., (2015), it would be interesting to replace the quadratic polynomial with a deep neural network or an LSTM network for potential improvements on the traffic reconstruction accuracy.

Thirdly, traffic participants are not limited to cars, mixed traffic involving cars, pedestrians, cyclists, and other motorists are commonly seen in many regions across the globe. A simulation model that can encompass all these different traffic modalities can enrich various real-world and virtual-world applications mentioned in the previous chapters. In order to achieve this goal, knowledge from traffic engineering literature can be exploited. Mixture models, although not necessarily built for simulation, have been explored and developed Faghri and Egyháziová, (1999); Laxman et al., (2010). Examining the possibility of extending these models for mixture traffic simulation is a promising research direction.

Microscopic Level

On the “autonomous driving” side, the system presented in Chapter 4 is an end-to-end system, which means a single model is trained to map the sensor input directly to the control command output. Such an approach is straightforward and usually results in a more compact model as it does not contain intermediate steps. However, an end-to-end system based on deep learning can be hard to interpret. Also, a large number of training examples are often needed to train an end-to-end model.

In contrast, the traditional engineering pipeline, which consists of several modules, can be adopted for autonomous driving. In this approach, the sensor input will get processed and passed to its subsequent modules such as detection, tracking, and planning, before the final control command is produced. This conventional approach has both advantages and disadvantages. Advantages include, since there are multiple modules in the pipeline, less training examples are needed to learn a model; the prior knowledge can be incorporated into the problem; the explainability is improved as the final control command is produced by a planning algorithm rather than directly from the raw sensor input. Disadvantages include, the uncertainty and errors of each module are difficult to propagate backwards to its preceding modules, thus causing the system to suffer potential compounding errors; the computation is not shared between modules: each module is trained independently for a different objective; human experts are usually needed to tailor each module so that the system can achieve maximum performance.

While both end-to-end and traditional engineering approaches have their own characteristics, given that the safety is of the leading concerns these days regarding autonomous driving, the traditional engineering approach is likely to prevail in the near future due to its superior explainability and controllability. Hence, it would be interesting to develop the “traditional engineering” version of ADAPS. The only element needs to be modified is the hierarchical control policy, which currently is represented by a single model with three neural networks. The other elements such as the simulation platforms and online learning mechanism remain applicable.

Another aspect can be improved in ADAPS is the generation of accident data. Currently, the accident simulation is simple and arbitrary. However, in traffic engineering, there exists rich literature on accident analysis and prevention. By exploring which, a systematically way of simulating accidents can be developed, which can bring further justifications to ADAPS on autonomous driving training and testing. One imminent research direction is to incorporate the pre-crash scenarios published by the National Highway Traffic Safety Administration of the United States Najm et al., (2013) into our simulation platform, and then develop a sampling mechanism to produce accident data for learning a control policy.

Beyond the abovemetioned immediate research topics that can be built on top of ADAPS, there are many other interesting research directions. In general, the safety, control, and coordination aspects of autonomous driving all need further exploration and development. One future research direction can be exploring the possibility of using simulations to assist sample-efficient learning for a control policy. Another direction, inspired by the observation that the training of autonomous driving is largely context-dependent, is to develop theory and practice in transferring the learned behaviors of an autonomous vehicle from one environment to other environments. This generalization ability, observed in humans, is largely missing in autonomous driving at the moment.

Connection Between The Two Levels

Although this dissertation has been addressing the macroscopic level and the microscopic level of traffic as two separate topics, the two aspects have tight connection, where many applications and developments can be drawn. From the macro-to-micro perspective, the estimated city-scale traffic conditions can be immediately adopted for better routing and planning of AVs. The reconstructed traffic can be incorporated into virtual environments to provide rich traffic semantics for training the navigation and decision-making of autonomous driving. From the micro-to-macro perspective, AVs can be treated as probe vehicles to gather traffic information in a city so that traffic reconstruction can be achieved with higher accuracy. The AV can also be dispatched to multiple road users as a sharing transportation tool. This way not only the number of vehicles on the road is reduced, which can assist in alleviating traffic jams, but also less space is needed to physically accommodate the large number of vehicles, which implies additional socio-economic benefits. Back to the macro-to-micro perspective, an efficient traffic reconstruction technique can contribute to the design of the dispatching algorithm for AVs to maximize their sharing functionality.

In conclusion, now and into the near future, AVs will be operating not only in traffic but also along with human-driven vehicles. This assembly brings many challenges as well as research opportunities. It would be interesting to develop simulation models for the mixture of human-driven and autonomous vehicles, with the flexibility to choose the percentage of each type of the vehicle. As AVs can be considered part of the overall cyber-physical system, they can serve as additional “degrees of freedom” to the traffic system, which can be potentially “tuned” to regulate traffic flows Wu et al., (2018). The applications range from alleviating traffic congestion to assisting flow distribution in social gatherings or evacuation situations. Lastly, it would be imperative to consider human factors in addition to technology development, given essentially autonomous and intelligent systems are designed and built to improve people’s life. As technology advances, developing collaborative rather than competitive relationships between autonomous systems and humans is the challenge that scientists and engineers will be facing. My future efforts will be centered around this challenge, with the focus on building an cooperative mixed traffic system.

BIBLIOGRAPHY

Références

• Abadi et al., (2015)

Abadi, A., Rajabioun, T., and Ioannou, P. A. (2015).

Traffic flow prediction for road transportation networks with limited traffic data.

IEEE Transactions on Intelligent Transportation Systems, 16(2):653–662.

• Agarwal et al., (2016)

Agarwal, S., Kachroo, P., and Contreras, S. (2016).

A dynamic network modeling-based approach for traffic observability problem.

IEEE Transactions on Intelligent Transportation Systems, 17(4):1168–1178.

• Anava et al., (2015)

Anava, O., Hazan, E., and Zeevi, A. (2015).

Online time series prediction with missing data.

In Proceedings of the 32nd International Conference on Machine Learning (ICML), pages 2191–2199.

• Andrienko and Andrienko, (2006)

Andrienko, N. and Andrienko, G. (2006).

Exploratory analysis of spatial and temporal data: a systematic approach.

Springer Science & Business Media.

• Asif et al., (2016)

Asif, M. T., Mitrovic, N., Dauwels, J., and Jaillet, P. (2016).

Matrix and tensor based methods for missing data estimation in large traffic networks.

IEEE Transactions on Intelligent Transportation Systems, 17(7):1816–1825.

• Atkeson, (1994)

Atkeson, C. G. (1994).

Using local trajectory optimizers to speed up global optimization in dynamic programming.

In Advances in neural information processing systems, pages 663–670.

• Atkeson et al., (1997)

Atkeson, C. G., Moore, A. W., and Schaal, S. (1997).

Locally weighted learning.

Artificial Intelligence Review, 11(1-5):11–73.

• Bagnell et al., (2004)

Bagnell, J. A., Kakade, S. M., Schneider, J. G., and Ng, A. Y. (2004).

Policy search by dynamic programming.

In Advances in neural information processing systems, pages 831–838.

• Barto and Mahadevan, (2003)

Barto, A. G. and Mahadevan, S. (2003).

Recent advances in hierarchical reinforcement learning.

Discrete Event Dynamic Systems, 13(4):341–379.

• Bayarri et al., (1996)

Bayarri, S., Fernandez, M., and Perez, M. (1996).

Virtual reality for driving simulation.

Commun. ACM, 39(5):72–76.

• Bera and Rao, (2011)

Bera, S. and Rao, K. V. K. (2011).

Estimation of origin-destination matrix from traffic counts: the state of the art.

European Transport\Trasporti Europei n., pages 3–23.

• Bi et al., (2016)

Bi, H., Mao, T., Wang, Z., and Deng, Z. (2016).

A data-driven model for lane-changing in traffic simulation.

In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pages 149–158.

• Bojarski et al., (2016)

Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016).

End to end learning for self-driving cars.

arXiv preprint arXiv:1604.07316.

• Candes et al., (2006)

Candes, E., Romberg, J., and Tao, T. (2006).

Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information.

Information Theory, IEEE Transactions on, 52(2):489–509.

• Candès and Wakin, (2008)

Candès, E. and Wakin, M. (2008).

An introduction to compressive sampling.

IEEE signal processing magazine, 25(2):21–30.

• Cascetta and Nguyen, (1988)

Cascetta, E. and Nguyen, S. (1988).

A unified framework for estimating or updating origin/destination matrices from traffic counts.

Transportation Research Part B: Methodological, 22(6):437–455.

• Castro et al., (2012)

Castro, P. S., Zhang, D., and Li, S. (2012).

Urban traffic modelling and prediction using large scale taxi gps traces.

In International Conference on Pervasive Computing, pages 57–72.

• Celikoglu, (2007)

Celikoglu, H. B. (2007).

A dynamic network loading model for traffic dynamics modeling.

IEEE Transactions on Intelligent Transportation Systems, 8(4):575–583.

• Celikoglu et al., (2009)

Celikoglu, H. B., Gedizlioglu, E., and Dell’Orco, M. (2009).

A node-based modeling approach for the continuous dynamic network loading problem.

IEEE Transactions on Intelligent Transportation Systems, 10(1):165–174.

• Celikoglu and Silgu, (2016)

Celikoglu, H. B. and Silgu, M. A. (2016).

Extension of traffic flow pattern dynamic classification by a macroscopic model using multivariate clustering.

Transportation Science, 50(3):966–981.

• Chao et al., (2019)

Chao, Q., Bi, H., Li, W., Mao, T., Wang, Z., Lin, M. C., and Deng, Z. (2019).

A survey on visual traffic simulation: Models, evaluations, and applications in autonomous driving.

Computer Graphics Forum.

• Chao et al., (2018)

Chao, Q., Deng, Z., Ren, J., Ye, Q., and Jin, X. (2018).

Realistic data-driven traffic flow animation using texture synthesis.

IEEE Transactions on Visualization and Computer Graphics, 24(2):1167–1178.

• Chao et al., (2013)

Chao, Q., Shen, J., and Jin, X. (2013).

Video-based personalized traffic learning.

Graphical Models, 75(6):305–317.

• Charalambous and Chrysanthou, (2014)

Charalambous, P. and Chrysanthou, Y. (2014).

The pag crowd: A graph based approach for efficient data-driven crowd simulation.

Computer Graphics Forum, 33(8):95–108.

• Chen et al., (2014)

Chen, B. Y., Yuan, H., Li, Q., Lam, W. H., Shaw, S.-L., and Yan, K. (2014).

Map-matching algorithm for large-scale low-frequency floating car data.

International Journal of Geographical Information Science, 28(1):22–38.

• Chen et al., (2015)

Chen, C., Seff, A., Kornhauser, A., and Xiao, J. (2015).

Deepdriving: Learning affordance for direct perception in autonomous driving.

In Computer Vision, 2015 IEEE International Conference on, pages 2722–2730.

• Chen et al., (2008)

Chen, G., Esch, G., Wonka, P., Müller, P., and Zhang, E. (2008).

Interactive procedural street modeling.

ACM Trans. Graph., 27(3):103:1–103:10.

• Codevilla et al., (2017)

Codevilla, F., Müller, M., Dosovitskiy, A., López, A., and Koltun, V. (2017).

End-to-end driving via conditional imitation learning.

In Robotics and Automation (ICRA), 2017 IEEE International Conference on, pages 746–753. IEEE.

• Conn et al., (2009)

Conn, A., Scheinberg, K., and Vicente, L. N. (2009).

Introduction to derivative-free optimization, volume 8.

SIAM.

• Daumé et al., (2009)

Daumé, H., Langford, J., and Marcu, D. (2009).

Search-based structured prediction.

Machine learning, 75(3):297–325.

• Donoho, (2006)

Donoho, D. (2006).

Compressed sensing.

Information Theory, IEEE Transactions on, 52(4):1289–1306.

• Faghri and Egyháziová, (1999)

Faghri, A. and Egyháziová, E. (1999).

Development of a computer simulation model of mixed motor vehicle and bicycle traffic on an urban road network.

Transportation research record, 1674(1):86–93.

• Ferreira et al., (2013)

Ferreira, N., Poco, J., Vo, H. T., Freire, J., and Silva, C. T. (2013).

Visual exploration of big spatio-temporal urban data: A study of new york city taxi trips.

IEEE Transactions on Visualization and Computer Graphics, 19(12):2149–2158.

• Friedman et al., (2001)

Friedman, J., Hastie, T., and Tibshirani, R. (2001).

The elements of statistical learning.

Springer series in statistics Springer, Berlin.

• Gao, (2012)

Gao, S. (2012).

Modeling strategic route choice and real-time information impacts in stochastic and time-dependent networks.

IEEE Transactions on Intelligent Transportation Systems, 13(3):1298–1311.

• Garcia-Dorado et al., (2017)

Garcia-Dorado, I., Aliaga, D., Bhalachandran, S., Schmid, P., and Niyogi, D. (2017).

Fast weather simulation for inverse procedural design of 3d urban models.

ACM Trans. Graph., 36(2):21:1–21:19.

• Garcia-Dorado et al., (2014)

Garcia-Dorado, I., Aliaga, D., and V. Ukkusuri, S. (2014).

Designing large-scale interactive traffic animations for urban modeling.

Computer Graphics Forum, 33(2):411–420.

• Gning et al., (2011)

Gning, A., Mihaylova, L., and Boel, R. K. (2011).

Interval macroscopic models for traffic networks.

IEEE Transactions on Intelligent Transportation Systems, 12(2):523–536.

• Gordon, (1995)

Gordon, G. J. (1995).

Stable function approximation in dynamic programming.

In Proceedings of the 12th International Conference on Machine Learning (ICML), pages 261–268.

• Greenshields et al., (1935)

Greenshields, B., Bibbins, J., Channing, W., and Miller, H. (1935).

A study of traffic capacity.

Highway Research Board Proceedings, 14(1):448–477.

• Hajiahmadi et al., (2016)

Hajiahmadi, M., van de Weg, G. S., Tampère, C. M., Corthout, R., Hegyi, A., De Schutter, B., and Hellendoorn, H. (2016).

Integrated predictive control of freeway networks using the extended link transmission model.

IEEE Transactions on Intelligent Transportation Systems, 17(1):65–78.

• Hato et al., (1999)

Hato, E., Taniguchi, M., Sugie, Y., Kuwahara, M., and Morita, H. (1999).

Incorporating an information acquisition process into a route choice model with multiple information sources.

Transportation Research Part C: Emerging Technologies, 7(2):109–129.

• Hazan et al., (2007)

Hazan, E., Agarwal, A., and Kale, S. (2007).

Logarithmic regret algorithms for online convex optimization.

Machine Learning, 69(2-3):169–192.

• Hellinga et al., (2008)

Hellinga, B., Izadpanah, P., Takada, H., and Fu, L. (2008).

Decomposing travel times measured by probe-based traffic monitoring systems to individual road segments.

Transportation Research Part C: Emerging Technologies, 16(6):768–782.

• Herrera et al., (2010)

Herrera, J. C., Work, D. B., Herring, R., Ban, X. J., Jacobson, Q., and Bayen, A. M. (2010).

Evaluation of traffic data obtained via gps-enabled mobile phones: The mobile century field experiment.

Transportation Research Part C: Emerging Technologies, 18(4):568–583.

• Herring, (2010)

Herring, R. (2010).

Real-time traffic modeling and estimation with streaming probe data using machine learning.

PhD thesis, University of California, Berkeley.

• Herring et al., (2010)

Herring, R., Hofleitner, A., Abbeel, P., and Bayen, A. (2010).

Estimating arterial traffic conditions using sparse probe data.

In Intelligent Transportation Systems (ITSC), 13th International IEEE Conference on, pages 929–936.

• Hochreiter and Schmidhuber, (1997)

Hochreiter, S. and Schmidhuber, J. (1997).

Long short-term memory.

Neural computation, 9(8):1735–1780.

• (49)

Hofleitner, A., Herring, R., Abbeel, P., and Bayen, A. (2012a).

Learning the dynamics of arterial traffic from probe data using a dynamic bayesian network.

IEEE Transactions on Intelligent Transportation Systems, 13(4):1679–1693.

• (50)

Hofleitner, A., Herring, R., and Bayen, A. (2012b).

Probability distributions of travel times on arterial networks: a traffic flow and horizontal queuing theory approach.

In Transportation Research Board 91st Annual Meeting.

• (51)

Hofleitner, A., Herring, R., Bayen, A., Han, Y., Moutarde, F., and De La Fortelle, A. (2012c).

Large scale estimation of arterial traffic and structural analysis of traffic patterns using probe vehicles.

In Transportation Research Board 91st Annual Meeting.

• Hunter et al., (2014)

Hunter, T., Abbeel, P., and Bayen, A. (2014).

The path inference filter: model-based low-latency map matching of probe vehicle data.

Intelligent Transportation Systems, IEEE Transactions on, 15(2):507–529.

• Hunter, (2014)

Hunter, T. J. (2014).

Large-Scale, Low-Latency State Estimation Of Cyberphysical Systems With An Application To Traffic Estimation.

PhD thesis, University of California, Berkeley.

• Johansson and Rumar, (1971)

Johansson, G. and Rumar, K. (1971).

Drivers’ brake reaction times.

Human factors, 13(1):23–27.

• Ju et al., (2010)

Ju, E., Choi, M. G., Park, M., Lee, J., Lee, K. H., and Takahashi, S. (2010).

Morphable crowds.

ACM Trans. Graph., 29(6):140:1–140:10.

• Kachroo and Sastry, (2016)

Kachroo, P. and Sastry, S. (2016).

Travel time dynamics for intelligent transportation systems: Theory and applications.

IEEE Transactions on Intelligent Transportation Systems, 17(2):385–394.

• Kakade and Langford, (2002)

Kakade, S. and Langford, J. (2002).

Approximately optimal approximate reinforcement learning.

In Proceedings of the 30th International Conference on Machine Learning (ICML), pages 267–274.

• Kakade and Tewari, (2009)

Kakade, S. M. and Tewari, A. (2009).

On the generalization ability of online strongly convex programming algorithms.

In Advances in Neural Information Processing Systems, pages 801–808.

• Kearns et al., (2002)

Kearns, M., Mansour, Y., and Ng, A. Y. (2002).

A sparse sampling algorithm for near-optimal planning in large markov decision processes.

Machine learning, 49(2-3):193–208.

• Khosravi et al., (2011)

Khosravi, A., Mazloumi, E., Nahavandi, S., Creighton, D., and Van Lint, J. (2011).

Prediction intervals to account for uncertainties in travel time prediction.

IEEE Transactions on Intelligent Transportation Systems, 12(2):537–547.

• Kingma and Ba, (2015)

Kingma, D. and Ba, J. (2015).

Adam: A method for stochastic optimization.

In International Conference on Learning Representations (ICLR).

• Kong et al., (2013)

Kong, Q.-J., Zhao, Q., Wei, C., and Liu, Y. (2013).

Efficient traffic state estimation for large-scale urban road networks.

IEEE Transactions on Intelligent Transportation Systems, 14(1):398–407.

• Konidaris et al., (2012)

Konidaris, G., Kuindersma, S., Grupen, R., and Barto, A. (2012).

Robot learning from demonstration by constructing skill trees.

The International Journal of Robotics Research, 31(3):360–375.

• (64)

Krajzewicz, D., Erdmann, J., Behrisch, M., and Bieker, L. (2012a).

Recent development and applications of SUMO - Simulation of Urban MObility.

International Journal On Advances in Systems and Measurements, 5(3-4):128–138.

• (65)

Krajzewicz, D., Erdmann, J., Behrisch, M., and Bieker, L. (2012b).

Recent development and applications of SUMO–simulation of urban mobility.

International Journal On Advances in Systems and Measurements, 5(3&4):128–138.

• Kuhi et al., (2015)

Kuhi, K., Kaare, K. K., and Koppel, O. (2015).

Using probabilistic models for missing data prediction in network industries performance measurement systems.

Procedia Engineering, 100:1348–1353.

• Kuhl et al., (1995)

Kuhl, J., Evans, D., Papelis, Y., Romano, R., and Watson, G. (1995).

The iowa driving simulator: An immersive research environment.

Computer, 28(7):35–41.

• Laxman et al., (2010)

Laxman, K. K., Rastogi, R., and Chandra, S. (2010).

Pedestrian flow characteristics in mixed traffic conditions.

Journal of Urban Planning and Development, 136(1):23–33.

• LeCun et al., (2015)

LeCun, Y., Bengio, Y., and Hinton, G. (2015).

Deep learning.

nature, 521(7553):436.

• LeCun et al., (2005)

LeCun, Y., Muller, U., Ben, J., Cosatto, E., and Flepp, B. (2005).

Off-road obstacle avoidance through end-to-end learning.

In Advances in neural information processing systems, pages 739–746.

• Leduc, (2008)

Leduc, G. (2008).

Road traffic data: Collection methods and applications.

Working Papers on Energy, Transport and Climate Change, 1(55).

• Lee et al., (2007)

Lee, K. H., Choi, M. G., Hong, Q., and Lee, J. (2007).

Group behavior from video: a data-driven approach to crowd simulation.

In Proceedings of the ACM SIGGRAPH/Eurographics symposium on Computer animation, pages 109–118.

• Lerner et al., (2007)

Lerner, A., Chrysanthou, Y., and Lischinski, D. (2007).

Crowds by example.

Computer Graphics Forum, 26(3):655–664.

• Levine and Koltun, (2013)

Levine, S. and Koltun, V. (2013).

Guided policy search.

In Proceedings of the 30th International Conference on Machine Learning (ICML), pages 1–9.

• Li et al., (2014)

Li, L., Chen, X., and Zhang, L. (2014).

Multimodel ensemble for freeway traffic state estimations.

IEEE Transactions on Intelligent Transportation Systems, 15(3):1323–1336.

• Li et al., (2016)

Li, M., Wonka, P., and Nan, L. (2016).

Manhattan-world urban reconstruction from point clouds.

In European Conference on Computer Vision, pages 54–69.

• Li and Allbeck, (2011)

Li, W. and Allbeck, J. M. (2011).

Populations with purpose.

In Proceedings of the 4th International Conference on Motion in Games, pages 132–143.

• (78)

Li, W. and Allbeck, J. M. (2012a).

The virtual apprentice.

In Proceedings of the 12th International Conference on Intelligent Virtual Agents, pages 15–27.

• (79)

Li, W. and Allbeck, J. M. (2012b).

Virtual humans: Evolving with common sense.

In Proceedings of the 5th International Conference on Motion in Games, pages 182–193.

• Li et al., (2013)

Li, W., Balint, T., and Allbeck, J. M. (2013).

Using a parameterized memory model to modulate npc AI.

In Proceedings of the 13th International Conference on Intelligent Virtual Agents, pages 1–14.

• Li et al., (2012)

Li, W., Di, Z., and Allbeck, J. M. (2012).

Crowd distribution and location preference.

Computer Animation and Virtual Worlds, 23(3-4):343–351.

• Li et al., (2018)

Li, W., Jiang, M., Chen, Y., and Lin, M. C. (2018).

Estimating urban traffic states using iterative refinement and wardrop equilibria.

IET Intelligent Transport Systems, 12(8):875–883.

• (83)

Li, W., Nie, D., Wilkie, D., and Lin, M. C. (2017a).

Citywide estimation of traffic dynamics via sparse GPS traces.

IEEE Intelligent Transportation Systems Magazine, 9(3):100–113.

• (84)

Li, W., Wolinski, D., and Lin, M. C. (2017b).

City-scale traffic animation using statistical learning and metamodel-based optimization.

ACM Trans. Graph., 36(6):200:1–200:12.

• Li et al., (2019)

Li, W., Wolinski, D., and Lin, M. C. (2019).

ADAPS: Autonomous driving via principled simulations.

In IEEE International Conference on Robotics and Automation (ICRA).

• Li et al., (2015)

Li, W., Wolinski, D., Pettré, J., and Lin, M. C. (2015).

Biologically-inspired visual simulation of insect swarms.

Computer Graphics Forum, 34(2):425–434.

• Lin et al., (2019)

Lin, L., Li, W., and Peeta, S. (2019).

A compressive sensing approach for connected vehicle data capture and its impact on travel time estimation.

In Transportation Research Board 98th Annual Meeting.

• Lin, (1992)

Lin, L.-J. (1992).

Self-improving reactive agents based on reinforcement learning, planning and teaching.

Machine learning, 8(3-4):293–321.

• Lin et al., (2016)

Lin, W.-C., Wong, S.-K., Li, C.-H., and Tseng, R. (2016).

Generating believable mixed-traffic animation.

IEEE Transactions on Intelligent Transportation Systems, 17(11):3171–3183.

• Lou et al., (2009)

Lou, Y., Zhang, C., Zheng, Y., Xie, X., Wang, W., and Huang, Y. (2009).

Map-matching for low-sampling-rate GPS trajectories.

In Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pages 352–361.

• Lu et al., (2014)

Lu, X., Wang, Z., Xu, M., Chen, W., and Deng, Z. (2014).

A personality model for animating heterogeneous traffic behaviors.

Computer animation and virtual worlds, 25(3-4):361–371.

• Maaten and Hinton, (2008)

Maaten, L. v. d. and Hinton, G. (2008).

Visualizing data using t-SNE.

Journal of machine learning research, 9:2579–2605.

• Mao et al., (2015)

Mao, T., Wang, H., Deng, Z., and Wang, Z. (2015).

An efficient lane model for complex traffic simulation.

Computer Animation and Virtual Worlds, 26(3-4):397–403.

• McGehee et al., (2000)

McGehee, D. V., Mazzae, E. N., and Baldwin, G. S. (2000).

Driver reaction time in crash avoidance research: validation of a driving simulator study on a test track.

Proceedings of the human factors and ergonomics society annual meeting, 44(20).

• Microsoft, (2009)

Microsoft (2009).

Geolife project.

www.microsoft.com/en-us/research/project/geolife-building-social-networks-using-human-location-history/.

• Microsoft, (2010)

Microsoft (2010).

T-drive project.

www.microsoft.com/en-us/research/project/t-drive-driving-directions-based-on-taxi-traces/.

• MIT, (2011)

MIT (2011).

Mit intelligent transportation systems.

its.mit.edu/.

• Mitrovic et al., (2015)

Mitrovic, N., Asif, M. T., Dauwels, J., and Jaillet, P. (2015).

Low-dimensional models for compressed sensing and prediction of large-scale traffic data.

IEEE Transactions on Intelligent Transportation Systems, 16(5):2949–2954.

• Miwa et al., (2012)

Miwa, T., Kiuchi, D., Yamamoto, T., and Morikawa, T. (2012).

Development of map matching algorithm for low frequency probe data.

Transportation Research Part C: Emerging Technologies, 22:132–145.

• Musialski et al., (2013)

Musialski, P., Wonka, P., Aliaga, D. G., Wimmer, M., van Gool, L., and Purgathofer, W. (2013).

A survey of urban reconstruction.

Computer Graphics Forum, 32(6):146–177.

• Najm et al., (2013)

Najm, W. G., Ranganathan, R., Srinivasan, G., Smith, J. D., Toma, S., Swanson, E., Burgett, A., et al. (2013).

Description of light-vehicle pre-crash scenarios for safety applications based on vehicle-to-vehicle communications.

Technical report, United States. National Highway Traffic Safety Administration.

• Osorio, (2010)

Osorio, C. (2010).

Mitigating network congestion: analytical models, optimization methods and their applications.

PhD thesis, École polytechnique fédérale de Lausanne (EPFL).

• Osorio and Bierlaire, (2013)

Osorio, C. and Bierlaire, M. (2013).

A simulation-based optimization framework for urban transportation problems.

Operations Research, 61(6):1333–1345.

• Osorio et al., (2015)

Osorio, C., Flötteröd, G., and Zhang, C. (2015).

A metamodel simulation-based optimization approach for the efficient calibration of stochastic traffic simulators.

Transportation Research Procedia, 6:213–223.

• Pan et al., (2018)

Pan, Y., Cheng, C.-A., Saigol, K., Lee, K., Yan, X., Theodorou, E., and Boots, B. (2018).

Agile off-road autonomous driving using end-to-end deep imitation learning.

In Robotics: Science and Systems.

• Perttunen et al., (2015)

Perttunen, M., Kostakos, V., Riekki, J., and Ojala, T. (2015).

Urban traffic analysis through multi-modal sensing.

Personal and Ubiquitous Computing, 19(3-4):709–721.

• Phan and Ferrie, (2011)

Phan, A. and Ferrie, F. P. (2011).

Interpolating sparse GPS measurements via relaxation labeling and belief propagation for the redeployment of ambulances.

IEEE Transactions on Intelligent Transportation Systems, 12(4):1587–1598.

• Piorkowski et al., (2009)

Piorkowski, M., Sarafijanovic-Djukic, N., and Grossglauser, M. (2009).

CRAWDAD dataset epfl/mobility (v. 2009-02-24).

Downloaded from http://crawdad.org/epfl/mobility/20090224.

• Pomerleau, (1989)

Pomerleau, D. (1989).

ALVINN: An autonomous land vehicle in a neural network.

In Advances in neural information processing systems, pages 305–313.

• Quddus and Washington, (2015)

Quddus, M. and Washington, S. (2015).

Shortest path and vehicle trajectory aided map-matching for low frequency GPS data.

Transportation Research Part C: Emerging Technologies, 55:328–339.

• Rahmani et al., (2015)

Rahmani, M., Jenelius, E., and Koutsopoulos, H. (2015).

Non-parametric estimation of route travel time distributions from low-frequency floating car data.

Transportation Research Part C: Emerging Technologies, 58:343–362.

• Rahmani and Koutsopoulos, (2013)

Rahmani, M. and Koutsopoulos, H. (2013).

Path inference from sparse floating car data for urban networks.

Transportation Research Part C: Emerging Technologies, 30:41–54.

• Ratliff, (2009)

Ratliff, N. (2009).

Learning to search: structured prediction techniques for imitation learning.

PhD thesis, Carnegie Mellon University.

• Ratliff et al., (2007)

Ratliff, N., Bagnell, J. A., and Srinivasa, S. S. (2007).

Imitation learning for locomotion and manipulation.

In 7th IEEE-RAS International Conference on Humanoid Robots, pages 392–397.

• Ross et al., (2011)

Ross, S., Gordon, G., and Bagnell, D. (2011).

A reduction of imitation learning and structured prediction to no-regret online learning.

In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, pages 627–635.

• Ross et al., (2013)

Ross, S., Melik-Barkhudarov, N., Shankar, K. S., Wendel, A., Dey, D., Bagnell, J. A., and Hebert, M. (2013).

Learning monocular reactive UAV control in cluttered natural environments.

In Robotics and Automation, IEEE International Conference on, pages 1765–1772.

• Schrank et al., (2015)

Schrank, D., Eisele, B., Lomax, T., and Bak, J. (2015).

2015 urban mobility scorecard.

Texas A&M Transportation Institute and INRIX.

• Schwarting et al., (2018)

Schwarting, W., Alonso-Mora, J., and Rus, D. (2018).

Planning and decision-making for autonomous vehicles.

Annual Review of Control, Robotics, and Autonomous Systems.

• (119)

Sewall, J., Van Den Berg, J., Lin, M., and Manocha, D. (2011a).

Virtualized traffic: Reconstructing traffic flows from discrete spatiotemporal data.

IEEE Transactions on Visualization and Computer Graphics, 17(1):26–37.

• (120)

Sewall, J., Wilkie, D., and Lin, M. C. (2011b).

Interactive hybrid simulation of large-scale traffic.

ACM Trans. Graph., 30(6):135:1–135:12.

• Sewall et al., (2010)

Sewall, J., Wilkie, D., Merrell, P., and Lin, M. C. (2010).

Continuum traffic simulation.

Computer Graphics Forum, 29(2):439–448.

• Sheffi, (1985)

Sheffi, Y. (1985).

Urban transportation network.

Pretince Hall.

• Shen and Jin, (2012)

Shen, J. and Jin, X. (2012).

Detailed traffic animation for urban road networks.

Graphical Models, 74(5):265–282.

• Silver, (2010)

Silver, D. (2010).

Learning preference models for autonomous mobile robots in complex domains.

PhD thesis, Carnegie Mellon University.

• Sugiyama and Kawanabe, (2012)

Sugiyama, M. and Kawanabe, M. (2012).

Machine learning in non-stationary environments: Introduction to covariate shift adaptation.

MIT press.

• Sun and Work, (2014)

Sun, Y. and Work, D. (2014).

A distributed local kalman consensus filter for traffic estimation.

In Decision and Control (CDC), 53rd IEEE Annual Conference on, pages 6484–6491.

• Sutton et al., (1999)

Sutton, R. S., Precup, D., and Singh, S. (1999).

Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning.

Artificial intelligence, 112(1-2):181–211.

• Syed and Schapire, (2010)

Syed, U. and Schapire, R. E. (2010).

A reduction from apprenticeship learning to classification.

In Advances in Neural Information Processing Systems, pages 2253–2261.

• Szepesvári and Munos, (2005)

Szepesvári, C. and Munos, R. (2005).

Finite time bounds for sampling based fitted value iteration.

In Proceedings of the 22nd international conference on Machine learning, pages 880–887.

• TAC, (2017)

TAC (2017).

TAC’s virtual reality driving school.

www.smh.com.au/victoria/.

• (131)

Tang, J., Song, Y., Miller, H., and Zhou, X. (2016a).

Estimating the most likely space–time paths, dwell times and path uncertainties from vehicle trajectory data: A time geographic method.

Transportation Research Part C: Emerging Technologies, 66:176–194.

• (132)

Tang, J., Song, Y., and Miller, Harveyand Zhou, X. (2016b).

Estimating the most likely space–time paths, dwell times and path uncertainties from vehicle trajectory data: A time geographic method.

Transportation Research Part C: Emerging Technologies, 66:176–194.

• Thomas and Donikian, (2000)

Thomas, G. and Donikian, S. (2000).

Modelling virtual cities dedicated to behavioural animation.

Computer Graphics Forum, 19(3):71–80.

• Tibshirani et al., (2005)

Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., and Knight, K. (2005).

Sparsity and smoothness via the fused lasso.

Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(1):91–108.

• Uber, (2017)

Uber (2017).

Uber movement.

movement.uber.com/.

• van den Berg et al., (2009)

van den Berg, J., Sewall, J., Lin, M., and Manocha, D. (2009).

Virtualized traffic: Reconstructing traffic flows from discrete spatio-temporal data.

In Virtual Reality Conference, IEEE, pages 183–190.

• Wang et al., (2005)

Wang, H., Kearney, J. K., Cremer, J., and Willemsen, P. (2005).

Steering behaviors for autonomous vehicles in virtual environments.

In Virtual Reality Conference, IEEE, pages 155–162.

• Wang et al., (2016)

Wang, Y., Cao, J., Li, W., and Gu, T. (2016).

Mining traffic congestion correlation between road segments on GPS trajectories.

In 2016 IEEE International Conference on Smart Computing (SMARTCOMP), pages 1–8.

• (139)

Wang, Y., Zheng, Y., and Xue, Y. (2014a).

Travel time estimation of a path using sparse trajectories.

In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 25–34.

• (140)

Wang, Z., Ye, T., Lu, M., Yuan, X., Qu, H., Yuan, J., and Wu, Q. (2014b).

Visual exploration of sparse traffic trajectory data.

IEEE Transactions on Visualization and Computer Graphics, 20(12):1813–1822.

• Wardrop, (1952)

Wardrop, J. (1952).

Some theoretical aspects of road traffic research.

Proceedings of the Institution of Civil Engineers, 1(3):325–362.

• Westgate et al., (2013)

Westgate, B., Woodard, D., Matteson, D., and Henderson, S. (2013).

Travel time estimation for ambulances using bayesian data augmentation.

The Annals of Applied Statistics, 7(2):1139–1161.

• Wilkie et al., (2014)

Wilkie, D., Baykal, C., and Lin, M. C. (2014).

Participatory route planning.

In Proceedings of the 22nd ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pages 213–222.

• Wilkie et al., (2015)

Wilkie, D., Sewall, J., Li, W., and Lin, M. C. (2015).

Virtualized traffic at metropolitan scales.

Frontiers in Robotics and AI, 2:11.

• Wilkie et al., (2012)

Wilkie, D., Sewall, J., and Lin, M. C. (2012).

Transforming GIS data into functional road models for large-scale traffic simulation.

IEEE transactions on visualization and computer graphics, 18(6):890–901.

• Wilkie et al., (2013)

Wilkie, D., Sewall, J., and Lin, M. C. (2013).

Flow reconstruction for data-driven traffic animation.

ACM Trans. Graph., 32(4):89:1–89:10.

• Wilkie et al., (2011)

Wilkie, D., van den Berg, J. P., Lin, M. C., and Manocha, D. (2011).

Self-aware traffic route planning.

In AAAI, volume 11, pages 1521–1527.

• Willemsen et al., (2006)

Willemsen, P., Kearney, J. K., and Wang, H. (2006).

Ribbon networks for modeling navigable paths of autonomous agents in virtual environments.

IEEE Transactions on Visualization and Computer Graphics, 12(3):331–342.

• Wolinski et al., (2016)

Wolinski, D., Lin, M., and Pettré, J. (2016).

Warpdriver: context-aware probabilistic motion prediction for crowd simulation.

ACM Trans. Graph., 35(6):164:1–164:11.

• Work et al., (2010)

Work, D., Blandin, S., Tossavainen, O.-P., Piccoli, B., and Bayen, A. (2010).

A traffic model for velocity data assimilation.

Applied Mathematics Research Express, 2010(1):1–35.

• Wu et al., (2018)

Wu, C., Bayen, A. M., and Mehta, A. (2018).

Stabilizing traffic with autonomous vehicles.

In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 1–7.

• Xu et al., (2017)

Xu, H., Gao, Y., Yu, F., and Darrell, T. (2017).

End-to-end learning of driving models from large-scale video datasets.

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3530–3538.

• Yang et al., (1992)

Yang, H., Sasaki, T., Iida, Y., and Asakura, Y. (1992).

Estimation of origin-destination matrices from link traffic counts on congested networks.

Transportation Research Part B: Methodological, 26(6):417–434.

• Yang et al., (2017)

Yang, X., Lu, Y., and Hao, W. (2017).

Origin-destination estimation using probe vehicle trajectory and link counts.

Journal of Advanced Transportation.

• Yuan et al., (2010)

Yuan, J., Zheng, Y., Zhang, C., Xie, X., and Sun, G. (2010).

An interactive-voting based map matching algorithm.

In Proceedings of the 11th International Conference on Mobile Data Management, pages 43–52.

• Zhang and Cho, (2017)

Zhang, J. and Cho, K. (2017).

Query-efficient imitation learning for end-to-end simulated driving.

In AAAI, pages 2891–2897.

• Zhang et al., (2013)

Zhang, J.-D., Xu, J., and Liao, S. S. (2013).

Aggregating and sampling methods for processing GPS data streams for traffic state estimation.

IEEE Transactions on Intelligent Transportation Systems, 14(4):1629–1641.

• Zheng et al., (2011)

Zheng, Y., Liu, Y., Yuan, J., and Xie, X. (2011).

Urban computing with taxicabs.

In Proceedings of the 13th International Conference on Ubiquitous Computing, pages 89–98.

• Zhu et al., (2013)

Zhu, Y., Li, Z., Zhu, H., Li, M., and Zhang, Q. (2013).

A compressive sensing approach to urban traffic estimation with probe vehicles.

Mobile Computing, IEEE Transactions on, 12(11):2289–2302.