# Parallelized Interactive Machine Learning on Autonomous Vehicles

## TL;DR

This paper presents a parallelized approach to interactive machine learning for autonomous vehicles, leveraging human demonstrations to accelerate deep reinforcement learning training and improve policy performance.

## Contribution

It introduces a method combining human demonstration pre-training with DQN, enhancing training speed and policy complexity in autonomous vehicle simulations.

## Key findings

- Pre-training with human demonstrations outperforms DQN alone in feature discovery.
- Initializing DQN with a pre-trained model significantly reduces training time.
- Human suggestions during training accelerate convergence and enable learning complex policies.

## Abstract

Deep reinforcement learning (deep RL) has achieved superior performance in complex sequential tasks by learning directly from image input. A deep neural network is used as a function approximator and requires no specific state information. However, one drawback of using only images as input is that this approach requires a prohibitively large amount of training time and data for the model to learn the state feature representation and approach reasonable performance. This is not feasible in real-world applications, especially when the data are expansive and training phase could introduce disasters that affect human safety. In this work, we use a human demonstration approach to speed up training for learning features and use the resulting pre-trained model to replace the neural network in the deep RL Deep Q-Network (DQN), followed by human interaction to further refine the model. We empirically evaluate our approach by using only a human demonstration model and modified DQN with human demonstration model included in the Microsoft AirSim car simulator. Our results show that (1) pre-training with human demonstration in a supervised learning approach is better and much faster at discovering features than DQN alone, (2) initializing the DQN with a pre-trained model provides a significant improvement in training time and performance even with limited human demonstration, and (3) providing the ability for humans to supply suggestions during DQN training can speed up the network's convergence on an optimal policy, as well as allow it to learn more complex policies that are harder to discover by random exploration.