Using Segmentation Masks in the ICCV 2019 Learning to Drive Challenge

TL;DR

This paper improves autonomous driving predictions by integrating segmentation masks from a pre-trained network into multiple neural network models, achieving second place in the ICCV 2019 challenge.

Contribution

It introduces the use of external segmentation masks and ensemble models for vehicle speed and steering angle prediction in driving scenarios.

Findings

Achieved second best in ICCV 2019 challenge

Ensemble of diverse models improves performance

Segmentation masks enhance prediction accuracy

Abstract

In this work we predict vehicle speed and steering angle given camera image frames. Our key contribution is using an external pre-trained neural network for segmentation. We augment the raw images with their segmentation masks and mirror images. We ensemble three diverse neural network models (i) a CNN using a single image and its segmentation mask, (ii) a stacked CNN taking as input a sequence of images and segmentation masks, and (iii) a bidirectional GRU, extracting image features using a pre-trained ResNet34, DenseNet121 and our own CNN single image model. We achieve the second best performance for MSE angle and second best performance overall, to win 2nd place in the ICCV Learning to Drive challenge. We make our models and code publicly available.