Convex Online Video Frame Subset Selection using Multiple Criteria for Data Efficient Autonomous Driving

TL;DR

This paper introduces a novel convex and submodular optimization-based online video frame selection method to reduce data volume in autonomous driving training, achieving significant data reduction with maintained performance.

Contribution

It develops a new convex optimization-based multi-criteria subset selection algorithm using a thresholded concave function, improving data efficiency in autonomous driving models.

Findings

Dropped 80% of video frames while completing 100% of episodes.

Reduced training time to less than 30% of full dataset training.

Improved prediction performance of the 'Relative Angle' affordance during turns.

Abstract

Training vision-based Urban Autonomous driving models is a challenging problem, which is highly researched in recent times. Training such models is a data-intensive task requiring the storage and processing of vast volumes of (possibly redundant) driving video data. In this paper, we study the problem of developing data-efficient autonomous driving systems. In this context, we study the problem of multi-criteria online video frame subset selection. We study convex optimization-based solutions and show that they are unable to provide solutions with high weightage to the loss of selected video frames. We design a novel convex optimization-based multi-criteria online subset selection algorithm that uses a thresholded concave function of selection variables. We also propose and study a submodular optimization-based algorithm. Extensive experiments using the driving simulator CARLA show that we are able to drop 80% of the frames while succeeding to complete 100% of the episodes w.r.t. the model trained on 100% data, in the most difficult task of taking turns. This results in a training time of less than 30% compared to training on the whole dataset. We also perform detailed experiments on prediction performances of various affordances used by the Conditional Affordance Learning (CAL) model and show that our subset selection improves performance on the crucial affordance "Relative Angle" during turns.