Bosch Deep Learning Hardware Benchmark

TL;DR

This paper introduces a specialized deep learning hardware benchmark for embedded hardware accelerators used in autonomous driving, featuring new evaluation granularity, a twofold benchmarking procedure, and extended performance metrics.

Contribution

It presents a novel benchmark tailored for embedded DL inference in autonomous driving, including new evaluation levels, a comprehensive benchmarking process, and extended performance indicators.

Findings

Benchmark reveals hardware-model mismatches

New evaluation granularity improves comparison accuracy

Extended metrics help identify hardware optimization issues

Abstract

The widespread use of Deep Learning (DL) applications in science and industry has created a large demand for efficient inference systems. This has resulted in a rapid increase of available Hardware Accelerators (HWAs) making comparison challenging and laborious. To address this, several DL hardware benchmarks have been proposed aiming at a comprehensive comparison for many models, tasks, and hardware platforms. Here, we present our DL hardware benchmark which has been specifically developed for inference on embedded HWAs and tasks required for autonomous driving. In addition to previous benchmarks, we propose a new granularity level to evaluate common submodules of DL models, a twofold benchmark procedure that accounts for hardware and model optimizations done by HWA manufacturers, and an extended set of performance indicators that can help to identify a mismatch between a HWA and the DL models used in our benchmark.