Introducing SLAMBench, a performance and accuracy benchmarking methodology for SLAM

TL;DR

SLAMBench is a software framework designed to enable quantitative benchmarking of dense SLAM algorithms, focusing on performance, accuracy, and energy consumption across various hardware platforms.

Contribution

It introduces a comprehensive benchmarking methodology and provides implementations of KinectFusion in multiple programming models, facilitating performance-portable research.

Findings

KinectFusion implementations vary significantly in execution time across platforms.

Energy efficiency analysis reveals trade-offs between performance and power consumption.

SLAMBench enables reliable accuracy comparison using synthetic RGB-D datasets.

Abstract

Real-time dense computer vision and SLAM offer great potential for a new level of scene modelling, tracking and real environmental interaction for many types of robot, but their high computational requirements mean that use on mass market embedded platforms is challenging. Meanwhile, trends in low-cost, low-power processing are towards massive parallelism and heterogeneity, making it difficult for robotics and vision researchers to implement their algorithms in a performance-portable way. In this paper we introduce SLAMBench, a publicly-available software framework which represents a starting point for quantitative, comparable and validatable experimental research to investigate trade-offs in performance, accuracy and energy consumption of a dense RGB-D SLAM system. SLAMBench provides a KinectFusion implementation in C++, OpenMP, OpenCL and CUDA, and harnesses the ICL-NUIM dataset of synthetic RGB-D sequences with trajectory and scene ground truth for reliable accuracy comparison of different implementation and algorithms. We present an analysis and breakdown of the constituent algorithmic elements of KinectFusion, and experimentally investigate their execution time on a variety of multicore and GPUaccelerated platforms. For a popular embedded platform, we also present an analysis of energy efficiency for different configuration alternatives.