TwistSLAM: Constrained SLAM in Dynamic Environment

TL;DR

TwistSLAM is a novel SLAM system that jointly estimates camera poses, world structure, and moving object velocities in dynamic environments using semantic clustering and constrained bundle adjustment.

Contribution

It introduces a semantic, dynamic stereo SLAM approach with inter-cluster constraints based on mechanical joints for improved tracking in dynamic scenes.

Findings

Outperforms state-of-the-art methods on KITTI dataset

Jointly estimates camera and object trajectories

Handles dynamic objects effectively

Abstract

Classical visual simultaneous localization and mapping (SLAM) algorithms usually assume the environment to be rigid. This assumption limits the applicability of those algorithms as they are unable to accurately estimate the camera poses and world structure in real life scenes containing moving objects (e.g. cars, bikes, pedestrians, etc.). To tackle this issue, we propose TwistSLAM: a semantic, dynamic and stereo SLAM system that can track dynamic objects in the environment. Our algorithm creates clusters of points according to their semantic class. Thanks to the definition of inter-cluster constraints modeled by mechanical joints (function of the semantic class), a novel constrained bundle adjustment is then able to jointly estimate both poses and velocities of moving objects along with the classical world structure and camera trajectory. We evaluate our approach on several sequences from the public KITTI dataset and demonstrate quantitatively that it improves camera and object tracking compared to state-of-the-art approaches.