Robust Ego and Object 6-DoF Motion Estimation and Tracking

TL;DR

This paper introduces a robust multi-body visual odometry framework that combines semantic segmentation and optical flow to improve the accuracy of ego and object motion estimation in dynamic scenes, validated on KITTI datasets.

Contribution

The paper presents a novel joint optimization method for SE(3) motion and optical flow, enhancing dynamic scene tracking accuracy in visual odometry.

Findings

Improved motion estimation accuracy demonstrated on KITTI datasets.

Effective integration of semantic segmentation and optical flow.

Open-source implementation provided for community use.

Abstract

The problem of tracking self-motion as well as motion of objects in the scene using information from a camera is known as multi-body visual odometry and is a challenging task. This paper proposes a robust solution to achieve accurate estimation and consistent track-ability for dynamic multi-body visual odometry. A compact and effective framework is proposed leveraging recent advances in semantic instance-level segmentation and accurate optical flow estimation. A novel formulation, jointly optimizing SE(3) motion and optical flow is introduced that improves the quality of the tracked points and the motion estimation accuracy. The proposed approach is evaluated on the virtual KITTI Dataset and tested on the real KITTI Dataset, demonstrating its applicability to autonomous driving applications. For the benefit of the community, we make the source code public.