Deep Forward and Inverse Perceptual Models for Tracking and Prediction

TL;DR

This paper introduces deep perceptual models for generating high-quality images from robot states and estimating states from images, enhancing tracking and prediction in robotics with real-world validation.

Contribution

It presents novel deep forward and inverse perceptual models that outperform existing methods in image generation and state estimation for robotic systems.

Findings

Deep models produce photo-realistic images from states.

Models outperform standard deconvolutional and GAN methods.

State estimation compares favorably to Extended Kalman Filter.

Abstract

We consider the problems of learning forward models that map state to high-dimensional images and inverse models that map high-dimensional images to state in robotics. Specifically, we present a perceptual model for generating video frames from state with deep networks, and provide a framework for its use in tracking and prediction tasks. We show that our proposed model greatly outperforms standard deconvolutional methods and GANs for image generation, producing clear, photo-realistic images. We also develop a convolutional neural network model for state estimation and compare the result to an Extended Kalman Filter to estimate robot trajectories. We validate all models on a real robotic system.