Pose2RGBD. Generating Depth and RGB images from absolute positions
Mihai Cristian P\^irvu

TL;DR
This paper introduces Pose2RGBD, a neural network-based method that generates RGBD images from pose data, enabling scene reconstruction and navigation, with new datasets and an unsupervised training approach.
Contribution
It presents a novel neural rendering technique for RGBD generation from pose, along with two new datasets and an unsupervised dataset creation method.
Findings
Successfully generates RGBD images from pose data.
Creates synthetic and real-world datasets for training.
Demonstrates scene navigation using generated RGBD images.
Abstract
We propose a method at the intersection of Computer Vision and Computer Graphics fields, which automatically generates RGBD images using neural networks, based on previously seen and synchronized video, depth and pose signals. Since the models must be able to reconstruct both texture (RGB) and structure (Depth), it creates an implicit representation of the scene, as opposed to explicit ones, such as meshes or point clouds. The process can be thought of as neural rendering, where we obtain a function f : Pose -> RGBD, which we can use to navigate through the generated scene, similarly to graphics simulations. We introduce two new datasets, one based on synthetic data with full ground truth information, while the other one being recorded from a drone flight in an university campus, using only video and GPS signals. Finally, we propose a fully unsupervised method of generating datasets…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Robotics and Sensor-Based Localization · Human Pose and Action Recognition
