MaRF: Representing Mars as Neural Radiance Fields

TL;DR

MaRF introduces a neural radiance field-based framework for synthesizing 3D models of Mars' surface from rover images, enabling efficient and generalizable planetary surface reconstruction for exploration tasks.

Contribution

This work presents MaRF, a novel NeRF-based method that uses neural graphics primitives to efficiently reconstruct Mars' surface from rover images, overcoming classical methods' limitations.

Findings

Successfully reconstructed Mars environments from real rover data.

Reduced computational resources using neural graphics primitives.

Demonstrated generalization to unseen scenes and new images.

Abstract

The aim of this work is to introduce MaRF, a novel framework able to synthesize the Martian environment using several collections of images from rover cameras. The idea is to generate a 3D scene of Mars' surface to address key challenges in planetary surface exploration such as: planetary geology, simulated navigation and shape analysis. Although there exist different methods to enable a 3D reconstruction of Mars' surface, they rely on classical computer graphics techniques that incur high amounts of computational resources during the reconstruction process, and have limitations with generalizing reconstructions to unseen scenes and adapting to new images coming from rover cameras. The proposed framework solves the aforementioned limitations by exploiting Neural Radiance Fields (NeRFs), a method that synthesize complex scenes by optimizing a continuous volumetric scene function using a sparse set of images. To speed up the learning process, we replaced the sparse set of rover images with their neural graphics primitives (NGPs), a set of vectors of fixed length that are learned to preserve the information of the original images in a significantly smaller size. In the experimental section, we demonstrate the environments created from actual Mars datasets captured by Curiosity rover, Perseverance rover and Ingenuity helicopter, all of which are available on the Planetary Data System (PDS).