FastPose-ViT: A Vision Transformer for Real-Time Spacecraft Pose Estimation

TL;DR

FastPose-ViT introduces a Vision Transformer-based method for real-time spacecraft pose estimation that directly regresses 6DoF pose from images, outperforming traditional iterative algorithms and suitable for resource-limited edge devices.

Contribution

The paper presents a novel ViT-based architecture with a new formalism for mapping localized predictions to full images, enabling real-time spacecraft pose estimation on edge hardware.

Findings

Outperforms non-PnP strategies in accuracy.

Achieves ~75 ms latency on NVIDIA Jetson Orin Nano.

Supports up to 33 FPS throughput with concurrent scheduling.

Abstract

Estimating the 6-degrees-of-freedom (6DoF) pose of a spacecraft from a single image is critical for autonomous operations like in-orbit servicing and space debris removal. Existing state-of-the-art methods often rely on iterative Perspective-n-Point (PnP)-based algorithms, which are computationally intensive and ill-suited for real-time deployment on resource-constrained edge devices. To overcome these limitations, we propose FastPose-ViT, a Vision Transformer (ViT)-based architecture that directly regresses the 6DoF pose. Our approach processes cropped images from object bounding boxes and introduces a novel mathematical formalism to map these localized predictions back to the full-image scale. This formalism is derived from the principles of projective geometry and the concept of "apparent rotation", where the model predicts an apparent rotation matrix that is then corrected to find the true orientation. We demonstrate that our method outperforms other non-PnP strategies and achieves performance competitive with state-of-the-art PnP-based techniques on the SPEED dataset. Furthermore, we validate our model's suitability for real-world space missions by quantizing it and deploying it on power-constrained edge hardware. On the NVIDIA Jetson Orin Nano, our end-to-end pipeline achieves a latency of ~75 ms per frame under sequential execution, and a non-blocking throughput of up to 33 FPS when stages are scheduled concurrently.