ChiNet: Deep Recurrent Convolutional Learning for Multimodal Spacecraft Pose Estimation

TL;DR

ChiNet is a deep learning framework that combines CNNs and LSTMs to estimate spacecraft pose from multimodal images, improving accuracy through a multi-strategy training approach and data fusion.

Contribution

The paper introduces ChiNet, a novel deep learning pipeline that integrates CNNs and LSTMs for multimodal spacecraft pose estimation, with a new training strategy and data fusion technique.

Findings

Effective fusion of thermal infrared and RGB data reduces artefacts.

Multi-strategy training improves pose estimation accuracy.

Validated on synthetic and experimental datasets.

Abstract

This paper presents an innovative deep learning pipeline which estimates the relative pose of a spacecraft by incorporating the temporal information from a rendezvous sequence. It leverages the performance of long short-term memory (LSTM) units in modelling sequences of data for the processing of features extracted by a convolutional neural network (CNN) backbone. Three distinct training strategies, which follow a coarse-to-fine funnelled approach, are combined to facilitate feature learning and improve end-to-end pose estimation by regression. The capability of CNNs to autonomously ascertain feature representations from images is exploited to fuse thermal infrared data with red-green-blue (RGB) inputs, thus mitigating the effects of artefacts from imaging space objects in the visible wavelength. Each contribution of the proposed framework, dubbed ChiNet, is demonstrated on a synthetic dataset, and the complete pipeline is validated on experimental data.