EndoSensorFusion: Particle Filtering-Based Multi-sensory Data Fusion with Switching State-Space Model for Endoscopic Capsule Robots

TL;DR

This paper introduces EndoSensorFusion, a particle filtering-based multi-sensor fusion method with online sensor reliability estimation and neural network learned kinematics, enhancing localization accuracy and robustness for endoscopic capsule robots.

Contribution

It presents a novel multi-sensor fusion approach combining particle filtering, neural network kinematics, and sensor reliability estimation for improved capsule robot localization.

Findings

Achieves high translational and rotational accuracy in ex-vivo tests.

Effectively detects and handles sensor failures.

Demonstrates robustness suitable for medical applications.

Abstract

A reliable, real time multi-sensor fusion functionality is crucial for localization of actively controlled capsule endoscopy robots, which are an emerging, minimally invasive diagnostic and therapeutic technology for the gastrointestinal (GI) tract. In this study, we propose a novel multi-sensor fusion approach based on a particle filter that incorporates an online estimation of sensor reliability and a non-linear kinematic model learned by a recurrent neural network. Our method sequentially estimates the true robot pose from noisy pose observations delivered by multiple sensors. We experimentally test the method using 5 degree-of-freedom (5-DoF) absolute pose measurement by a magnetic localization system and a 6-DoF relative pose measurement by visual odometry. In addition, the proposed method is capable of detecting and handling sensor failures by ignoring corrupted data, providing the robustness expected of a medical device. Detailed analyses and evaluations are presented using ex-vivo experiments on a porcine stomach model prove that our system achieves high translational and rotational accuracies for different types of endoscopic capsule robot trajectories.