Three-dimensional multimodal medical imaging system based on free-hand ultrasound and structured light

TL;DR

This paper introduces a novel 3D multimodal medical imaging system combining freehand ultrasound and structured light, achieving high accuracy without registration, suitable for preoperative and intraoperative use.

Contribution

It is the first to integrate freehand ultrasound with structured light 3D reconstruction in a single coordinate system without registration.

Findings

Multimodal error of 0.12 mm demonstrating high accuracy.

System shows high reproducibility in experimental tests.

Potential applications in preoperative and intraoperative settings.

Abstract

We propose a three-dimensional (3D) multimodal medical imaging system that combines freehand ultrasound and structured light 3D reconstruction in a single coordinate system without requiring registration. To the best of our knowledge, these techniques have not been combined before as a multimodal imaging technique. The system complements the internal 3D information acquired with ultrasound, with the external surface measured with the structure light technique. Moreover, the ultrasound probe's optical tracking for pose estimation was implemented based on a convolutional neural network. Experimental results show the system's high accuracy and reproducibility, as well as its potential for preoperative and intraoperative applications. The experimental multimodal error, or the distance from two surfaces obtained with different modalities, was 0.12 mm. The code is available as a Github repository.