Needle Segmentation Using GAN: Restoring Thin Instrument Visibility in Robotic Ultrasound

TL;DR

This paper presents a GAN-based method for segmenting and restoring visibility of needles in robotic ultrasound, enabling precise needle tracking and automatic repositioning during ultrasound-guided procedures.

Contribution

It introduces a novel adversarial segmentation framework combined with a robot-assisted system for automatic needle visibility restoration in ultrasound imaging.

Findings

Precise needle segmentation with tip error of 0.37mm and angle error of 1.19°

Successful needle visibility restoration in all 45 trials

Effective automatic repositioning with minimal errors

Abstract

Ultrasound-guided percutaneous needle insertion is a standard procedure employed in both biopsy and ablation in clinical practices. However, due to the complex interaction between tissue and instrument, the needle may deviate from the in-plane view, resulting in a lack of close monitoring of the percutaneous needle. To address this challenge, we introduce a robot-assisted ultrasound (US) imaging system designed to seamlessly monitor the insertion process and autonomously restore the visibility of the inserted instrument when misalignment happens. To this end, the adversarial structure is presented to encourage the generation of segmentation masks that align consistently with the ground truth in high-order space. This study also systematically investigates the effects on segmentation performance by exploring various training loss functions and their combinations. When misalignment between the probe and the percutaneous needle is detected, the robot is triggered to perform transverse searching to optimize the positional and rotational adjustment to restore needle visibility. The experimental results on ex-vivo porcine samples demonstrate that the proposed method can precisely segment the percutaneous needle (with a tip error of $0.37\pm0.29mm$ and an angle error of $1.19\pm 0.29^{\circ}$). Furthermore, the needle appearance can be successfully restored under the repositioned probe pose in all 45 trials, with repositioning errors of $1.51\pm0.95mm$ and $1.25\pm0.79^{\circ}$. from latex to text with math symbols