4D Spatio-Temporal Convolutional Networks for Object Position Estimation in OCT Volumes

TL;DR

This paper introduces 4D spatio-temporal CNNs for improved object position estimation in OCT volumes, leveraging temporal data to enhance tracking accuracy in surgical applications.

Contribution

It systematically extends 3D CNNs to 4D CNNs for OCT-based object tracking, demonstrating significant accuracy improvements by incorporating temporal information.

Findings

4D CNNs reduce mean absolute error by 30%

Temporal streams improve marker object tracking accuracy

4D approach outperforms single volume 3D CNNs

Abstract

Tracking and localizing objects is a central problem in computer-assisted surgery. Optical coherence tomography (OCT) can be employed as an optical tracking system, due to its high spatial and temporal resolution. Recently, 3D convolutional neural networks (CNNs) have shown promising performance for pose estimation of a marker object using single volumetric OCT images. While this approach relied on spatial information only, OCT allows for a temporal stream of OCT image volumes capturing the motion of an object at high volumes rates. In this work, we systematically extend 3D CNNs to 4D spatio-temporal CNNs to evaluate the impact of additional temporal information for marker object tracking. Across various architectures, our results demonstrate that using a stream of OCT volumes and employing 4D spatio-temporal convolutions leads to a 30% lower mean absolute error compared to single volume processing with 3D CNNs.