Patient-Specific Domain Adaptation for Fast Optical Flow Based on Teacher-Student Knowledge Transfer

TL;DR

This paper introduces a patient-specific domain adaptation method for optical flow in computer-aided surgery, using teacher-student knowledge transfer to achieve high accuracy and real-time speed in tissue motion tracking.

Contribution

It proposes a fast, patient-specific fine-tuning approach for optical flow models using teacher-student learning, enabling accurate, real-time tissue motion estimation in surgery.

Findings

Student model achieves accuracy comparable to teacher model

Fine-tuning takes only minutes, suitable for intraoperative use

Method enables real-time, patient-specific motion tracking in CAS

Abstract

Fast motion feedback is crucial in computer-aided surgery (CAS) on moving tissue. Image-assistance in safety-critical vision applications requires a dense tracking of tissue motion. This can be done using optical flow (OF). Accurate motion predictions at high processing rates lead to higher patient safety. Current deep learning OF models show the common speed vs. accuracy trade-off. To achieve high accuracy at high processing rates, we propose patient-specific fine-tuning of a fast model. This minimizes the domain gap between training and application data, while reducing the target domain to the capability of the lower complex, fast model. We propose to obtain training sequences pre-operatively in the operation room. We handle missing ground truth, by employing teacher-student learning. Using flow estimations from teacher model FlowNet2 we specialize a fast student model FlowNet2S on the patient-specific domain. Evaluation is performed on sequences from the Hamlyn dataset. Our student model shows very good performance after fine-tuning. Tracking accuracy is comparable to the teacher model at a speed up of factor six. Fine-tuning can be performed within minutes, making it feasible for the operation room. Our method allows to use a real-time capable model that was previously not suited for this task. This method is laying the path for improved patient-specific motion estimation in CAS.