Semi-Supervised Training of Optical Flow Convolutional Neural Networks in Ultrasound Elastography

TL;DR

This paper introduces an unsupervised fine-tuning method for CNN-based optical flow in ultrasound elastography, improving displacement estimation accuracy by leveraging large unlabeled ultrasound datasets and addressing unique ultrasound data challenges.

Contribution

It presents a novel unsupervised fine-tuning approach specifically designed for ultrasound elastography, overcoming limitations of pre-trained networks and expensive simulations.

Findings

Significant performance improvement in displacement estimation.

Reduction of artifacts compared to networks trained on natural images.

Effective use of large unlabeled ultrasound datasets for fine-tuning.

Abstract

Convolutional Neural Networks (CNN) have been found to have great potential in optical flow problems thanks to an abundance of data available for training a deep network. The displacement estimation step in UltraSound Elastography (USE) can be viewed as an optical flow problem. Despite the high performance of CNNs in optical flow, they have been rarely used for USE due to unique challenges that both input and output of USE networks impose. Ultrasound data has much higher high-frequency content compared to natural images. The outputs are also drastically different, where displacement values in USE are often smooth without sharp motions or discontinuities. The general trend is currently to use pre-trained networks and fine-tune them on a small simulation ultrasound database. However, realistic ultrasound simulation is computationally expensive. Also, the simulation techniques do not model complex motions, nonlinear and frequency-dependent acoustics, and many sources of artifact in ultrasound imaging. Herein, we propose an unsupervised fine-tuning technique which enables us to employ a large unlabeled dataset for fine-tuning of a CNN optical flow network. We show that the proposed unsupervised fine-tuning method substantially improves the performance of the network and reduces the artifacts generated by networks trained on computer vision databases.