Extracting full-field subpixel structural displacements from videos via deep learning

TL;DR

This paper introduces a deep learning framework with novel CNN architectures for real-time, full-field subpixel displacement extraction from videos, effectively handling texture contrast limitations and demonstrating good generalizability.

Contribution

It presents two new CNN architectures trained on phase-based motion data to accurately extract subpixel displacements, considering texture contrast sparsity, and shows their effectiveness across different videos.

Findings

Networks accurately identify pixels with sufficient texture contrast.

The trained models generalize well to various structures.

Subpixel displacements are reliably extracted in real-time.

Abstract

This paper develops a deep learning framework based on convolutional neural networks (CNNs) that enable real-time extraction of full-field subpixel structural displacements from videos. In particular, two new CNN architectures are designed and trained on a dataset generated by the phase-based motion extraction method from a single lab-recorded high-speed video of a dynamic structure. As displacement is only reliable in the regions with sufficient texture contrast, the sparsity of motion field induced by the texture mask is considered via the network architecture design and loss function definition. Results show that, with the supervision of full and sparse motion field, the trained network is capable of identifying the pixels with sufficient texture contrast as well as their subpixel motions. The performance of the trained networks is tested on various videos of other structures to extract the full-field motion (e.g., displacement time histories), which indicates that the trained networks have generalizability to accurately extract full-field subtle displacements for pixels with sufficient texture contrast.