Deep Learning based Virtual Point Tracking for Real-Time Target-less Dynamic Displacement Measurement in Railway Applications

TL;DR

This paper introduces a deep learning-based virtual point tracking method for real-time, target-less dynamic displacement measurement in railway applications, overcoming limitations of traditional photogrammetry techniques in complex scenes.

Contribution

It presents a novel approach combining deep learning, automatic calibration, and domain knowledge for real-time displacement measurement without optical targets.

Findings

Achieves over 30 frames per second processing speed.

Demonstrates robustness in noisy, complex railway environments.

Outperforms baseline template matching in accuracy and speed.

Abstract

In the application of computer-vision based displacement measurement, an optical target is usually required to prove the reference. In the case that the optical target cannot be attached to the measuring objective, edge detection, feature matching and template matching are the most common approaches in target-less photogrammetry. However, their performance significantly relies on parameter settings. This becomes problematic in dynamic scenes where complicated background texture exists and varies over time. To tackle this issue, we propose virtual point tracking for real-time target-less dynamic displacement measurement, incorporating deep learning techniques and domain knowledge. Our approach consists of three steps: 1) automatic calibration for detection of region of interest; 2) virtual point detection for each video frame using deep convolutional neural network; 3) domain-knowledge based rule engine for point tracking in adjacent frames. The proposed approach can be executed on an edge computer in a real-time manner (i.e. over 30 frames per second). We demonstrate our approach for a railway application, where the lateral displacement of the wheel on the rail is measured during operation. We also implement an algorithm using template matching and line detection as the baseline for comparison. The numerical experiments have been performed to evaluate the performance and the latency of our approach in the harsh railway environment with noisy and varying backgrounds.