Extracting and Analyzing Rail Crossing Behavior Signatures from Videos using Tensor Methods

TL;DR

This paper introduces a tensor-based video analysis method to identify and compare driver behavior patterns at railway crossings across different locations and times, aiding safety improvements.

Contribution

It presents a novel multi-view tensor decomposition framework using TimeSformer embeddings to analyze and cluster crossing behaviors across multiple sites.

Findings

Location influences behavior more than time of day

Approach phase behavior is highly discriminative

Crossings form distinct behavioral clusters

Abstract

Railway crossings present complex safety challenges where driver behavior varies by location, time, and conditions. Traditional approaches analyze crossings individually, limiting the ability to identify shared behavioral patterns across locations. We propose a multi-view tensor decomposition framework that captures behavioral similarities across three temporal phases: Approach (warning activation to gate lowering), Waiting (gates down to train passage), and Clearance (train passage to gate raising). We analyze railway crossing videos from multiple locations using TimeSformer embeddings to represent each phase. By constructing phase-specific similarity matrices and applying non-negative symmetric CP decomposition, we discover latent behavioral components with distinct temporal signatures. Our tensor analysis reveals that crossing location appears to be a stronger determinant of behavior patterns than time of day, and that approach-phase behavior provides particularly discriminative signatures. Visualization of the learned component space confirms location-based clustering, with certain crossings forming distinct behavioral clusters. This automated framework enables scalable pattern discovery across multiple crossings, providing a foundation for grouping locations by behavioral similarity to inform targeted safety interventions.