# Real-Time Panoramic Surveillance Video Stitching Method for Complex Industrial Environments

## TL;DR

This paper introduces a real-time video stitching method for industrial environments that improves accuracy and visual quality despite challenging conditions.

## Contribution

A novel video stitching method integrating ECA and CA modules with ResNet for better feature extraction and a new loss function for improved registration and fusion.

## Key findings

- The proposed method achieves RMSE, PSNR, and SSIM values of 1.965, 25.338, and 0.8366 in image registration experiments.
- The new image fusion method produces smoother seam transitions and avoids moving objects in overlapping regions.
- The real-time stitching frame rate reaches 23 fps, suitable for industrial surveillance.

## Abstract

In complex industrial environments, surveillance videos often exhibit large parallax, low illumination, low texture, and low overlap rate, making it difficult to extract reliable image feature points and consequently leading to video suboptimal stitching performance. To address these challenges, this study proposes a real-time panoramic surveillance video stitching method specifically designed for complex industrial scenarios. In the image registration stage, the Efficient Channel Attention (ECA) and Channel Attention (CA) modules are integrated with ResNet to enhance the feature extraction layers of the UDIS algorithm, thereby improving feature extraction and matching accuracy. A loss function incorporating similarity loss Lsim and smoothness loss Lsmooth is designed to optimize registration errors. In the image fusion stage, gradient terms and motion terms are introduced for improving the energy function of the optimal seam line, enabling the optimal seam line to avoid moving objects in overlapping regions and thus achieve video stitching. Experimental validation is conducted by comparing the proposed image registration method with SIFT + RANSAC, UDIS, UDIS++, and NIS, and the proposed image fusion method with weighted average fusion, dynamic programming, and graph cut. The results show that, in image registration experiments, the proposed method achieves RMSE, PSNR, and SSIM values of 1.965, 25.338, and 0.8366, respectively. In image fusion experiments, the seam transition is smoother and effectively avoids moving objects, significantly improving the visual quality of the stitched videos. Moreover, the real-time stitching frame rate reaches 23 fps, meeting the real-time requirements of industrial surveillance applications.