Image and AIS Data Fusion Technique for Maritime Computer Vision Applications

TL;DR

This paper presents a novel technique that fuses AIS data with vessel images to enhance dataset quality for maritime applications, improving vessel detection and trajectory prediction accuracy.

Contribution

The paper introduces a new AIS-image fusion method and a comprehensive dataset, advancing maritime computer vision by enabling better vessel detection and data association.

Findings

Achieved 74.76% overall association accuracy.

Fixed camera association accuracy reached 85.06%.

Demonstrated effectiveness for dataset creation and vessel tracking.

Abstract

Deep learning object detection methods, like YOLOv5, are effective in identifying maritime vessels but often lack detailed information important for practical applications. In this paper, we addressed this problem by developing a technique that fuses Automatic Identification System (AIS) data with vessels detected in images to create datasets. This fusion enriches ship images with vessel-related data, such as type, size, speed, and direction. Our approach associates detected ships to their corresponding AIS messages by estimating distance and azimuth using a homography-based method suitable for both fixed and periodically panning cameras. This technique is useful for creating datasets for waterway traffic management, encounter detection, and surveillance. We introduce a novel dataset comprising of images taken in various weather conditions and their corresponding AIS messages. This dataset offers a stable baseline for refining vessel detection algorithms and trajectory prediction models. To assess our method's performance, we manually annotated a portion of this dataset. The results are showing an overall association accuracy of 74.76 %, with the association accuracy for fixed cameras reaching 85.06 %. This demonstrates the potential of our approach in creating datasets for vessel detection, pose estimation and auto-labelling pipelines.