Aerial View River Landform Video segmentation: A Weakly Supervised Context-aware Temporal Consistency Distillation Approach

TL;DR

This paper introduces a weakly supervised, context-aware, temporal consistency distillation method for aerial river landform video segmentation, effectively reducing labeled data requirements while improving segmentation accuracy and temporal stability.

Contribution

It proposes a novel teacher-student framework with key frame algorithms for weakly supervised learning and temporal consistency in aerial landform segmentation.

Findings

Achieves high mIoU and TC with only 30% labeled data.

Outperforms traditional methods in aerial terrain segmentation.

Demonstrates stable localization of terrain objects.

Abstract

The study of terrain and landform classification through UAV remote sensing diverges significantly from ground vehicle patrol tasks. Besides grappling with the complexity of data annotation and ensuring temporal consistency, it also confronts the scarcity of relevant data and the limitations imposed by the effective range of many technologies. This research substantiates that, in aerial positioning tasks, both the mean Intersection over Union (mIoU) and temporal consistency (TC) metrics are of paramount importance. It is demonstrated that fully labeled data is not the optimal choice, as selecting only key data lacks the enhancement in TC, leading to failures. Hence, a teacher-student architecture, coupled with key frame selection and key frame updating algorithms, is proposed. This framework successfully performs weakly supervised learning and TC knowledge distillation, overcoming the deficiencies of traditional TC training in aerial tasks. The experimental results reveal that our method utilizing merely 30\% of labeled data, concurrently elevates mIoU and temporal consistency ensuring stable localization of terrain objects. Result demo : https://gitlab.com/prophet.ai.inc/drone-based-riverbed-inspection