Applying Unsupervised Semantic Segmentation to High-Resolution UAV Imagery for Enhanced Road Scene Parsing

TL;DR

This paper introduces an unsupervised framework for high-resolution UAV road scene parsing that combines vision language models, SAM, and self-supervised learning to achieve high accuracy without manual annotations.

Contribution

It presents a novel unsupervised approach that leverages vision language models and self-training to effectively parse road scenes from ultra-high resolution UAV images without manual labels.

Findings

Achieves 89.96% mIoU without manual annotations

Effectively identifies and segments road regions in high-resolution UAV images

Surpasses traditional supervised methods in flexibility and accuracy

Abstract

There are two challenges presented in parsing road scenes from UAV images: the complexity of processing high-resolution images and the dependency on extensive manual annotations required by traditional supervised deep learning methods to train robust and accurate models. In this paper, a novel unsupervised road parsing framework that leverages advancements in vision language models with fundamental computer vision techniques is introduced to address these critical challenges. Our approach initiates with a vision language model that efficiently processes ultra-high resolution images to rapidly identify road regions of interest. Subsequent application of the vision foundation model, SAM, generates masks for these regions without requiring category information. A self-supervised learning network then processes these masked regions to extract feature representations, which are clustered using an unsupervised algorithm that assigns unique IDs to each feature cluster. The masked regions are combined with the corresponding IDs to generate initial pseudo-labels, which initiate an iterative self-training process for regular semantic segmentation. Remarkably, the proposed method achieves a mean Intersection over Union (mIoU) of 89.96% on the development dataset without any manual annotation, demonstrating extraordinary flexibility by surpassing the limitations of human-defined categories, and autonomously acquiring knowledge of new categories from the dataset itself.