Weakly-supervised Semantic Segmentation in Cityscape via Hyperspectral Image

TL;DR

This paper introduces a weakly-supervised semantic segmentation method using high-resolution hyperspectral images in city scenes, leveraging spectral information to enhance label accuracy and edge fineness without manual annotation.

Contribution

It presents a novel framework that utilizes spectral data from HSIs to refine coarse labels into more accurate semantic segmentations in complex city scenes.

Findings

Achieves highly competitive semantic labels with improved edge fineness.

Refined labels significantly enhance segmentation performance.

Demonstrates the potential of HSIs in high-level visual tasks.

Abstract

High-resolution hyperspectral images (HSIs) contain the response of each pixel in different spectral bands, which can be used to effectively distinguish various objects in complex scenes. While HSI cameras have become low cost, algorithms based on it have not been well exploited. In this paper, we focus on a novel topic, weakly-supervised semantic segmentation in cityscape via HSIs. It is based on the idea that high-resolution HSIs in city scenes contain rich spectral information, which can be easily associated to semantics without manual labeling. Therefore, it enables low cost, highly reliable semantic segmentation in complex scenes. Specifically, in this paper, we theoretically analyze the HSIs and introduce a weakly-supervised HSI semantic segmentation framework, which utilizes spectral information to improve the coarse labels to a finer degree. The experimental results show that our method can obtain highly competitive labels and even have higher edge fineness than artificial fine labels in some classes. At the same time, the results also show that the refined labels can effectively improve the effect of semantic segmentation. The combination of HSIs and semantic segmentation proves that HSIs have great potential in high-level visual tasks.