An easy zero-shot learning combination: Texture Sensitive Semantic Segmentation IceHrNet and Advanced Style Transfer Learning Strategy

TL;DR

This paper introduces a novel zero-shot semantic segmentation approach using style transfer and a specialized texture-sensitive network, achieving high accuracy on river ice imagery without target domain training data.

Contribution

The work presents IceHrNet, a high-resolution texture fusion segmentation network, and an advanced style transfer strategy enabling effective zero-shot transfer learning across domains.

Findings

Achieved 87% mIoU in zero-shot river ice segmentation

IceHrNet outperformed state-of-the-art methods on texture-focused datasets

Zero-shot transfer improved by 22% mIoU using the proposed style transfer strategy

Abstract

We proposed an easy method of Zero-Shot semantic segmentation by using style transfer. In this case, we successfully used a medical imaging dataset (Blood Cell Imagery) to train a model for river ice semantic segmentation. First, we built a river ice semantic segmentation dataset IPC_RI_SEG using a fixed camera and covering the entire ice melting process of the river. Second, a high-resolution texture fusion semantic segmentation network named IceHrNet is proposed. The network used HRNet as the backbone and added ASPP and Decoder segmentation heads to retain low-level texture features for fine semantic segmentation. Finally, a simple and effective advanced style transfer learning strategy was proposed, which can perform zero-shot transfer learning based on cross-domain semantic segmentation datasets, achieving a practical effect of 87% mIoU for semantic segmentation of river ice without target training dataset (25% mIoU for None Stylized, 65% mIoU for Conventional Stylized, our strategy improved by 22%). Experiments showed that the IceHrNet outperformed the state-of-the-art methods on the texture-focused dataset IPC_RI_SEG, and achieved an excellent result on the shape-focused river ice datasets. In zero-shot transfer learning, IceHrNet achieved an increase of 2 percentage points compared to other methods. Our code and model are published on https://github.com/PL23K/IceHrNet.