TriangleNet: Edge Prior Augmented Network for Semantic Segmentation through Cross-Task Consistency

TL;DR

This paper introduces TriangleNet, a semantic segmentation model that explicitly enforces cross-task consistency between edge detection and segmentation, resulting in improved accuracy and real-time performance.

Contribution

The paper proposes a decoupled cross-task consistency loss for multi-task learning, significantly enhancing segmentation accuracy and efficiency over baseline models.

Findings

Achieves 77.4% mIoU at 46.2 FPS on Cityscapes

Improves mIoU by 2.88% over baseline

Outperforms baseline on FloodNet dataset

Abstract

This paper addresses the task of semantic segmentation in computer vision, aiming to achieve precise pixel-wise classification. We investigate the joint training of models for semantic edge detection and semantic segmentation, which has shown promise. However, implicit cross-task consistency learning in multi-task networks is limited. To address this, we propose a novel "decoupled cross-task consistency loss" that explicitly enhances cross-task consistency. Our semantic segmentation network, TriangleNet, achieves a substantial 2.88\% improvement over the Baseline in mean Intersection over Union (mIoU) on the Cityscapes test set. Notably, TriangleNet operates at 77.4\% mIoU/46.2 FPS on Cityscapes, showcasing real-time inference capabilities at full resolution. With multi-scale inference, performance is further enhanced to 77.8\%. Furthermore, TriangleNet consistently outperforms the Baseline on the FloodNet dataset, demonstrating its robust generalization capabilities. The proposed method underscores the significance of multi-task learning and explicit cross-task consistency enhancement for advancing semantic segmentation and highlights the potential of multitasking in real-time semantic segmentation.