TL;DR
HyperSeg introduces a real-time semantic segmentation network with a novel hypernetwork architecture that generates spatially adaptive decoder weights, achieving state-of-the-art accuracy on key benchmarks.
Contribution
It proposes a new hypernetwork design with a nested U-Net and dynamic patch-wise convolutions for efficient, adaptive real-time segmentation.
Findings
Surpasses state-of-the-art on PASCAL VOC 2012
Achieves real-time performance on Cityscapes and CamVid
Uses less conventional blocks for efficiency
Abstract
We present a novel, real-time, semantic segmentation network in which the encoder both encodes and generates the parameters (weights) of the decoder. Furthermore, to allow maximal adaptivity, the weights at each decoder block vary spatially. For this purpose, we design a new type of hypernetwork, composed of a nested U-Net for drawing higher level context features, a multi-headed weight generating module which generates the weights of each block in the decoder immediately before they are consumed, for efficient memory utilization, and a primary network that is composed of novel dynamic patch-wise convolutions. Despite the usage of less-conventional blocks, our architecture obtains real-time performance. In terms of the runtime vs. accuracy trade-off, we surpass state of the art (SotA) results on popular semantic segmentation benchmarks: PASCAL VOC 2012 (val. set) and real-time semantic…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsHyperNetwork · *Communicated@Fast*How Do I Communicate to Expedia? · Concatenated Skip Connection · Convolution · Max Pooling · U-Net
