SegBlocks: Block-Based Dynamic Resolution Networks for Real-Time Segmentation
Thomas Verelst, Tinne Tuytelaars

TL;DR
SegBlocks introduces a dynamic resolution approach for real-time image segmentation, reducing computational costs while maintaining high accuracy through block-based processing and reinforcement learning.
Contribution
It presents a novel block-based dynamic resolution network with CUDA modules and a reinforcement learning policy for efficient real-time segmentation.
Findings
Reduces floating-point operations by 60% on SwiftNet-RN18.
Increases inference speed by 50%.
Maintains high accuracy with only 0.3% mIoU decrease.
Abstract
SegBlocks reduces the computational cost of existing neural networks, by dynamically adjusting the processing resolution of image regions based on their complexity. Our method splits an image into blocks and downsamples blocks of low complexity, reducing the number of operations and memory consumption. A lightweight policy network, selecting the complex regions, is trained using reinforcement learning. In addition, we introduce several modules implemented in CUDA to process images in blocks. Most important, our novel BlockPad module prevents the feature discontinuities at block borders of which existing methods suffer, while keeping memory consumption under control. Our experiments on Cityscapes, Camvid and Mapillary Vistas datasets for semantic segmentation show that dynamically processing images offers a better accuracy versus complexity trade-off compared to static baselines of…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Multimodal Machine Learning Applications · Adversarial Robustness in Machine Learning
