Resource Efficient Perception for Vision Systems

TL;DR

This paper presents a memory-efficient patch-based framework for high-resolution image perception, enabling effective training and deployment on resource-constrained devices across various vision tasks.

Contribution

It introduces a novel global-local perception framework that reduces memory usage, allowing training of ultra high-resolution images and improving performance on multiple benchmarks.

Findings

Superior performance on 7 benchmarks

Effective on resource-constrained devices like Jetson Nano

Enables training of ultra high-resolution images

Abstract

Despite the rapid advancement in the field of image recognition, the processing of high-resolution imagery remains a computational challenge. However, this processing is pivotal for extracting detailed object insights in areas ranging from autonomous vehicle navigation to medical imaging analyses. Our study introduces a framework aimed at mitigating these challenges by leveraging memory efficient patch based processing for high resolution images. It incorporates a global context representation alongside local patch information, enabling a comprehensive understanding of the image content. In contrast to traditional training methods which are limited by memory constraints, our method enables training of ultra high resolution images. We demonstrate the effectiveness of our method through superior performance on 7 different benchmarks across classification, object detection, and segmentation. Notably, the proposed method achieves strong performance even on resource-constrained devices like Jetson Nano. Our code is available at https://github.com/Visual-Conception-Group/Localized-Perception-Constrained-Vision-Systems.