TinySeg: Model Optimizing Framework for Image Segmentation on Tiny Embedded Systems

TL;DR

TinySeg is a framework that optimizes memory usage for image segmentation models on tiny embedded systems, enabling more efficient deployment by reducing peak memory consumption.

Contribution

It introduces a novel memory optimization approach analyzing tensor lifetimes and applying spilling and fused fetching techniques for tiny embedded devices.

Findings

Reduces peak memory usage by 39.3% in tested models.

Enables more efficient image segmentation on resource-constrained devices.

Improves deployment feasibility of complex models on tiny systems.

Abstract

Image segmentation is one of the major computer vision tasks, which is applicable in a variety of domains, such as autonomous navigation of an unmanned aerial vehicle. However, image segmentation cannot easily materialize on tiny embedded systems because image segmentation models generally have high peak memory usage due to their architectural characteristics. This work finds that image segmentation models unnecessarily require large memory space with an existing tiny machine learning framework. That is, the existing framework cannot effectively manage the memory space for the image segmentation models. This work proposes TinySeg, a new model optimizing framework that enables memory-efficient image segmentation for tiny embedded systems. TinySeg analyzes the lifetimes of tensors in the target model and identifies long-living tensors. Then, TinySeg optimizes the memory usage of the target model mainly with two methods: (i) tensor spilling into local or remote storage and (ii) fused fetching of spilled tensors. This work implements TinySeg on top of the existing tiny machine learning framework and demonstrates that TinySeg can reduce the peak memory usage of an image segmentation model by 39.3% for tiny embedded systems.