Memory-aware Scheduling for Complex Wired Networks with Iterative Graph Optimization

TL;DR

This paper introduces an efficient memory-aware scheduling framework for complex neural network graphs, improving scalability and reducing peak memory footprint on resource-constrained devices.

Contribution

It presents an iterative graph fusion algorithm and an ILP-based scheduling method that outperforms prior algorithms in scalability and memory efficiency.

Findings

Reduces peak memory footprint by 13.4% on benchmarks.

Outperforms existing algorithms in scalability and efficiency.

Effective for complex network topologies.

Abstract

Memory-aware network scheduling is becoming increasingly important for deep neural network (DNN) inference on resource-constrained devices. However, due to the complex cell-level and network-level topologies, memory-aware scheduling becomes very challenging. While previous algorithms all suffer from poor scalability, in this paper, we propose an efficient memory-aware scheduling framework based on iterative computation graph optimization. Our framework features an iterative graph fusion algorithm that simplifies the computation graph while preserving the scheduling optimality. We further propose an integer linear programming formulation together with topology-aware variable pruning to schedule the simplified graph efficiently. We evaluate our method against prior-art algorithms on different networks and demonstrate that our method outperforms existing techniques in all the benchmarks, reducing the peak memory footprint by 13.4%, and achieving better scalability for networks with complex network-level topologies.