Deep Reinforcement Learning for System-on-Chip: Myths and Realities

TL;DR

This paper evaluates the applicability of deep reinforcement learning-based neural schedulers for System-on-Chip resource allocation, identifies key challenges, proposes a novel solution called Eclectic Interaction Matching, and discusses factors influencing performance.

Contribution

The paper demonstrates that existing neural schedulers for cluster computing are ineffective for SoC, introduces EIM to improve neural scheduling, and highlights critical performance-impacting metrics.

Findings

Neural schedulers for cluster computing do not perform well on SoC.

EIM significantly improves neural scheduler performance for SoC.

Performance depends on the ratio of PE switching delay to computation time.

Abstract

Neural schedulers based on deep reinforcement learning (DRL) have shown considerable potential for solving real-world resource allocation problems, as they have demonstrated significant performance gain in the domain of cluster computing. In this paper, we investigate the feasibility of neural schedulers for the domain of System-on-Chip (SoC) resource allocation through extensive experiments and comparison with non-neural, heuristic schedulers. The key finding is three-fold. First, neural schedulers designed for cluster computing domain do not work well for SoC due to i) heterogeneity of SoC computing resources and ii) variable action set caused by randomness in incoming jobs. Second, our novel neural scheduler technique, Eclectic Interaction Matching (EIM), overcomes the above challenges, thus significantly improving the existing neural schedulers. Specifically, we rationalize the underlying reasons behind the performance gain by the EIM-based neural scheduler. Third, we discover that the ratio of the average processing elements (PE) switching delay and the average PE computation time significantly impacts the performance of neural SoC schedulers even with EIM. Consequently, future neural SoC scheduler design must consider this metric as well as its implementation overhead for practical utility.