Relay: A New IR for Machine Learning Frameworks

TL;DR

Relay introduces a high-level, purely-functional, statically-typed intermediate representation designed to improve efficiency, expressiveness, and portability in machine learning frameworks across diverse hardware devices.

Contribution

It presents a novel IR called Relay that balances efficiency, expressiveness, and portability, integrated into the NNVM compiler framework used by MxNet.

Findings

Relay is part of the open source NNVM compiler framework.

Designed as a purely-functional, statically-typed language.

Aims to optimize compilation for heterogeneous hardware.

Abstract

Machine learning powers diverse services in industry including search, translation, recommendation systems, and security. The scale and importance of these models require that they be efficient, expressive, and portable across an array of heterogeneous hardware devices. These constraints are often at odds; in order to better accommodate them we propose a new high-level intermediate representation (IR) called Relay. Relay is being designed as a purely-functional, statically-typed language with the goal of balancing efficient compilation, expressiveness, and portability. We discuss the goals of Relay and highlight its important design constraints. Our prototype is part of the open source NNVM compiler framework, which powers Amazon's deep learning framework MxNet.