Tornado: A Practical And Efficient Heterogeneous Programming Framework For Managed Languages

TL;DR

Tornado is a practical heterogeneous programming framework that dynamically configures applications at runtime, enabling efficient use of accelerators without complex pre-configuration or extensive code modifications.

Contribution

It introduces a runtime system that allows applications to adapt to specific hardware in situ, simplifying heterogeneous programming for managed languages.

Findings

Achieves up to 124 FPS in a real-world vision application

Provides up to 166x speedup over reference implementation

Maintains within 21% of hand-optimized OpenCL performance

Abstract

This paper describes our experiences creating Tornado: a practical and efficient heterogeneous programming framework for managed languages. The novel aspect of Tornado is that it turns the programming of heterogeneous systems from an activity predominantly based on a priori knowledge into one based on a posteriori knowledge. Alternatively put, it simply means developers do not need to overcomplicate their code by catering for all possible eventualities. Instead, Tornado provides the ability to specialize each application for a specific system in situ which avoids the need for it to be pre-configured by the developer. To enable this, Tornado employs a sophisticated runtime system that can dynamically configure all aspects of the application - from selecting which parallelization scheme to apply to specifying which accelerators to use. By using this ability, the end-user, and not the developer, can transparently make use of any available multi-/many-core processor or hardware accelerator. To showcase the impact of Tornado, we implement a real-world computer vision application and deploy it across nine accelerators without having to modify the source code or even explicitly re-compile the application. Using dynamic configuration, we show that our implementation can achieve up to 124 frames per second (FPS) - up to 166x speedup over the reference implementation. Finally, our implementation is always within 21% of a hand-written OpenCL version but avoids much of the programming tedium.