TreeTensor: Boost AI System on Nested Data with Constrained Tree-Like Tensor

TL;DR

TreeTensor introduces a hierarchical tensor data structure that efficiently handles nested, multi-modal data in AI systems, enabling seamless integration with existing libraries and improving computational efficiency in complex tasks.

Contribution

We propose TreeTensor, a novel nested data container that models hierarchical data relationships with minimal overhead, compatible with popular machine learning libraries.

Findings

Enables arbitrary functions on nested data with near-zero cost

Demonstrates effectiveness in complex AI systems like AlphaStar

Exhibits excellent runtime efficiency without overhead

Abstract

Tensor is the most basic and essential data structure of nowadays artificial intelligence (AI) system. The natural properties of Tensor, especially the memory-continuity and slice-independence, make it feasible for training system to leverage parallel computing unit like GPU to process data simultaneously in batch, spatial or temporal dimensions. However, if we look beyond perception tasks, the data in a complicated cognitive AI system usually has hierarchical structures (i.e. nested data) with various modalities. They are inconvenient and inefficient to program directly with conventional Tensor with fixed shape. To address this issue, we summarize two main computational patterns of nested data, and then propose a general nested data container: TreeTensor. Through various constraints and magic utilities of TreeTensor, one can apply arbitrary functions and operations to nested data with almost zero cost, including some famous machine learning libraries, such as Scikit-Learn, Numpy and PyTorch. Our approach utilizes a constrained tree-structure perspective to systematically model data relationships, and it can also easily be combined with other methods to extend more usages, such as asynchronous execution and variable-length data computation. Detailed examples and benchmarks show TreeTensor not only provides powerful usability in various problems, especially one of the most complicated AI systems at present: AlphaStar for StarCraftII, but also exhibits excellent runtime efficiency without any overhead. Our project is available at https://github.com/opendilab/DI-treetensor.