Tetra-AML: Automatic Machine Learning via Tensor Networks

TL;DR

Tetra-AML introduces an automated framework combining neural architecture search, hyperparameter tuning, and tensor network-based compression to optimize neural networks efficiently, demonstrated on vision tasks with significant resource savings.

Contribution

The paper presents Tetra-AML, a novel toolbox integrating tensor network techniques with automated machine learning for neural architecture search and model compression.

Findings

Outperforms Bayesian optimization on CIFAR-10

Achieves 14.5x memory reduction in ResNet-18

Maintains accuracy with minimal loss during compression

Abstract

Neural networks have revolutionized many aspects of society but in the era of huge models with billions of parameters, optimizing and deploying them for commercial applications can require significant computational and financial resources. To address these challenges, we introduce the Tetra-AML toolbox, which automates neural architecture search and hyperparameter optimization via a custom-developed black-box Tensor train Optimization algorithm, TetraOpt. The toolbox also provides model compression through quantization and pruning, augmented by compression using tensor networks. Here, we analyze a unified benchmark for optimizing neural networks in computer vision tasks and show the superior performance of our approach compared to Bayesian optimization on the CIFAR-10 dataset. We also demonstrate the compression of ResNet-18 neural networks, where we use 14.5 times less memory while losing just 3.2% of accuracy. The presented framework is generic, not limited by computer vision problems, supports hardware acceleration (such as with GPUs and TPUs) and can be further extended to quantum hardware and to hybrid quantum machine learning models.