Impact of Bit Allocation Strategies on Machine Learning Performance in Rate Limited Systems

TL;DR

This paper investigates how different bit allocation strategies for quantizing correlated data sources affect machine learning performance in bandwidth-limited systems, introducing a new distortion measure based on Kullback-Leibler divergence.

Contribution

It proposes a novel bit allocation criterion using KL divergence that improves ML decision accuracy without prior data statistics, demonstrated on a neural network controlled inverted pendulum.

Findings

Significant performance improvements with optimized bit allocation.

Higher relevance of certain features for the MLU requires more precise quantization.

Effective in scenarios with limited bandwidth, especially in critical regions.

Abstract

Intelligent entities such as self-driving vehicles, with their data being processed by machine learning units (MLU), are developing into an intertwined part of networks. These units handle distorted input but their sensitivity to noisy observations varies for different input attributes. Since blind transport of massive data burdens the system, identifying and delivering relevant information to MLUs leads in improved system performance and efficient resource utilization. Here, we study the integer bit allocation problem for quantizing multiple correlated sources providing input of a MLU with a bandwidth constraint. Unlike conventional distance measures between original and quantized input attributes, a new Kullback-Leibler divergence based distortion measure is defined to account for accuracy of MLU decisions. The proposed criterion is applicable to many practical cases with no prior knowledge on data statistics and independent of selected MLU instance. Here, we examine an inverted pendulum on a cart with a neural network controller assuming scalar quantization. Simulation results present a significant performance gain, particularly for regions with smaller available bandwidth. Furthermore, the pattern of successful rate allocations demonstrates higher relevancy of some features for the MLU and the need to quantize them with higher accuracy.