A Theory on AI Uncertainty Based on Rademacher Complexity and Shannon Entropy

TL;DR

This paper explores the theoretical relationship between Rademacher complexity and Shannon entropy to establish criteria for AI classification accuracy and correctness, providing a foundation for future complexity-based assessments.

Contribution

It introduces a novel theoretical framework linking Shannon entropy and Rademacher complexity to evaluate AI neural network uncertainty and correctness in classification tasks.

Findings

Derived criteria for AI correctness based on Shannon entropy.

Established a close relationship between Rademacher complexity and Shannon entropy.

Proposed conditions to guarantee AI classification accuracy.

Abstract

In this paper, we present a theoretical discussion on AI deep learning neural network uncertainty investigation based on the classical Rademacher complexity and Shannon entropy. First it is shown that the classical Rademacher complexity and Shannon entropy is closely related by quantity by definitions. Secondly based on the Shannon mathematical theory on communication [3], we derive a criteria to ensure AI correctness and accuracy in classifications problems. Last but not the least based on Peter Barlette's work, we show both a relaxing condition and a stricter condition to guarantee the correctness and accuracy in AI classification . By elucidating in this paper criteria condition in terms of Shannon entropy based on Shannon theory, it becomes easier to explore other criteria in terms of other complexity measurements such as Vapnik-Cheronenkis, Gaussian complexity by taking advantage of the relations studies results in other references. A close to 0.5 criteria on Shannon entropy is derived in this paper for the theoretical investigation of AI accuracy and correctness for classification problems.