Information Measure Similarity Theory: Message Importance Measure via Shannon Entropy

TL;DR

This paper introduces the message importance measure (MIM) as a probabilistic tool for analyzing rare events, exploring its properties, similarities to Shannon entropy, and applications in information processing, transmission, and compression.

Contribution

It constructs a system model for MIM, proposes message importance loss and capacity, and extends Shannon theory to include message importance considerations.

Findings

MIM has three operational regions based on parameter $\

,

,

Abstract

Rare events attract more attention and interests in many scenarios of big data such as anomaly detection and security systems. To characterize the rare events importance from probabilistic perspective, the message importance measure (MIM) is proposed as a kind of semantics analysis tool. Similar to Shannon entropy, the MIM has its special functional on information processing, in which the parameter $\varpi$ of MIM plays a vital role. Actually, the parameter $\varpi$ dominates the properties of MIM, based on which the MIM has three work regions where the corresponding parameters satisfy $ 0 \le \varpi \le 2/\max\{p(x_i)\}$, $\varpi > 2/\max\{p(x_i)\}$ and $\varpi < 0$ respectively. Furthermore, in the case $ 0 \le \varpi \le 2/\max\{p(x_i)\}$, there are some similarity between the MIM and Shannon entropy in the information compression and transmission, which provide a new viewpoint for information theory. This paper first constructs a system model with message importance measure and proposes the message importance loss to enrich the information processing strategies. Moreover, we propose the message importance loss capacity to measure the information importance harvest in a transmission. Furthermore, the message importance distortion function is presented to give an upper bound of information compression based on message importance measure. Additionally, the bitrate transmission constrained by the message importance loss is investigated to broaden the scope for Shannon information theory.