A Rate-Distortion Framework for Characterizing Semantic Information

TL;DR

This paper introduces a rate-distortion framework that models semantic information by considering an unobservable intrinsic state and an observable extrinsic observation, optimizing encoding under dual distortion constraints.

Contribution

It formulates and solves a novel rate-distortion problem incorporating semantic aspects with dual constraints, providing theoretical insights and case studies.

Findings

Derived the state-observation rate-distortion function.

Analyzed Gaussian intrinsic state estimation case.

Studied binary intrinsic state classification.

Abstract

A rate-distortion problem motivated by the consideration of semantic information is formulated and solved. The starting point is to model an information source as a pair consisting of an intrinsic state which is not observable, corresponding to the semantic aspect of the source, and an extrinsic observation which is subject to lossy source coding. The proposed rate-distortion problem seeks a description of the information source, via encoding the extrinsic observation, under two distortion constraints, one for the intrinsic state and the other for the extrinsic observation. The corresponding state-observation rate-distortion function is obtained, and a few case studies of Gaussian intrinsic state estimation and binary intrinsic state classification are studied.