Questions, relevance and relative entropy

TL;DR

This paper introduces a novel approach to representing questions as probability distributions, using relative entropy to measure relevance between questions, with potential applications in statistical physics.

Contribution

It proposes representing questions as probability distributions and derives a relevance measure based on relative entropy, linking question relevance to information theory.

Findings

Relevance between questions is quantified by relative entropy.

Questions can be represented by probability distributions.

Application to statistical physics demonstrates the approach.

Abstract

What is a question? According to Cox a question can be identified with the set of assertions that constitute possible answers. In this paper we propose a different approach that combines the notion that questions are requests for information with the notion that probability distributions represent uncertainties resulting from lack of information. This suggests that to each probability distribution one can naturally associate that particular question which requests the information that is missing and vice-versa. We propose to represent questions q by probability distributions Next we consider how questions relate to each other: to what extent is finding the answer to one question relevant to answering another? A natural measure of relevance is derived by requiring that it satisfy three desirable features (three axioms). We find that the relevance of a question q to another question Q turns out to be the relative entropy S[q,Q] of the corresponding distributions. An application to statistical physics is briefly considered.