Convex Stochastic Dominance in Bayesian Localization, Filtering and Controlled Sensing POMDPs
Vikram Krishnamurthy
TL;DR
This paper establishes conditions under which Bayesian localization, filtering, and controlled sensing estimators can be compared using convex stochastic dominance, simplifying error analysis and policy bounds in POMDPs.
Contribution
It introduces new conditions on observation distributions ensuring convex dominance of estimates, enabling easier comparison of estimator errors and policy bounds in POMDPs.
Findings
Convex dominance conditions are derived for Bayesian estimators.
Optimal policies are lower bounded by myopic policies under certain conditions.
Numerical examples demonstrate the dominance relations in sensor capacity and estimation accuracy.
Abstract
This paper provides conditions on the observation probability distribution in Bayesian localization and optimal filtering so that the conditional mean estimate satisfies convex stochastic dominance. Convex dominance allows us to compare the unconditional mean square error between two optimal Bayesian state estimators over arbitrary time horizons instead of using brute force Monte-Carlo computations. The proof uses two key ideas from microeconomics, namely, integral precision dominance and aggregation of single crossing.The convex dominance result is then used to give sufficient conditions so that the optimal policy of a controlled sensing two-state partially observed Markov decision process (POMDP) is lower bounded by a myopic policy. Numerical examples are presented where the Shannon capacity of the observation distribution using one sensor dominates that of another, and convex…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Convex Stochastic Dominance in Bayesian Localization, Filtering and Controlled Sensing POMDPs
Vikram Krishnamurthy, *Fellow IEEE
*Electrical & Computer Engineering,
Cornell University, USA.
[email protected] This research was funded in part by the U.S. Army Research Office under grant W911NF-19-1-0365, U.S. Air Force Office of Scientific Research under grant FA9550-18-1-0007 and National Science Foundation under grant 1714180.
Abstract
This paper provides conditions on the observation probability distribution in Bayesian localization and optimal filtering so that the conditional mean estimate satisfies convex stochastic dominance. Convex dominance allows us to compare the unconditional mean square error between two optimal Bayesian state estimators over arbitrary time horizons instead of using brute force Monte-Carlo computations. The proof uses two key ideas from microeconomics, namely, integral precision dominance and aggregation of single crossing. The convex dominance result is then used to give sufficient conditions so that the optimal policy of a controlled sensing two-state partially observed Markov decision process (POMDP) is lower bounded by a myopic policy. Numerical examples are presented where the Shannon capacity of the observation distribution using one sensor dominates that of another, and convex dominance holds but Blackwell dominance does not hold. These illustrate the usefulness of the main result in localization, filtering and controlled sensing applications.
Keywords: Convex dominance, mean squared error, integral precision, aggregation of single crossing, Bayesian localization, optimal filtering, Hidden Markov Model filtering, POMDP, controlled sensing, Blackwell dominance
I Introduction
Consider the following Bayesian localization problem: an underlying random variable with prior is observed via the discrete time noisy observation process where each observation has conditional cumulative distribution function (cdf) . (We use upper case for random variables and lower case for realizations.) Bayesian localization is concerned with recursively computing the posterior distribution , of the state given observation sample path sequence and prior . The posterior distribution is then used to compute the conditional mean estimate of the state given observations as
[TABLE]
where we have indicated the explicit dependence on the prior .
Let denote the sequence of random variables . A natural question is: how accurate is the conditional mean state estimate ? Clearly is the minimum mean square error estimate (more generally it minimizes a Bregman loss), i.e., for all priors ,
[TABLE]
over the class of all Borel functions . But unfortunately, apart from the well known linear Gaussian case,111In the linear Gaussian case, the MSE is computed by the Kalman filter covariance update (Riccati equation) which is completely determined by the model parameters. can only be estimated222Computing the conditional based on a specific observation sample path given the posterior is straightforward but not useful since it only holds for the specific sample path . We are interested in characterizing its expectation, i.e., the unconditional MSE. via Monte-Carlo simulations. Thus for general state space models, there is strong motivation to derive analytical results that compare for different observation models. Specifically consider two sensors, where sensor 1 records observation random variables of state with conditional distribution , and sensor 2 records observation random variables of with conditional distribution , . Then which sensor yields a smaller MSE for the conditional mean estimate at time ?
In this paper, we give sufficient conditions on the observation probabilities so that the conditional mean estimate of sensor 1 is convex stochastic dominated by the estimate of sensor 2. Informally, our main result is:
Theorem A**.**
(Informal) Consider two sensor observation models with the observation process and generated by cdfs and , respectively. Suppose satisfy a single crossing and signed-ratio monotonicity condition (defined in Sec.II-B). Then convex stochastic dominance holds for the conditional mean: , i.e., for any convex function and prior ,
[TABLE]
Here denotes expectation wrt the joint distribution of . Therefore,333Note . So clearly (1) with implies . for all time .
Theorem 3 says that localization using sensor 2 is always more accurate than using sensor 1 for any prior and this holds globally for all time . To the best of our knowledge this result is new. Convex stochastic dominance (1) in a Bayesian framework has been studied extensively in economics under the area of integral precision dominance, see [1, 2]. Theorem 3 asserts convex dominance of the conditional mean for all , i.e., a global property. The proof involves combining two powerful results introduced recently in economics: integral precision dominance (which ensures that Theorem 3 holds for ) and signed ratio monotonicity [3] (which makes Theorem 3 hold globally for all ). The usefulness of Theorem 3 stems from the fact that checking (1) numerically is impossible since it involves checking over a continuum of priors and evaluating intractable multidimensional integrals for the expected value.
The intuition behind (1) is that of integral precision: if the observation is noisy, then the posterior is concentrated around the prior while if the observation is more informative, then the posterior is more dispersed from the prior (large variance). This in turn implies that the noisy observation incurs a larger MSE. In this paper, we show that Theorem 3 holds if (scalar valued) or finite state. Intuitively, if sensor 1 has a higher noise variance than sensor 2, then (1) holds - we will interpret this in terms of stochastic dispersion dominance in Sec.III-B. But there are many other interesting cases where (1) holds; the case with finite observation alphabets is particularly interesting, since there is no noise variance interpretation in that case (the interpretation is in terms of Shannon capacity). The single crossing assumption and signed monotonicity condition in Theorem 3 are straightforward to check compared to the well known Blackwell dominance [4, 5] (see Definition 4) which requires factorization of probability measures; and they hold in several new examples where Blackwell dominance does not. For example, Blackwell dominance does not, in general, hold globally for all ; due to lack of commutativity of matrix multiplication.
Applications in optimal filtering and controlled sensing. Since Theorem 3 applies to any convex function, it has more applications than just characterizing the mean square error of Bayesian localization.
As a first application, we will show that convex dominance applies to the one-step optimal (Bayesian) filtering update in a two-time scale model. That is, consider a Markov process which evolves over the slow time scale with transition kernel , and is observed in noise via the observation process at a fast time scale. So at each time , we obtain multiple fast time scale observations denoted as the vector for some integer where each component is conditionally independent of . Then one step of the optimal filter updates the posterior distribution given . Then the conditional mean is determined by and and denoted as .
Theorem B**.**
(Optimal Filtering). Under the conditions of Theorem 3, convex dominance holds for the conditional mean for one step of the optimal filter. That is, for any convex function and any prior ,
[TABLE]
Therefore .
Thus the optimal filter with sensor 2 is always more accurate than sensor 1. Similar to the discussion for Theorem 3, Theorem B is useful since in general, exact computation of the expectations is impossible (apart from the linear Gaussian case involving the Kalman filter). The classical way of establishing (2) is Blackwell dominance. The main point is that Theorem B covers several cases where Blackwell dominance does not hold (even for ). Moreover, for , in general Blackwell dominance will not hold.
As a second application, we will show that Theorem B is a crucial step in constructing myopic bounds for the optimal policy of controlled sensing partially observed Markov decision processes (POMDPs). In controlled sensing POMDPs, the observation probabilities (which model an adaptive sensor) are controlled whereas the transition probabilities (which model the Markov signal being observed by the sensor) are not controlled. Controlled sensing arises in reconfigurable sensing resource allocation problems (how can a sensor reconfigure its behavior in real time), cognitive radio, adaptive radars and optimal search problems. For such problems, the value function arising from stochastic dynamic programming is convex but not known in closed form; nevertheless Theorem B applies. By using Theorem B, the following useful structural result will be established
Theorem C**.**
(Controlled Sensing POMDP) For a 2-state Markov chain , under suitable conditions on the observation distributions, the optimal controlled sensing policy is lower bounded by a myopic policy.
The motivation for Theorem C is two-fold. First, since in general solving a POMDP for the optimal policy is computationally intractable, there is substantial motivation to derive structural results that bound the optimal policy; see [6, 7, 8, 9, 10] for an extensive discussion of POMDP structural results and construction of myopic bounds. Second, the myopic bounds we propose are straightforward to compute and implement and can be used as an initialization for more sophisticated sub-optimal algorithms. Existing works [7, 10] in constructing myopic lower bounds to the optimal policy use Blackwell dominance of probability measures. Theorem C includes several classes of POMDPs where Blackwell dominance does not hold.
Limitations. Our results have two limitations. First, for continuous-state problems, we require a scalar state (). This is essential for convex dominance; multivariate convex dominance is an open area. Actually, for finite states (Hidden Markov Model localization and filter) this is not a limitation since a multivariate finite state is straightforwardly mapped to a scalar finite state. Second, while we show global convex dominance for localization (Theorem 3), for optimal filtering we can only show one-step convex dominance (Theorem B). Note however, Theorem B does hold for the two-time scale problem where the state remains fixed for multiple observations. We emphasize that for the POMDP controlled sensing application, neither of these are limitations, since stochastic dynamic programming relies only on the one-step filtering update. Also, despite these limitations, the sufficient conditions given cover numerous new examples where the only competing methodology (Blackwell dominance) does not hold. Finally, using the ingenious proof of [11], it is possible to give global convex dominance results for the optimal filter; but the corresponding sufficient conditions involve strong conditions and are complicated to check (albeit still finite dimensional); see Sec.III-D for a discussion.
Related Works. As mentioned above, Integral Precision dominance which refers to convex dominance of conditional expectations, has been studied in [1]. The single crossing condition proposed in [12] is a sufficient condition for integral precision dominance (for continuous-valued random variables observed in noise). Our main result, namely Theorem 1, generalizes this to hold for an arbitrary sequence of observations - this requires generalizing the single crossing condition of the observation probabilities in [12] to aggregating the single crossing condition [3] and dealing with boundary conditions when the observation distribution has finite support. For a textbook treatment of convex dominance and stochastic orders in general, see [13, 14].
Regarding controlled sensing POMDPs, [4, 7, 5, 10] used convexity of the value function together with Blackwell dominance to construct a myopic lower bound. [15] considers controlled sensing with hypothesis testing.
As mentioned earlier, Blackwell dominance [16, 4, 5] requires factorization of probability measures; and does not, in general, hold globally for all ; due to lack of commutativity of matrix multiplication. We refer the reader to [17, 18] for an excellent recent discussion on Blackwell dominance in an information theoretic setting. Finally, there are other approaches for quantifying the MSE in estimation; [19] uses an interesting approach involving finite time anticipative rate distortion.
Organization. Sec.II formulates the localization and filtering models, key assumptions, and main theorem (Theorem 1) on convex dominance of the conditional mean. Sec.III discusses important examples where Theorem 1 applies including discrete memoryless channels, additive noise with log-concave density and power law density. Sec.IV shows how local convex dominance of the optimal filter can be used to construct a myopic lower bound for the optimal policy of a controlled sensing POMDP.
II Convex Dominance for Bayesian Localization and Filtering
In this section we formulate the Bayesian estimation (localization and filtering problems), and then present our main result on convex dominance of the conditional mean estimate, namely, Theorem 1. The various assumptions required for Theorem 1 to hold are then discussed. Regarding notation, we use uppercase for random variables and lower case for realizations. The superscript ′ denotes transpose.
II-A Bayesian Localization and Filtering Models
For notational simplicity, we first formulate the filtering problem with finite underlying state space . Then we formulate the continuous state filtering with state space on . In either case, choosing the transition probability (density) as identity (Dirac mass) for the underlying Markov process results in the Bayesian localization problem.
Model 1. Finite State Estimation. Consider a discrete time Markov chain with finite state space , initial probability vector and transition matrix , . The Markov chain is observed in noise by sensor . We consider two sensors which generate the corresponding observation process , . Here lies in observation space and has conditional distribution , i.e., is conditionally independent of , . We consider three types of observation spaces : either is a finite set of action dependent alphabets, , ; or ; or , i.e., finite support for . Let denote the unit simplex of -dimensional probability vectors.
Definition 1.A** (Finite State Filtering and Localization).**
Assume are known. Given an observation sequence from sensor , the aim of filtering is to estimate the Markov state , , by computing the posterior probability mass function recursively over time . Localization refers to the special case with transition matrix (identity matrix), and the aim is to estimate the random variable by computing the posterior recursively over time .
The solution to the filtering problem is as follows: Starting with initial distribution , the posterior using sensor is computed recursively using the classical hidden Markov model (HMM) state filter as
[TABLE]
Here represents a -dimensional vector of ones. When the observation space of sensor is a finite set, , denotes the observation probabilities for sensor . When is continuum, we assume that the conditional distribution is absolutely continuous wrt the Lebesgue measure and so the controlled conditional probability density function exists. We assume for each , for at least one state ; otherwise and are not well defined.
The notation in (3) specifies the filtering/localization update for a single observation . Given a sequence of observations and prior , we denote the resulting computation of the posterior as with normalization term . Let denote the physical state levels associated with the states , respectively. Then, for sensor , the conditional mean estimate of the state is defined as the measurable random variable
[TABLE]
Finally, for sensors , the mean square error (MSE) of the conditional mean given prior is
[TABLE]
where denotes the -dimensional integral over .
Given the complicated nature of (4) and (5), evaluating the MSE analytically for all priors is impossible, even when the observation space is finite. The MSE is computed by Monte-Carlo simulation by averaging over a large number of sample paths . Our main result below gives an analytical characterization for any convex function: given two sensors , with observation processes , , where observation and respectively, we give sufficient conditions so that for all priors .
Model 2. Continuous State Estimation: Here we assume a continuous state Markov process with space , initial distribution , and transition distribution for any Borel set . We assume absolute continuity wrt Lebesgue measure so that the initial density and transition density exists. The Markov process is observed by noise sensor . For each sensor , we assume the observation space is . The observations are generated with conditional cdf with support on . We assume is absolutely continuous wrt the Lebesgue measure and so the controlled conditional pdf exists.
Definition 1.B** (Continuous State Filtering and Localization).**
Assume are known. Identical to Definition 1.A except that posterior is now a probability density function. In the localization problem, the transition density is a Dirac mass.
The solution of the filtering problem is as follows: Starting with initial density , the posterior state density for sensor is computed recursively using the optimal filter (Bayesian recursion)
[TABLE]
The conditional mean estimate of the state and associated MSE for sensor are given by
[TABLE]
Apart from the case where the densities , and are Gaussian444In the Gaussian case, posterior is Gaussian and its mean and variance are computed via the Kalman filter., in (6) does not have a finite dimensional statistic and can only be computed approximately (using, for example, sequential Markov-chain Monte-Carlo methods). It is impossible to evaluate the MSE analytically over the continuum of priors ; thus there is strong motivation to give sufficient conditions that yield convex dominance and therefore an ordering of the MSE between two sensor models and .
Remark. Two time scale filtering: In Sec.I we discussed a two time scale system where the state process evolved on a slow time scale and observations are recorded on a fast time scale. That is, at each time corresponding to state , we obtain fast time scale observations represented by the vector for some integer where each component is conditionally independent of . Then the filtering recursions (3) and (6) apply with .
II-B Assumptions and Main Result
We are now ready to state our main results. The key condition we will use is that of single crossing.
Definition 2** (Single Crossing [20]).**
A function is single crossing, denoted as in , if
[TABLE]
In words, crosses zero at most once from negative to positive as increases. (Note that in our case is a totally ordered set; actually the single crossing definition applies more generally to partially ordered sets.)
II-B1 Assumptions
The following are our main assumptions; recall is the conditional observation pdf, is the conditional observation cdf and is the complementary conditional cdf for sensor :
- (A1)
[TP2 observation probabilities] The observation probability kernel (matrix) is totally positive of order 2 (TP2). 555That is, where the monotone likelihood ratio (MLR) order is defined in Appendix A. Equivalently, for finite, the -th row of is MLR dominated by the -th row, i.e., the rows of the matrix are totally monotone wrt the MLR order. When is finite, TP2 is equivalent to all second-order minors of matrix being nonnegative. 2. (A2)
[Single Crossing Condition] For any , , in . Equivalently, in terms of complementary cdfs, . 3. (A3)
[Boundary conditions] If , , then for the boundary values and :
[TABLE]
If then the above equation holds with and replaced by and . (A3) is not required if . 4. (A4)
[Signed Ratio Monotonicity] If and then for all and ,
[TABLE]
for .
If and then for all and ,
[TABLE]
for .
The assumptions are discussed below in Sec.II-C. However, we note at this stage that (A4) is equivalent to the following single crossing condition (proof in Theorem 14 in the appendix): for any ,
[TABLE]
The main point is that (9) globalizes (A2), namely , to a product from time 1 to arbitrary time . (A4) is a tractable condition for (9) in terms of the model parameters (observation probabilities); see discussion below.
II-B2 Main result
Our main result involves convex dominance of the conditional mean. Let us define this formally.
Definition 3** (Convex dominance of conditional mean).**
Consider two sensor models with observation process and generated by cdfs and , respectively. Let denote the -dimensional integral over . Consider the Bayesian localization/filtering problem of Definition 1.B.
Global convex stochastic dominance of the conditional mean estimates (4) or (7) denoted as holds if for all time , \mathbb{E}_{1}\{\phi\big{(}m_{1}(Y^{(1)}_{1:k},\pi_{0})\big{)}\}\leq\mathbb{E}_{2}\{\phi\big{(}m_{2}(Y^{(2)}_{1:k},\pi_{0})\big{)}\} for any666Providing the integral exists.* convex function and prior . Equivalently, for all time ,*
[TABLE] 2. 2.
*Local (one step) convex dominance of the conditional mean estimates (4) or (7) denoted as holds at each time if \mathbb{E}_{1}\{\phi\big{(}m_{1}(Y^{(1)}_{k},\pi_{k-1})\big{)}\}\leq\mathbb{E}_{2}\{\phi\big{(}m_{2}(Y^{(2)}_{k},\pi_{k-1})\big{)}\} for any convex function and prior . Equivalently, at each time , *
[TABLE]
We are now ready to state our main results for Bayesian localization and filtering.
Theorem 1** (Global Convex Dominance for Bayesian Localization).**
Consider the Bayesian localization problem of Definition 1.B:
For the finite state model (3), under (A1), (A2), (A3), (A4) (or (9)), global convex stochastic dominance of the conditional mean estimates (4) holds for all time , i.e., . 2. 2.
For the continuous state model (6), under (A1), (A2), (A4) (or (9)), global convex stochastic dominance of the conditional mean estimates (7) holds for all time .
Therefore, in both cases,
[TABLE]
holds globally for all time .
The proof of Theorem 1 is in Appendix A.
Corollary 2** (Local Convex Dominance for Optimal Filtering).**
Consider the optimal filtering problem of Definition 1.B:
For the finite state model under (A1), (A2), (A3), local convex dominance of the conditional mean estimates (4) of the Hidden Markov Model (HMM) filter (3) holds at each time , i.e., . 2. 2.
For the continuous state model under (A1), (A2), local convex dominance of the conditional mean estimates (7) for the optimal filter (6) holds at each time .
Therefore, for both cases, holds at each time .
Corollary 3** (Two time-scale filtering).**
For the two-time scale filtering problem discussed in Sec.II-A,
For the HMM filter, local convex dominance (11) holds under (A1), (A2), (A3), (A4) 2. 2.
For the continuous state filter, local convex dominance (11) holds under (A1), (A2), (A3), (A4).
In either case, in (11) denotes the -dimensional integral.
Proof.
The one step filtering update (3) is identical to localization with replaced by . Since Theorem 1 holds for all , Corollary 2 and 3 follow. ∎
Let us reiterate the main point: It is clear from (4), (5) that evaluating the MSE analytically for all priors is impossible, even when the observation space is finite. Theorem 1 and its corollaries are useful since they give sufficient conditions that ensure one sensor observation model yields a MSE that dominates another sensor observation model; indeed they guarantee dominance for any convex function. Also for continuous state optimal filtering, in general there is no finite dimensional statistic for thereby making it impossible to compute exactly; yet Corollary 2 and 3 give useful insight into how the observation probabilities affect the mean square error of the conditional mean.
II-B3 Why can’t we establish global convex dominance of the optimal filter?
The above results establish global convex dominance for Bayesian localization and local convex dominance for optimal filtering. The key step in the proof of global convex dominance is (20) in the appendix: in simpler notation the task is to prove that for where is the vector of state levels of the Markov chain, is a square matrix, and is the prior. In the localization problem, is a diagonal matrix involving the observation distributions. Because of this diagonal structure, useful sufficient conditions can be given in terms of the model parameters , . In the filtering case is no longer a diagonal matrix - it is the non-commutative product of transition matrices and observation matrices. Then there is no obvious way of giving useful sufficient conditions for in terms of the model parameters.
In Sec.III-D we will give an alternative set of sufficient conditions for global convex dominance that apply to the optimal filter when the observation spaces , are finite. However, checking these sufficient conditions for length observation sequences requires a computational cost that is exponential in and so intractable for large . Nevertheless, the sufficient conditions of Sec.III-D guarantee global convex dominance for all (continuum of) priors and so are useful for small .
II-C Discussion of Assumptions (A1)-(A4)
This subsection discusses the main assumptions of Theorem 1. Section III below discusses several examples.
(A1). The TP2 condition (A1) is widely used to characterize the structural properties of Bayesian estimation. (A1) is necessary and sufficient for the Bayesian filter update to be monotone likelihood ratio increasing wrt ; see [10] for proof. This implies is increasing in . This monotonicity wrt is a crucial step in proving Theorem 1. [10] gives several examples of continuous and discrete distributions that satisfy (A1) in the context of controlled sensing. We refer to the classic work [21] for details and examples of TP2 dominance, see also [10].
(A2). (A2) is the key condition required for integral precision dominance. First a few words about integral precision dominance. For random variable with prior and posterior , Definition 2(ii), pp.1011 in [1] says that integral precision dominance holds if the conditional expectations exhibit convex dominance:
[TABLE]
Equivalently
[TABLE]
for any convex function , providing the integrals exist. For , [12] gives a single crossing condition similar to (A2) for integral precision dominance; see also footnote 9, pp.1016 in [1]. Our setting is different since we consider a Markov process observed in noise and we are considering convex dominance wrt the process . However, our main proof is similar in spirit to [12], but in addition to (A2), we also need the boundary condition (A3) for finite support and finite set observations; also we need (A4) for global convex dominance. Finally, note that [22] examines integral precision dominance as a special case of Lehmann precision (see Corollary 4.6 of [22]) after the seminal paper by [23].
Returning to the single crossing condition (A2), it can also be viewed as signed-submodularity of the observation probability distributions. A function is submodular if is increasing in . In comparison, (A2) says777For , define the signum function for , respectively. Note that increasing in (ignoring excursions to zero) is equivalent to in Definition 2. \operatorname{sgn}\big{(}\Delta(x,u)\big{)} is increasing in where . Requiring to be increasing in is impossibly restrictive, whereas requiring \operatorname{sgn}\big{(}\Delta(x,u)\big{)} to be increasing in leads to numerous examples as discussed below. We will use this signed-submodularity assumption in the FKG inequality (Theorem 12) to prove integral precision dominance.
(A3). The boundary condition (A3) is not required if the observation space for . (A3) is only required when has finite support or is finite. (A3) is not restrictive since it only imposes conditions on the observation probabilities at the boundary values of . (A3) is a sufficient condition for the range of the posterior for sensor to be a subset of that for sensor , i.e., , . Several examples that satisfy (A3) are given below. Also to give further insight, the end of Appendix A-B gives numerical examples where integral precision dominance does not hold when (A3) is not satisfied.
(A4). Signed ratio monotonicity (A4) is a key condition from the paper [3, Proposition 1]; it is a necessary and sufficient for any non-negative linear combination of single crossing functions to be single crossing. Translated to our problem, (A4) is required for establishing Theorem 1 for (global convex dominance), i.e., when multiple observations are used to compute the posterior. (A4) is not required for the case (local convex dominance). In simple terms (A4) extends the single crossing condition (A2) to the sum of single crossing functions. Note that (A2) involves each individual sensor , whereas (A4) involves both sensors’ observation probabilities.
To motivate (A4), start with (9). The ordinal property of single crossing [20] implies that (9) is equivalent to the difference in logs being single crossing, i.e., . Note (A2) implies that each term ; but this does not imply that the sum over is single crossing. (In general the sum of single crossing functions is not single crossing.) The main point is that signed ratio monotonicity condition (A4) is necessary and sufficient for any non-negative linear combination of single crossing functions to be single crossing [3, Proposition 1]. This allows us to express (9) as the tractable condition (A4) which directly involves the observation density. Finally, in the special case of additive log-concave noise densities, (A4) automatically holds if (A2) holds; this is discussed below in Sec.III-B.
Another intuitive way of viewing (9) is: a sufficient condition for local convex dominance is that is increasing in (this is stronger than (A2) which only needs to increase in ); a sufficient condition for global convex dominance requires that is increasing in (this is stronger than (9)).
III Examples of Convex Dominance in Localization and Filtering
To illustrate Theorem 1 and its corollaries, we discus 3 important examples of convex dominance in Bayesian estimation. Then we briefly discuss conditions for global convex dominance of the optimal filter.
III-A Example 1. Blackwell Dominance, Integral Precision Dominance and Channel Capacity
Here we discuss our first main example; namely how Theorem 1 and its corollaries apply to finite set observation models and HMMs. As mentioned in Section I, Blackwell dominance is a widely used condition for convex dominance. Since Theorem 1 uses integral precision dominance to give a new set of conditions for convex dominance compared to Blackwell dominance, we compare them using several numerical examples below.
Definition 4** (Blackwell dominance ).**
Suppose for where is a stochastic kernel, i.e., and . Then Blackwell dominates ; denoted as . So when are finite, if where is a stochastic (not necessarily square) matrix.
Intuitively is noisier than . Using a straightforward Jensen’s inequality argument, the following result holds:
Theorem 4** (Blackwell dominance [7]).**
* is a sufficient condition for the one step (local) stochastic dominance conclusion of Theorem 1 to hold.*
Insight. Both integral precision dominance (Theorem 1) and Blackwell dominance (Theorem 4) exploit convexity. But there is an important difference: Blackwell dominance implies that for any convex function , \sum_{\mathbb{Y}_{u}}\phi\big{(}T(\pi,y,u)\big{)}\sigma(\pi,y,u) is increasing in for all . In comparison, integral precision dominance (Theorem 1) implies convex dominance in one dimension, namely, for any scalar convex function , \sum_{\mathbb{Y}_{u}}\phi\big{(}g^{\prime}T(\pi,y,u)\big{)}\sigma(\pi,y,u) is increasing in for all . As will be shown below there any many important examples where integral precision dominance holds but Blackwell dominance does not hold.
Note that Blackwell dominance (Theorem 4) does not hold globally for all unlike integral precision (Theorem 1). This is because does not imply that the -th powers satisfy , apart from the pathological case where matrix multiplication commutes (i.e., the pathological case when and are simultaneously diagonalizable). Thus global convex dominance in Theorem 1 is a useful and substantial generalization.888Le Cam deficiency is a useful way of finding the closest Blackwell dominant matrix to given ; it also yields the loss (deficiency) in choosing this closest matrix, see [17] for a nice discussion. However, this loss is impossible to compute for an arbitrary convex function such as the value function of a controlled sensing POMDP which is apriori unknown and intractable to compute.
Examples
Example (i): Here are examples of observation matrices that satisfy assumptions (A1), (A2), (A3), (A4) implying that integral precision dominance and global convex dominance in Theorem 1 holds. But Blackwell dominance does not hold.
[TABLE]
Note the third example has different observation spaces for the two actions. Interestingly, in all three examples above, does not Blackwell dominate ; i.e., for stochastic matrix .
Example (ii). A consequence of [24] is that for symmetric matrices , if , then Blackwell dominance is equivalent to integral precision dominance (A2). Then (A3) automatically holds. This is easy to show, see [1]: since is a valid stochastic matrix as can be verified by explicit symbolic computation.
Example (iii). Channel Capacity. Shannon [25] establishes the following result in terms of channel capacity; see [26] for a detailed exposition.
Theorem 5** ([25]).**
If where and are stochastic matrices, then discrete memoryless channel has a smaller Shannon capacity (conveys less information) than .
Blackwell dominance is a special case of Theorem 5 when . However, if the multiplication order is reversed, i.e., suppose where is a stochastic matrix, then even though is still more “noisy” (conveys less information according to Theorem 5) than , Blackwell dominance does not hold.
Motivated by Theorem 5, a natural question is: Does integral precision dominance and hence Theorem 1 hold for examples where where is a stochastic matrix? As an example consider
[TABLE]
Then there exists a stochastic matrix such that but Blackwell dominance does not hold since for stochastic matrix . But (A1), single crossing condition (A2), boundary condition (A3), and signed ratio monotonicity (A4) hold for this example; therefore Theorem 1 holds.
Further examples involving hierarchical sensing and word-of-mouth social learning are discussed in Section IV.
Summary: This subsection discussed several examples where integral precision dominance and global convex dominance of the conditional mean holds but Blackwell dominance does not hold. The two specific cases we discussed are:
where and are stochastic matrices, 2. 2.
Blackwell dominance does not imply global Blackwell dominance . In comparison, Theorem 1 gives conditions for which global convex dominance holds.
III-B Example 2. Sensing in Additive Noise with Log-concave density
We now discuss how Theorem 1 and its corollaries apply to sensing in additive noise, where the additive noise has a log-concave density. The main point is that for additive noise with log-concave density, higher differential entropy or variance of the additive noise is a necessary condition for the MSE of the Bayesian localization and filtered estimate to be higher. (Sec.III-C below shows that if the noise does not have a log-concave density, then higher differential entropy or variance is not a necessary condition).
In the additive noise setting, the sensor observation models are , . The additive noise is independent and identically distributed with a log-concave pdf . Recall [27] that a log-concave density has the form where is a concave function of . There are numerous examples of log-concave densities: normal exponential, uniform, Gamma (with shape parameter ), Laplace, logistic, Chi, Chi-squared, etc.
We assume for that the density has either support on (then (A3) is not required) in which case ; or has support on in which case (then (A3) holds straightforwardly; e.g. if , then in (A3) and both sides of the first inequality in (A3) are zero.)
The following result characterizes the assumptions of Theorem 1 for additive noise models with a log-concave density.
Theorem 6**.**
Consider the additive noise sensing model , where the additive noise is independent and identically distributed with pdf and cdf . Then:
(A1)* holds iff and are log-concave densities.* 2. 2.
(A2)* holds iff holds where denotes the dispersive stochastic order.999Cdf dominates cdf wrt dispersive order, denoted , if for .* 3. 3.
(9) or equivalently (A4) holds if and are log-concave densities and , i.e., (A2) holds. 4. 4.
* having smaller differential entropy than is a necessary condition for (A2) to hold. Also having smaller variance than is a necessary condition for (A2) to hold.*
Therefore for log-concave additive noise and , if , then Theorem 1 and Corollaries 2, 3 hold.
Proof.
Statement 1 is proved in [14, Theorem 1.C.52 (iii)]. Statement 2 is proved in [12, Remark 3]. Statement 4 follows from [13, Theorems 1.5.42 and 1.7.8].
Statement 3: Since the pdfs are log-concave, their complementary cdfs and are log-concave; see [27, Theorem 2(i)]. Next from [14, Theorem B 20. pp156], and the complementary cdfs being log-concave implies that hazard rate dominance holds, i.e., is decreasing in . This implies is increasing in for all and . Therefore, is increasing in which in turn implies that is increasing . Therefore is increasing in which implies . Finally, implies that for any monotone function101010This is the well known ordinal property of single crossing [20]. . Thus (9) holds. ∎
Theorem 6 gives a complete characterization of global convex dominance in additive noise models. It confirms the intuition that additive noise with higher differential entropy (or variance) results in larger MSE for Bayesian localization and optimal filtering. More precisely, higher differential entropy (or variance) is a necessary condition for (A2); indeed (A2) (dispersion dominance) is a stronger condition than dominance of differential entropy.
Examples of log-concave densities that satisfy (A1), dispersive dominance (A2) and therefore (A4) include:
Normal cdf: with , . 2. 2.
Exponential cdf: , with rate parameter , . 3. 3.
Gamma distribution [28]: , with shape parameter .
For these examples Theorem 1 and Corollaries 2, 3 hold. Also for these examples, (A2) is equivalent to having smaller differential entropy (or variance) than ; that is Statement 4 of Theorem 6 is necessary and sufficient.
III-C Example 3. Additive Sensing. Power Law vs Exponential Noise in Social Networks
Motivated by sampling social networks, we now discuss an example where instead of the TP2 condition (A1), a reverse TP2 condition holds (due to log convex density additive noise). The main point below is that regardless of whether the power law noise has a smaller variance than exponential noise, the MSE is always larger due to convex dominance.
Suppose we wish to compare the MSE of the conditional mean estimates when the additive noise is a log convex density that decays according to a power law while is an exponential density (log-concave). That is:
[TABLE]
For example, the empirical degree distribution (number of neighbors of per node normalized by the total number of nodes) of several social media networks such as Twitter [29] have a power law with exponent ; while social health networks in epidemiology have an exponential degree distribution. Based on observations obtained by sampling individuals in the network and asking each such individual how many friends it has (degree), a natural question is: how accurate is the Bayesian conditional mean estimate for the average degree of the network?
Theorem 7**.**
Consider the additive noise model , where the additive noise is independent and identically distributed with pdf . Then the conclusions of Theorem 1 hold for the following cases:
Power law density and exponential density 2. 2.
Power law densities and with power law coefficients .
Theorem 7(1) is interesting because it asserts convex dominance between two different types of noise densities. It says that the conditional mean estimate in additive exponential noise is always more accurate than that in power law noise. Interestingly, the variance for a power law density can be smaller than that of an exponential density; for power law exponent , the variance is 17.35 which is smaller than the variance of an exponential for ; yet the MSE of the conditional mean is larger in power law noise. (Note for , the power law variance is not finite). Theorem 7(2) is intuitive; a larger power law implies the density decays faster to zero; and therefore the MSE is smaller.
Proof.
Statement (1): (A1) holds for the observation likelihood , but (A1) does not hold for . Instead satisfies a reverse TP2 ordering: , . Indeed, is increasing in for . Then using a similar proof to Theorem 1, global convex dominance holds if (recall SC is defined in (8)):
[TABLE]
A similar proof to Theorem 6 shows that the above condition holds because
[TABLE]
is increasing in for all and .
Statement (2): Since an are reverse TP2, the global convex dominance condition becomes . This holds because is increasing in for . ∎
III-D Single crossing in conditional mean and Global Convex Dominance of HMM filter
So far we used the single crossing of the conditional distributions (A2), (A4), to establish convex dominance. We conclude this section by discussing an alternative condition based on an ingenious result from [11]; it uses single crossing of the conditional mean to establish global convex dominance of the conditional mean; but the conditions are computationally expensive to verify.
Proposition 8** ([11, Proposition 2.1]).**
Suppose , is increasing in and in . Then convex dominance holds for the conditional means. (Recall SC is defined in (8)).
We now use Proposition 8 to establish global convex dominance for the HMM filter (3); but the sufficient conditions given below are expensive to check and only tractable for finite observation spaces and .
Note that with elements. Label the elements lexicographically and denote them as . For and each , define the matrices
[TABLE]
We introduce the following assumptions for global convex dominance:
- (A3)
The matrices are elementwise positive for all , , . 2. (A4)
The matrices are elementwise negative for and positive for , for all , , for some .
Theorem 9**.**
Under (A3) and (A4), global convex dominance holds for all priors for the HMM filter (3) with finite observation space , .
Proof.
(A3) is sufficient for for ; this can be verified from the definition of likelihood ratio dominance, namely
[TABLE]
This in turn is sufficient for the first condition of Proposition 8, namely, is increasing in .
Similarly it can be shown that (A4) is sufficient for for and for . This implies the second condition of Proposition 8 is satisfied, namely , and , . ∎
Example. It can be verified numerically that , satisfies (A3) and (A4) for .
Summary: In contrast to previous subsections, this subsection used the single crossing property of conditional means to propose sufficient conditions (A3) and (A4) for global convex dominance of the HMM filter. Verifying (A3) and (A4) involve checking negative/positive elements for matrices is computationally intractable for large . However, the conditions guarantee global convex dominance for all (continuum) of priors and are useful for small .
IV Example. Controlled Sensing Partially Observed Markov Decision Process (POMDP)
Thus far we have discussed convex dominance of the conditional mean (in filtering and localization) between two fixed sensors. This section considers a POMDP controlled sensing problem where we optimize the dynamic switching between multiple sensors. The main result of this section is an important application of Corollary 2 (local convex dominance for the HMM filter): we construct a myopic lower bound to the optimal policy of a 2-state (but arbitrary observation space ) controlled sensing POMDP. Thus far, the only known way of constructing such lower bounds involved Blackwell dominance [4, 7, 10]. The plethora of examples in Sec. III where integral precision dominance holds (but Blackwell dominance does not), demonstrates the usefulness of Theorem 1 in controlled sensing.
In controlled sensing, the aim is to dynamically decide which sensor (or sensing mode) to choose at each time to optimize the objective defined in (13) below. In general, POMDPs are computationally intractable to solve (PSPACE complete). Therefore, from a practical point of view, constructing a myopic lower bound is useful since myopic policies are trivial to compute/implement in large scale POMDPs and provide a useful initialization for more sophisticated sub-optimal solutions.
IV-A Controlled Sensing POMDP
We consider an infinite horizon discounted reward controlled sensing POMDP. It is customary to call the posterior as the “belief”. A discrete time two-state Markov chain evolves with transition matrix on the state space . So the belief space is a two-dimensional simplex, namely , . Denote the action space as . For each action denote the observation space as . We assume either , i.e., finite set of action dependent alphabets for all , or , or , i.e., finite support for all . For stationary policy , initial belief , discount factor , define the discounted cumulative reward:
[TABLE]
Here is the reward vector for each sensing action , and the belief state evolves according to hidden Markov model filter defined in (3) where , denotes the controlled observation probabilities.
The aim is to compute the optimal stationary policy such that for all . Obtaining the optimal stationary policy is equivalent to solving Bellman’s stochastic dynamic programming equation: , , where
[TABLE]
The value function is the fixed point of the following value iteration algorithm: Initialize for . Then for
[TABLE]
The sequence of value functions converges uniformly to on geometrically fast. Since is continuum, Bellman’s equation (14) and the value iteration algorithm (15) do not directly translate into practical solution methodologies since they need to be evaluated at each . Almost 50 years ago, [30] showed that when is finite, then for any , has a finite dimensional piecewise linear and convex characterization. Unfortunately, the number of piecewise linear segments can increase exponentially with the action space dimension and double exponentially with time . Thus there is strong motivation for structural results to construct useful myopic lower bounds for the optimal policy .
Remark 1. For controlled sensing POMDPs, the transition matrix , which characterizes the dynamics of the signal being sensed, does not depend on action . Only , which models the information acquisition reward of the sensor, and observation probabilities , which model the sensor’s accuracy when it operates in mode , are action dependent.
Remark 2. A POMDP with finite horizon has objective where and is the terminal reward vector. Then (15) initialized as for iterations yields the optimal policy sequence .
IV-B Main Result – Myopic Lower Bound
Theorem 10** (Controlled sensing POMDP).**
*Assume (A1), (A2), (A3) hold. Then is increasing111111By increasing, we mean non-decreasing. in . Therefore, the myopic policy forms a lower bound to the optimal policy in the sense that for all . Hence, for beliefs where , the optimal policy coincides with the myopic policy . An identical result holds in the finite horizon case for the policy sequence , . *
Proof.
The value function is convex in [10]. Since , is completely specified by where . So is convex. Assuming (A1), (A2), (A3), it follows from Theorem 1 that for all ,
[TABLE]
Equivalently, see (15), . Then Lemma 2 in [6] implies121212 Proof: If , then (16) implies for . This implies . So . If is not unique, the proof needs more care, see Lemma 2, [6].
for all . The same argument applies to and for the finite horizon case with terminal reward. ∎
From a practical point of view, Theorem 10 is useful since the myopic policy is trivial to compute and implement and gives a guaranteed lower bound to the optimal policy of the POMDP which is intractable to compute.
The main point is that Theorem 10 provides an alternative to Blackwell dominance for POMDPs which has been widely studied since the 1980s and also has the same conclusion:
Theorem 11** (Blackwell dominance for Controlled Sensing. [4, 7]).**
, is a sufficient condition for the conclusion of Theorem 10 to hold.
Blackwell dominance holds for any number of states . In comparison Theorem 10 applies only to POMDPs with 2 underlying states. However, there are numerous 2 state examples where Theorem 10 applies and Blackwell dominance does not.
IV-C Examples
-
Theorem 10 applies to all the 2-state examples in Sec.III where (A1), (A2), (A3) hold. As discussed in Sec.III-A, there are many examples where Blackwell dominance does not hold, but integral precision dominance (A2) does hold.
-
In controlled radar sensing problems [31], observations are obtained at a faster time scale than the state evolution. That is, for state (e.g., threat level at time ), an observation vector is obtained where and are conditionally independent given . In such cases, under (A4), convex dominance holds, and then Theorem 10 holds. However, Blackwell dominance (Theorem 11) does not hold for this case.
-
Optimal filter vs predictor scheduling is an important application of controlled sensing. Filtering uses a sensor with observation matrix to obtain measurements of the Markov chain and incurs a measurement cost but a performance reward. Prediction (no measurement) has non-informative observation matrix and incurs no measurement cost but yields a low performance reward. Clearly . If satisfies (A1), then (A2) holds automatically because is constant wrt ( is non-informative), while (A1) implies is increasing wrt .
-
Controlled Hierarchical Sensing: In controlled sensing involving hierarchical sensors (such as hierarchical social networks), level of the network receives signal distorted by the confusion matrix (-th power of stochastic matrix ), where . That is, each level of the network observes a noisy version of the previous level. Observing (polling) level of the network has observation probabilities conditional on the noisy message at level . Therefore the conditional probabilities of the observation given the state are where is the degree of separation from the underlying source (state). This is illustrated in Figure 1 for . The controlled sensing POMDP is to choose which level to poll at each time in order to optimize an infinite horizon discounted reward. By Theorem 5, is more noisy (has lower Shannon capacity) than ; yet Blackwell dominance does not hold due to the reverse multiplication order. But using integral precision dominance, Theorem 10 holds (under assumptions).
-
Word-of-Mouth Social Learning: Sensor 2 observes the Markov state in noise with observation probabilities . Sensor 1 receives the observations of sensor 1 in noise, but these probabilities also depend on the underlying state. Denote these state dependent probabilities as . Thus sensor 1 observation probabilities are
[TABLE]
Such models arise in multi-agent social learning where agents use observations/decisions of previous agents and also their own private observations of the state to estimate the underlying state [32, 8]. Sensor 1 is influenced by the word-of-mouth message from sensor 2 but interprets (critiques) this message based on its own observation of the state. The controlled sensing problem involves dynamically choosing between sensor 1 (direct measurement from source) versus sensor 2 (word of mouth measurement) to optimize the cumulative reward (13).
Even though from (17), appears more noisy than , Blackwell dominance does not necessarily hold. Also the Blackwell dominance proof of convex dominance breaks down due to the state dependent probabilities . However, integral precision dominance does hold in many cases. Here is one such example:
[TABLE]
It can be verified that (A1), (A2), (A3) and (A4) hold for this model, and therefore Theorem 1 and Theorem 10 hold.
V Discussion
This paper developed sufficient conditions for local and global convex dominance of the conditional mean in Bayesian estimation (localization and filtering). We used two techniques that have recently been developed in economics, namely, integral precision dominance (this yields local convex dominance) and aggregating the single crossing property (this yields global convex dominance). The convex dominance results apply to several examples where Blackwell dominance does not hold. As an application, we showed how convex dominance can be used to construct myopic lower bound to the optimal policy of a controlled sensing POMDP. The recent preprint [33] has interesting results on Blackwell dominance in large samples for two state random variables. In comparison the integral precision dominance used in the current paper yields global convex dominance for an arbitrary number of states.
Our main result was to give concise sufficient conditions for global convex dominance in Bayesian localization (and for local convex dominance in Bayesian filtering). In future work it is of interest to develop concise sufficient conditions for global convex dominance of Bayesian filtering; the conditions in Sec.III-D are difficult to verify. It is also worthwhile relating integral precision dominance (single crossing condition) to channel capacity. We know that Blackwell dominance implies that has higher capacity that (Theorem 5). Since both Blackwell dominance and integral precision dominance imply convex stochastic dominance, giving sufficient conditions on integral precision dominance to relate to channel capacity provides useful links between the MSE of optimal filters, myopic policies of POMDPs and information theory.
Finally, this paper considered the effect of sensing (observation kernels) on convex dominance and MSE when the transition kernels are identical. If the transition kernels are different for the two observation processes, then the MSE of the conditional means are meaningless since the state processes are different. However, one can still establish local convex dominance of the optimal filter by introducing suitable conditions on the transition kernel.
Appendix A Proof of Theorem 1
Definition 5**.**
*Let denote two univariate pdfs (or pmfs). Then dominates with respect to the monotone likelihood ratio (MLR) order, denoted as , if for .
dominates with respect to first order dominance, denoted as if for all . A function is said to be MLR (resp. first order) increasing if (resp. ) implies .*
For finite state space , when , is a complete order and coincides with . For , and both , are partial orders since it is not always possible to order any two arbitrary beliefs .
Proceeding to the proof of Theorem 1, for notational convenience we present the proof for finite state space. The proof for the continuous-state space case is virtually identical and outlined in Sec.A-C. We assume that the state levels , associated with the state space , are ordered so that .
First note that the expectations of are identical for , because . Therefore Theorem 1.5.3 in [13] implies convex dominance is equivalent to increasing convex dominance. Next, by Theorem 1.5.7 in [13], increasing convex dominance holds iff for ,
[TABLE]
Here we use the notation . The remainder of the proof focuses on establishing (18).
Defining ,
[TABLE]
for some and which depend on . Here each diagonal matrix
[TABLE]
where is the complementary cdf. Equation (20) follows since under (A1), is MLR increasing in each element , . Therefore, the set
[TABLE]
for some dependent real numbers and hence (20) involves complementary cdfs.
A-A Proof of Theorem 1 when is finite and
Theorem 12** (Convex dominance for finite state localization).**
Assume (A1), (A2), (A4) and . Then the following global convex dominance holds for all : That is, for any convex function ,
[TABLE]
Proof.
Since , clearly from (19), . We establish (18) for by showing131313Since is continuously differentiable (Lemma 13) with , clearly if at its stationary points (minima), then for all . that at all stationary points such that . Defining for , respectively, (20) yields
[TABLE]
Let us next evaluate the stationary points of for .
Lemma 13**.**
* defined in (18) is continuously differentiable wrt with gradient*
[TABLE]
(Proof at the end of this subsection). ∎
Thus the stationary points of satisfy (using the notation , defined in (23))
[TABLE]
So to prove Theorem 12, it only remains to show that is non-negative at these stationary points. To establish this we use the Fortuin-Kasteleyn-Ginibre (FKG) inequality [34] on (23). In our framework the FKG inequality141414Proof: Since and are increasing vectors, therefore for all . This implies the expectation which immediately yields the inequality (26). reads: If , are generic increasing vectors and a generic probability mass function, then
[TABLE]
Clearly in (23):
is increasing since the elements of are increasing by assumption; 2. 2.
is increasing by Theorem 14 below; 3. 3.
is non-negative and thus proportional to a probability mass function.
Also from (25), . So, applying FKG inequality to (23) yields . Thus we have established (18) for . ∎
Proof of Lemma 13 Here we prove Lemma 13 that was used to evaluate the gradient of in the proof above. For , similar to (21) define . Start with (18), and use the so called “integrated survival function” on page 19, [13], namely, integration by parts yields . Therefore \psi(\lambda)=\int_{\lambda}^{\infty}\mathbf{1}^{\prime}\bigl{[}\int_{{\mathbb{Y}}^{k,t}_{2}}B_{y_{k}}(2)\cdots B_{y_{1}}(2)\,dy_{1:k}-\int_{{\mathbb{Y}}^{k,t}_{1}}B_{y_{k}}(1)\cdots B_{y_{1}}(1)\,dy_{1:k}\bigr{]}\,\pi\,dt. Then evaluating yields (24). Finally, (24) implies is continuously differentiable because is continuous wrt (since is absolutely continuous wrt Lebesgue measure by assumption.) ∎
Theorem 14**.**
[TABLE]
in (23) is increasing in . (This property was used to prove Theorem 12).
Proof.
Showing that is increasing in is equivalent to showing that is a single crossing function in . By the ordinal property of single crossing [20], this in turn is equivalent to showing that is a single crossing function in or equivalently, is single crossing.
(A2) implies that each term is single crossing. So proving boils down to showing that the sum of single crossing functions is single crossing. The paper [3] shows that the signed monotonicity ratio (A4) is a necessary and sufficient condition for this to hold. ∎
A-B *Proof of Theorem 1 when is finite and
has finite support or is finite*
The following result is required for establishing our main result when is either finite or has finite support; see Case 1 and Case 2 of proof of Theorem 12 below. It is here that (A3) is the crucial assumption.
Theorem 15** (Finite support observation distributions).**
Suppose , . Assume (A1), (A3). Then
[TABLE]
Thus, defining and , it follows that and are impossible,
Proof.
Since is MLR increasing wrt under (A1), it suffices to show that
[TABLE]
The first inequality in (27) is equivalent to \sum_{i=1}^{\mathcal{X}}\sum_{j=1}^{\mathcal{X}}g_{i}\big{(}B_{i1}(2)B_{j1}(1)-B_{i1}(1)B_{j1}(2)\big{)}\pi(i)\pi(j)\leq 0. So (A3) is a sufficient condition for the inequality to hold. A similar proof holds for the second inequality in (27). ∎
Case 1. : Next we prove (18) for the finite support case where is the interval . The key difference compared to the case is due to the possible discontinuity of the conditional probability densities at the end points and . Without appropriate assumptions, defined in (18) can become negative in two ways: (i) If and is non-empty (ii) and is non-empty. Assumption (A3), see Theorem 15, ensures that these two cases do not occur.
To prove for , boundary conditions need to be handled. Define as
[TABLE]
Clearly, since is increasing in by (A1) and for . We now consider split into the following 5 sub-cases and show that for each sub-case:
[TABLE]
Note that (19) implies for Case 1a and Case 1e. For Case 1b, re-expressing , (20) implies that . Equivalently, (24) implies ; since , therefore for . For Case 1d, it follows immediately from (18) that . Equivalently, (24) implies ; since , therefore for .
Finally, for Case 1c, since both and are non-empty, the single crossing condition (A2) kicks in and an identical argument as the case applies. Indeed, , , and is differentiable for ; so for because at each stationary point .
Remark: The case (Theorem 12) can be viewed as a special instance of (29) with , and (but to enhance clarity we described it before Case 1). The main point when is that are never empty for and therefore only Case 1c occurs.
Case 2. is finite: Finally, we prove (18) for the case . Construct the piecewise constant probability density function for and . It is easily verified that , , and the value function and optimal policy remain unchanged. Then the above proof for Case 1 (finite support) applies. ∎
Remark. To emphasize the importance of sufficient condition (A3), the following examples show that (A3) is in some sense necessary; when it fails to hold, then for some interval of and convex dominance does not hold. Consider , , .
Example 1. ,
, .
Then for . This example violates Case 1b.
Example 2. ,
, .
Then for . This example violates Case 1d.
A-C Proof of Theorem 1 when and
In complete analogy to (18) convex dominance holds if for ,
[TABLE]
where
[TABLE]
Defining ,
[TABLE]
for some and which depend on .
In complete analogy to Lemma 13, for and and is continuously differentiable wrt with gradient
[TABLE]
The remainder of the proof is similar to that of Theorem 12.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] J.-J. Ganuza and J. S. Penalva, “Signal orderings based on dispersion and the supply of private information in auctions,” Econometrica , vol. 78, no. 3, pp. 1007–1030, 2010.
- 2[2] S. Athey and J. Levin, “The value of information in monotone decision problems,” Research in Economics , vol. 72, pp. 101–116, 2018.
- 3[3] J. Quah and B. Strulovici, “Aggregating the single crossing property,” Econometrica , vol. 80, no. 5, pp. 2333–2348, 2012.
- 4[4] C. C. White and D. P. Harrington, “Application of Jensen’s inequality to adaptive suboptimal design,” Journal of Optimization Theory and Applications , vol. 32, no. 1, pp. 89–99, 1980.
- 5[5] U. Rieder and R. Zagst, “Monotonicity and bounds for convex stochastic control models,” Mathematical Methods of Operations Research , vol. 39, no. 2, pp. 187–207, June 1994.
- 6[6] W. S. Lovejoy, “Some monotonicity results for partially observed Markov decision processes,” Operations Research , vol. 35, no. 5, pp. 736–743, Sept.-Oct. 1987.
- 7[7] U. Rieder, “Structural results for partially observed control models,” Methods and Models of Operations Research , vol. 35, no. 6, pp. 473–490, 1991.
- 8[8] V. Krishnamurthy, “Quickest detection POMD Ps with social learning: Interaction of local and global decision makers,” IEEE Transactions on Information Theory , vol. 58, no. 8, pp. 5563–5587, 2012.