Informational Substitutes
Yiling Chen, Bo Waggoner
TL;DR
This paper introduces formal definitions of informational substitutes and complements, exploring their implications in game theory, prediction markets, crowdsourcing, and optimization, and analyzing their structural properties and design considerations.
Contribution
It provides a unified framework for understanding informational substitutes and complements across game-theoretic and algorithmic contexts, with new definitions and structural insights.
Findings
Substitutes capture diminishing marginal value of information.
Substitutes characterize optimal equilibria in prediction markets.
Designing scoring rules can promote substitutability or complementarity.
Abstract
We propose definitions of substitutes and complements for pieces of information ("signals") in the context of a decision or optimization problem, with game-theoretic and algorithmic applications. In a game-theoretic context, substitutes capture diminishing marginal value of information to a rational decision maker. We use the definitions to address the question of how and when information is aggregated in prediction markets. Substitutes characterize "best-possible" equilibria with immediate information aggregation, while complements characterize "worst-possible", delayed aggregation. Game-theoretic applications also include settings such as crowdsourcing contests and Q\&A forums. In an algorithmic context, where substitutes capture diminishing marginal improvement of information to an optimization problem, substitutes imply efficient approximation algorithms for a very general class of…
Click any figure to enlarge with its caption.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAuction Theory and Applications · Experimental Behavioral Economics Studies · Game Theory and Applications
Informational Substitutes
Yiling Chen and Bo Waggoner
Harvard, University of Pennsylvania
[email protected], [email protected]
(Updated: March 2017)
Abstract
We propose definitions of substitutes and complements for pieces of information (“signals”) in the context of a decision or optimization problem, with game-theoretic and algorithmic applications. In a game-theoretic context, substitutes capture diminishing marginal value of information to a rational decision maker. We use the definitions to address the question of how and when information is aggregated in prediction markets. Substitutes characterize “best-possible” equilibria with immediate information aggregation, while complements characterize “worst-possible”, delayed aggregation. Game-theoretic applications also include settings such as crowdsourcing contests and Q&A forums. In an algorithmic context, where substitutes capture diminishing marginal improvement of information to an optimization problem, substitutes imply efficient approximation algorithms for a very general class of (adaptive) information acquisition problems.
In tandem with these broad applications, we examine the structure and design of informational substitutes and complements. They have equivalent, intuitive definitions from disparate perspectives: submodularity, geometry, and information theory. We also consider the design of scoring rules or optimization problems so as to encourage substitutability or complementarity, with positive and negative results. Taken as a whole, the results give some evidence that, in parallel with substitutable items, informational substitutes play a natural conceptual and formal role in game theory and algorithms.
Contents
-
B Defining S&C: intuition, challenges, and historical context
-
B.0.3 Our approach: dependence on context and internal structure
1 Introduction and Overview
1.1 Motivation and challenge
An agent living in an uncertain world wishes to make some decision (whether to bring an umbrella on her commute, how to design her company’s website, …). She can improve her expected utility by obtaining pieces of information about the world prior to acting (a weatherman’s forecast or a barometer reading, market research or automated A/B testing, …). This naturally leads her to assign value to different pieces of information and combinations thereof. The value of information arises as the expected improvement it imparts to her optimization problem.
We would like to generally understand, predict, or design algorithms to guide such agents in acquiring and using information. Consider the analogous case where the agent has value for items or goods, represented by a valuation function over subsets of items. A set of items are substitutes if, intuitively, each’s value decreases given more of the others; they are complements if it increases. Here, we have rich game-theoretic and algorithmic theories leveraging the structure of substitutes and complements (S&C). For instance, in many settings, foundational work shows that substitutability captures positive results for existence of market equilibria, while complements capture negative results (Kelso Jr and Crawford, 1982; Roth, 1984; Gul and Stacchetti, 1999; Hatfield and Milgrom, 2005; Ostrovsky, 2008). When substitutes are captured by submodular valuation functions (Lehmann et al., 2001), algorithmic results show how to efficiently optimize (or approximately optimize) subject to constraints imposed by the environment (e.g. Calinescu et al. (2011)). For example, an agent wishing to select from a set of costly items with a budget constraint has a -approximation algorithm if her valuation function is submodular (Sviridenko, 2004).
Can we obtain similar structural and algorithmic results for information? Here, a piece of information is modeled as a signal or random variable that is correlated in some way with the state of the world that the agent cares about (whether it will rain, how profitable are different website designs, …). Intuitively, one might often expect information to satisfy substitutable or complementary structure. For instance, a barometer reading and an observation of whether the sky is cloudy both yield valuable information about whether it will rain to an umbrella-toting commuter; but these are substitutable observations for our commuter in that each is probably worth less once one has observed the other. On the other hand, the dew point and the temperature tend to be complementary observations for our commuter: Rain may be only somewhat correlated with dew point and only somewhat correlated with temperature, but is highly correlated with cases where temperature and dew point are close (i.e. the relative humidity is high).
Despite this appealing intuition, there are significant challenges to overcome in defining informational S&C. Pieces of information, unlike items, may have complex probabilistic structure and relationships. But on the other hand, this structure alone cannot capture the value of that information, which (again unlike items) seemingly must arise from the context in which it is used. Next, even given a measure of value, it is unclear how to formalize an intuition such as “diminishing marginal value”. Finally, it remains to demonstrate that the definitions are tractable and have game-theoretic and/or algorithmic applications. These challenges seem to have prevented a successful theory of informational S&C thus far.
1.2 This paper: summary and contributions
This paper has four components.
. We propose a definition of informational substitutes and complements (S&C). Beginning from the very general notion of value of information in the context of any specific decision or optimization problem, we define S&C in terms of diminishing (increasing) marginal value for that problem. This requires a definition of “marginal unit” of information. We consider a hierarchy of three kinds of marginal information: learning another signal, learning some deterministic function of another signal, and learning some randomized function (“garbling”) of another signal. These correspond respectively to weak, moderate, and strong versions of the definitions, formalized by strengthenings of submodularity and supermoduarity.
We also investigate some useful tools and equivalent definitons. From an information-theoretic perspective, substitutes can be defined as signals that reveal a diminishing amount of information about a particular event, where the measure of information is some generalized entropy function. From a geometric perspective, substitutes can be defined using a measure of distance, namely some Bregman divergence, on the space of beliefs about the event; signals are substitutes if the average change in one’s belief about the event is dimininishing in the amount of information already known.
. We give game-theoretic applications of these definitions, primarily on information aggregation in prediction markets. When strategic agents have heterogeneous, valuable information, we would like to understand when and how their information is revealed and aggregated in an equilibrium of strategic play. Prediction markets, which are toy models of financial markets, are possibly the simplest setting capturing the essence of this question. However, although the efficient market hypothesis states that information is quickly aggregated in financial markets (Fama, 1970), despite much research on this question in economics (e.g. Kyle (1985); Ostrovsky (2012)) and computer science (e.g. Chen et al. (2007); Dimitrov and Sami (2008); Gao et al. (2013)), very little was previously known about how quickly information is aggregated in markets except in very special cases.
We address the main open question regarding strategic play in prediction markets: When and how is information aggregated? We show that informational substitutes imply that all equilibria are of the “best possible” form where information is aggregated immediately, while complements imply “worst possible” equilibria where aggregation is delayed as long as possible. Furthermore, the respective converses hold as well; e.g., if an information structure guarantees the “best possible” equilibria, then it must satisfy substitutes.
Theorem** (Informal).**
In a prediction where trader’s signals are strict, strong substitutes with respect to the market scoring rule, in all equilibria, traders rush to reveal and aggregate information immediately. Conversely, if signals are not strong substitutes, then there are arrival orders for traders where no equilibrium has immediate aggregation.
Theorem** (Informal).**
In a prediction market where trader’s signals are strict, strong complements with respect to the market scoring rule, in all equilibria, traders delay revealing and aggregating information as long as possible. Conversely, if signals are not strong complements, then there are arrival orders where no equilibrium exhibits full delaying.
Informational S&C thus seem as fundamental to equilibria of (informational) markets as substitutable items are in markets for goods.
We believe that informational S&C have the potential for broad applicability in other game-theoretic settings involving strategic information revelation, and toward this end, give some additional example applications. We show that S&C characterize analogous “rush/delay” equilibria in some models of machine-learning or crowdsourcing contests (Abernethy and Frongillo, 2011; Waggoner et al., 2015) and question-and-answer forums (Jain et al., 2009).
. We give algorithmic applications, focusing on the complexity of approximately-optimal information acquisition. Namely, we define a very broad class of problems, termed SignalSelection, in which a decision maker wishes to acquire information prior to making a decision, but has constraints on the acquisition process. For instance, a company wishes to purchase heterogeneous, pricey data sets subject to a budget constraint, or to place up to sensors in an environment. We show that substitutes imply efficient approximation algorithms in many cases such as a budget constraint; this extends to an adaptive version of the problem as well. We also show that the problem is hard in general and in the complements case, even when signals are independent uniform bits.
Theorem** (Informal).**
For the SignalSelection problem with e.g. cardinality or budget (“knapsack”) constraints, in the oracle model of input: If signals are weak substitutes, then polynomial-time approximation algorithms exist; but in general, or even if signals are assumed to be complements, no algorithm can achieve nonzero approximation with subexponentially many oracle queries.
These results offer a unifying perspective on a variety of similar “submodularity-based” solutions in the literature (Krause and Guestrin, 2005b; Guestrin et al., 2005; Krause and Guestrin, 2009; Golovin and Krause, 2011).
. We investigate the structure of informational S&C. We give a variety of tools and insights for both identifying substitutable structure and designing for it. For instance, we provide natural geometric and information-theoretic definitions of S&C and show they are equivalent to the submodularity-based definitions.
We address two fundamental questions: Are there (nontrivial) signals that are substitutes for every decision problem? Second, given a set of signals, can we always design a decision problem for which they are substitutes? In the game-theoretic settings above, this corresponds to design of mechanisms for immediate aggregation, somewhat of a holy grail for prediction markets. In algorithmic settings, it has relevance for the design of submodular surrogates (Chen et al., 2015). Unfortunately, we give quite general negative answers to both questions. Surprisingly, more positive results arise for complements. We give the geometric intuition behind these results and point toward heuristics for substitutable design in practice.
In summary, the contributions of this paper are twofold: (a) in the definitions of informational S&C, along with a body of evidence that they are natural, tractable, and useful; and (b) in the applications, in which we resolve a major open problem on strategic information revelation as well as give a unifying and general framework for a broad algorithmic problem. Our results on structure and design of informational S&C points to potential for these very general definitions and results to have concrete applications.
Taken all together, we believe these results give evidence that informational S&C, in analogy with the successful theories of substitutable goods, have a natural and useful role to play in game theory, algorithms, and in connecting the two.
1.3 Outline
Two sections have been placed in the appendix for convenience. Appendix A gives a detailed survey of related work in a variety of areas. In Appendix B, we overview and justify our general approach to defining informational S&C, including historical context, tradeoffs, and intuition. We particularly focus on a comparison to the proposed definitions of Börgers et al. (2013), which we build on in this paper.
In Section 2, we concisely and formally define informational S&C. We show that prediction problems, and the modern convex analysis understanding of them, can be used to analyze general decision problems. Leveraging these tools, we give three equivalent definitions from seemingly-disparate perspectives.
In Section 3, we present game-theoretic applications. Primarily, we show that informational substitutes (complements) characterize best-case (worst-case) information aggregation in prediction markets.
In Section 4, we present algorithmic applications. We define SignalSelection, a class of information acquisition problems, and show that substitutes correspond to efficient approximation algorithms while there are strong hardness results in general.
In Section 5, we investigate informational S&C themselves with an eye toward the previous applications. We give some results on general classes of S&C and on the design of prediction or optimization problems for which a given information structure is substitutable, with both game-theoretic and algorithmic implications.
Section 6 summarizes and discusses future work.
2 Definitions and Foundations
2.1 Setting: information structure and decision problems
We now formally present the setting and definitions. Motivation for the choices made and relation to prior work, particularly Börgers et al. (2013), are described in depth in Section B.
2.1.1 Model of information and decision problems
Information structure.
We take a standard Bayesian model of probabilistic information. There is a random event of interest to the decisionmaker, e.g. . There are also “base signals” , modeled as random events. These represent potential information obtained by a decision-maker, e.g. . An information structure is given by , , and a prior distribution on outcomes . For simplicity, we assume that all and have a finite set of possible outcomes.
In addition to the base signals, there will be other signals that intuitively represent combinations of base signals. Formally, there is a set of signals, with a generic signal usually denoted or . Any subscripted always refers to a base signal, while may in general be any signal in . We will describe how is generated from momentarily, in Section 2.1.2.
We will use lower-case to refer to probability distributions on , the event of interest. The notation refers to the probability that , while , and so on. The notation refers to the probability that conditioned on , obtained from the prior via a Bayesian update: . We will sometimes use the shorthand notation to refer to the posterior distribution on conditioned on , similarly for when and , and so on. We will abuse notation and write to represent a set of outcomes, so for instance we may write ; similarly for signals. We also sometimes write for the expectation over outcomes of .
Decision problems and value function.
A single-agent decision problem consists of a set of event outcomes , a decision space , and a utility function , where is the utility for taking action when the event’s outcome is . This decision problem, in the context of an information structure, will be how signals derive their value.
Specifically, given the prior , the decision that maximizes expected utility is . But now suppose a Bayesian, rational agent knows and will first observe the signal , then update to the posterior on , and then choose a decision maximizing expected utility for this posterior belief. In this case, her utility will be given by the following “value” function:
[TABLE]
We will use to denote a null signal, so that is the expected utility for deciding based only on the prior distribution. Where the decision problem and information structure are evident from context, we will omit the superscripts .
Intuitively, is analogous to a valuation function over subsets of items. However, inputs to may not only represent subsets of , but also signals that give partial information about them.
2.1.2 Signal lattices
We will consider three kinds of signal sets , leading to “weak”, “moderate”, and “strong” substitutes and complements. In each case, the set of signals will form a lattice.
Definition 2.1.1**.**
A lattice is a set together with a partial order on it such that for all , there are a meet A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B and join A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B in satisfying:
A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B\preceq A\preceq A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B and A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B\preceq B\preceq A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B; and 2. 2.
the meet and join are the “highest” and “lowest” (respectively) elements in the order satisfying these inequalities.
In a lattice, denotes the “bottom” element and the “top” element, i.e. for all , if they exist.
The following definition illustrates one very common lattice, that of subsets of a ground set.
Definition 2.1.2**.**
The subsets signal lattice generated by consists of an element for each subset of , where is the signal conveying all realizations . Its partial order is if and only if . Hence, its meet operation is given by set intersection and join by set union.
The bottom element of the subsets lattice exists and is a null signal corresponding to the empty set (we will use this notation somewhat often), while the top element also exists and corresponds to observing all signals. Also, the partial ordering denotes less informative. These facts will continue to hold for the other two signal lattices we define.
For the other two lattices, we utilize the main idea from the classic model of information due to Aumann (1976). Let the set consist of all signal realizations in the support of the prior distribution. Now, a partition is a collection of subsets of such that each is in exactly one subset. Each signal corresponds to a partition of with one subset for each outcome , namely, the set of realizations . Example 2.1.1, given after the definition of discrete signal lattice, illustrates the partition model.
As in Aumann’s model, the partitions of form a lattice, each partition corresponding to a possible signal. The partial ordering is that if the partition of is “coarser” than that of . One partition is coarser than another (which is finer) if each element of the former is partitioned by elements of the latter. The join of two partitions is the coarsest common refinement (the coarsest partition that is finer than each of the two), while the meet is the finest common coarsening. Example 2.1.2, given after the definition, illustrates coarsenings and refinements.
Definition 2.1.3**.**
The discrete signal lattice generated by consists of all signals corresponding to partitions of , where is the subset of with positive probability. Its partial order has if the partition associated to is coarser than that of .
Example 2.1.1* (Signals modeled as partitions).*
(a) We have two independent uniform bits and . In this case . Here is modeled as the partition consisting of two elements: and . The first element of the partition is the set of realizations where , while the second is the set of realizations where .
(b) Now modify the example so that and are perfectly correlated: with probability , , and with probability , . Here, and corresponds to the partition consisting of and .
(c) Now revisit the first case where and are independent. Imagine an agent who observes both base signals and wishes to reveal only the XOR of the bits. This new signal released by the agent, call it , may also be modeled as a partition of , where the elements of the partition are and .
Example 2.1.2* (The order given by coarsenings).*
(a) We have a single signal which is distributed uniformly on . Then and ’s partition contains these six subsets: .
(b) Given the above information structure, suppose that an agent holding will commit to releasing some deterministic function of . In terms of information revealed, the agent may map each realization to a different report – this is the same as just revealing – or she may map some realizations to the same report. Suppose that she reports “small” whenever and reports “large” whenever . The information revealed by this report is captured by a binary signal corresponding to the partition with two elements: and . The partition of is coarser than that of , so on the discrete lattice.
(c) Given the same information structure, imagine that the agent will commit to releasing “even” whenever and “odd” whenever . This corresponds to a signal whose partition has these two elements and is again coarser than that of .
(d) Consider the above two signals and . They are incomparable: Neither is coarser nor finer than the other. The meet A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B will be the null signal111Formally, this is the signal whose partition contains a single element: all of . , intuitively because given , one cannot guarantee anything about the outcome of . The join A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B intuitively corresponds to observing both signals. Let C=A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B. The partition corresponding to has the following four elements: , , , . These each correspond to a realization of the signal ; call the realizations respectively . Here, when for example “small” and “even”, then and an observer of would know that . When “large” and “even”, then and an observer of would know that , updating to a posterior on these possibilities.
For the third and strongest notion, we extend the model by, intuitively, appending randomness to the signals on the discrete lattice. Given any signal on the discrete lattice, a “garbling” of can be captured by a randomized function of ; but this may be modeled as a deterministic function where is a uniform random variable222In some applications, it may be more desirable to use an infinite string of independent uniform bits.. This observation allows us to “reduce” to the deterministic case, but where each possible signal carries extra information in the form of some independent randomness.
Specifically, let be defined as above (the subset of with positive probability) and, for each partition of , let drawn independently from the uniform distribution. Let where \mathbf{R}=\operatorname*{\mathrel{\text{\scalebox{1.3}{\times}}}}_{\Pi}R_{\Pi}. Now, we proceed as before, but using .333To be precise, we should restrict to measurable subsets using the Lebesgue measure on . We will omit this discussion for simplicity; if concerned, the reader may alternatively assume that each is drawn uniformly from a massive but finite set, with some tiny approximation carried through our results.
Definition 2.1.4**.**
The continuous signal lattice consists of a signal corresponding to each partition of . Its partial order has if the partition associated to is coarser than that of .
Example 2.1.3* (Modeling garblings via the continuous lattice).*
Consider a uniformly random bit as the only base signal; the resulting is . Now consider the garbling where, if , then output “happy” with probability and “sad” otherwise; if it equals , then output “happy” with probability and “sad” otherwise. Call the output of the garbling . Then can be modeled as a partition of with the following two subsets: , and . Here the first realization of corresponds to the output “happy”, while the second corresponds to output “sad”. To see this, note for instance that the first realization contains all the elements of where and the randomness variable . So when , assuming is drawn uniformly and independently from , then the outcome of is “happy” with probability . On the continuous lattice, corresponds to the partition of singletons such as , , and so on. That is, it corresponds to observing both the original binary bit as well as the random real number . Because this partition is finer than that corresponding to , we have on the continuous lattice.
The use of “happy” and “sad” for the outputs in the above example illustrates that it is not important, when considering the information conveyed by signal , to consider what its realizations were named. All that matters is their distributions, e.g. the partitions they represent.
Example 2.1.4* (Modeling garblings, continued).*
Again let be a uniformly random bit, and now suppose is obtained by adding to independent Gaussian noise with mean [math] and variance . In this case, intuitively, each outcome of (say ) represents two possibilities (such as and the Gaussian is , or and the Gaussian is ). can be modeled as a partition of where is interpreted as the quantile of the outcome of the Gaussian. Each member of the partition has two elements. These can be written and with and , using the standard normal CDF . Given the realization , the posterior distribution on is given by a Bayesian update depending on the probability density of the Gaussian at [math] and at .
2.2 The definitions of substitutes and complements
We utilize a common notion of diminishing and increasing marginal value. For example, the idea of submodularity is that a lattice element ’s marginal contribution to should be smaller that to some .
Definition 2.2.1**.**
A function from a lattice to the reals is submodular if it exhibits diminishing marginal value: For all on the lattice with A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B\preceq A^{\prime}\preceq A,
[TABLE]
It is supermodular if it exhibits increasing marginal value: For all on the lattice with A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B\preceq A^{\prime}\preceq A, the above inequality is reversed. The sub- or super-modularity is strict if, whenever and are incomparable on the lattice’s ordering and , the inequality is strict.
Definition 2.2.2** (Weak Informational S&C).**
In the context of a decision problem and prior , the signals are weak substitutes if is submodular on the subsets signal lattice generated by . They are weak complements if is supermodular on their subsets signal lattice. The signals are strict substitutes (complements) if the is strictly submodular (supermodular).
For moderate and strong substitutes, we will use a strengthening of submodularity by requiring diminishing marginal value with respect to, respectively, all deterministic and randomized functions of a signal.444Previous versions of this paper used submodularity on the discrete and continuous signal lattices, which is a more restrictive definition. We only need this weaker definition for results in this paper, but in general both are interesting and the right choice may be context-dependent; or there could be other interesting variations.
Definition 2.2.3** (Moderate and Strong Informational S&C).**
In the context of a decision problem and prior , the signals are moderate (respectively, strong) substitutes if, for all on the subsets signal lattice, and for all on the discrete (respectively, continuous) signal lattice with A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B\preceq A^{\prime}\preceq A,
[TABLE]
They are moderate (strong) complements if, for all on the subsets signal lattice and all on their discrete (continuous) signal lattice with A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B\preceq A^{\prime}\preceq A, the above inequality is reversed.
The signals are strict substitutes (complements) if whenever and are incomparable and , the respective inequality is strict.
Verbally, S&C capture that the more pieces of information one has, the less valuable (respectively, more valuable) becomes. The levels of weak, moderate, and strong capture the senses in which “pieces of information” is interpreted. Weak substitutes satisfy diminishing marginal value whenever a whole signal is added to a subset of signals. However, they do not give guarantees about marginal value with respect to partial information about signals. Moderate and strong substitutes respectively satisfy diminishing marginal value when deterministic (randomized) partial information about a signal is revealed.
Observation 2.2.1**.**
Strong substitutes imply moderate substitutes, which imply weak substitutes. The same holds for complements.
Proof.
The respective lattices are supersets, i.e. the continuous signal lattice is a superset of the discrete lattice which is a superset of the subsets lattice, and the partial orderings agree. So each substitutes definition requires to satisfy a set of inequalities at various points, and in going from weak to moderate to strong substitutes, we simply increase the set of required inequalities that signals must satisfy. ∎
Example 2.2.1* (Substitutes).*
The event is a uniformly random bit and the two signals and . That is, both signals are always equal to . The decision problem is to predict the outcome of by deciding either [math] or , with a payoff of for correctness and [math] otherwise. In this case, one can immediately see that and are e.g. weak substitutes, as a second signal never gives marginal benefit over the first.
Example 2.2.2* (Complements).*
The event and decision problem are the same as in Example 2.2.1, but this time and are uniformly random bits with , the XOR of and . In this case, and are immediately seen to be e.g. weak complements, as a first signal never gives marginal benefit over the prior.
Example 2.2.3* (Weak vs moderate).*
Here is an example of weak substitutes that are not moderate substitutes. Intuitively, we will pair the previous two examples. The event consists of a pair of independent uniformly random bits. The decision problem is to predict both components of , getting one point for each correct answer. Let the random variable and . Let the random variables and be uniformly random bits such that .
Now, consider the signals and . Intuitively, the first component of each signal completely determines , while the second component gives no information about until combined with the other signal. Hence these signals intuitively have both substitutable and complementary internal structure. Consider the subsets lattice . If we modify the decision problem such that predicting the first component of is worth points, then these signals are weak substitutes: Each alone is worth points, while together they are worth points. On the other hand, if we modify the decision problem such that the second component of is worth points, then these signals become weak complements for analogous reasons.
On the other hand, these signals are neither moderate substitutes nor moderate complements. One way to see this is to consider “coarsening” into the signal ; this has diminishing marginal value when added to . However, we could also coarsen into the signal , which has increasing marginal value when added to .
2.3 Scoring rules and a revelation principle
We now introduce proper scoring rules and prove a useful “revelation principle”.
A scoring rule for an event is a function , so that is the score assigned to a prediction (probability distribution) when the true outcome realized is . Define the useful notation for the expected score under true belief for reporting to the scoring rule.
The scoring rule is (strictly) proper if for all , setting (uniquely) maximizes the expected score . In other words, if is distributed according to , then truthfully reporting to the scoring rule (uniquely) maximizes expected score.
A fundamental characterization of scoring rules is as follows:
Fact 2.3.1** (McCarthy (1956); Savage (1971); Gneiting and Raftery (2007)).**
For every (strictly) proper scoring rule , there exists a (strictly) convex function with (1) and (2)
[TABLE]
where is a subgradient of at and is the probability distribution on putting probability on and [math] elsewhere.
Furthermore, for every (strictly) convex function , there exists a (strictly) proper scoring rule such that (1) and (2) hold.
Proof.
Given any (strictly) convex , we first check that the induced is (strictly) proper. Select a subgradient at each point . The expected score for reporting when is distributed according to is
[TABLE]
Note that the inequality follows simply because, for any convex , if we take the linear approximation at some point and evaluate it at a different point , this lies below . Furthermore, if is strictly convex, then this inequality is strict, implying strict properness.
Now, given a (strictly) proper , we show that it has the stated form. Define . Note that is a linear function of . By properness, each . Since is a pointwise maximum over a set of linear functions of , is convex. If was strictly proper, then was the unique maximum at every point, implying that is strictly convex.
Now we claim that is a subtangent of at : it is linear, equal to at , and everywhere below by definition of . So in particular , as promised. ∎
Example 2.3.1*.*
The scoring rule is , i.e. the logarithm (usually base ) of the probability assigned to the realized event. The expected score function is , where is the Shannon entropy function.
Notice that a scoring rule is a special case of a decision problem: The utility function is the scoring rule , is the event picked by nature, and the decision space . We now show that in a sense, scoring rules capture all decision problems. This is not surprising or difficult, and may have been observed prior to this work; but we formalize it because it captures a very nice and useful intuition.
Theorem 2.3.1** (Revelation principle).**
For any decision problem , there exists a proper scoring rule that is equivalent to the original decision problem in that for all information structures and signals , .
Proof.
The idea of the proof, as suggested by the name, is simply for the agent to report her belief about to the scoring rule and for the scoring rule to simulate the optimal decision for this belief, paying the agent according to the utility derived from that decision. For a given distribution (“belief”) on , let be the optimal decision, i.e. . Now, given , let
[TABLE]
First let us show properness, i.e. . We have
[TABLE]
using the definition of .
Now let us check equivalence to the original problem. Let be the distribution on conditioned on . We have
[TABLE]
∎
This reduction is not necessarily computationally efficient, because the input to the scoring rule is a probability distribution over which may have a large number of outcomes. We note two positives, however. First, the reduction does not necessarily need to be computationally efficient to be useful for proofs and analysis. Second, in any case where it seems reasonable to assume that the agent can solve her decision problem, which involves an expectation over possible outcomes of , it seems reasonable to suppose that she can efficiently represent or query her beliefs. In this case we may often expect a computationally efficient reduction and construction of . This is a direction for future work.
The revelation principle (Theorem 2.3.1) and scoring rule characterization (Fact 2.3.1) together imply the following extremely useful fact about general decision problems. We do not claim originality for it; the idea can be found in Savage (1971) and similar ideas or statements are present in e.g. Frongillo and Kash (2014) and Babaioff et al. (2012). But it is worth emphasizing because we will put it to extensive use in this paper.
Corollary 2.3.1**.**
For any decision problem there exists a corresponding convex function , and for every such there exists a decision problem , such that is the expected utility for acting optimally when the agent’s posterior belief on is .
Hence for instance , where is the posterior on given .
As an example of usefulness, we provide a concise proof of the following classic theorem.
Fact 2.3.2** (More information always helps).**
In any decision problem, for any signals , \mathcal{V}(A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B)\geq\mathcal{V}(A). In other words, more information always improves the expected utility of a decision problem. In other words, is a monotone increasing function on the signal lattices.
Proof.
Recall that we are using the notation for the distribution on conditioned on , and so on. In particular, is a vector, i.e. . By the revelation principle, for some convex we have , and
[TABLE]
To obtain the last equality: Each term in the sum consists of the scalar multiplied by the vector , and for each coordinate of the vector, we have . Then . ∎
2.4 Characterizations
In this section, we show how the substitutes and complements conditions can be phrased using the convexity connection just derived. We will leverage this structure to identify characterizations or alternative definitions of substitutes and complements. For brevity, we will focus on substitutes, but in all cases the extension to complements is immediate.
From Corollary 2.3.1, we also get immediately the following characterization:
Definition 2.4.1** (Substitutes via convex functions).**
For any decision problem, letting be the associated expected score function, signals are respectively (weak, moderate, strong) substitutes for that decision problem if and only if, for all on the subsets lattice and all on the respectively (subsets, discrete, continuous) lattice with A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B\preceq A^{\prime}\preceq A,
[TABLE]
To be clear, our use of parentheses here means that we are giving three definitions: one for weak substitutes, where the condition must hold for all on the subsets lattice; and analogously for moderate substitutes with the discrete lattice and strong substitutes with the continuous lattice.
We view Definition 2.4.1 mostly as a tool, although it may convey some intuition on its own as well. Definition 2.4.1 will be pictured in Figure 3 along with the final characterization.
2.4.1 Generalized entropies
Here, we seek an alternative interpretation of the definitions of S&C in terms of information and uncertainty. To this end, for any decision problem, consider the convex expected score function and define . Then is concave, and we interpret as a generalized entropy or measure of information. The justification for this is as follows: Define the notation , where is the distribution on conditioned on . Then concavity of implies via Jensen’s inequality that for all , we have . In other words, more information always decreases uncertainty/entropy.
We propose that this is the critical axiom a generalized entropy must satisfy: If more information always decreases , then in a sense it measures uncertainty, while if more information sometimes increases , then it should not be considered a measure of uncertainty. However, admittedly, the appeal of this definition may increase by adding additional axioms as are common in the literature, such as maximization at the uniform distribution and value zero at degenerate distributions. Another very intriguing axiom would be a relaxation of the “chain rule” in either direction: is restricted to be either greater than or less than . (Note that the chain rule itself, , along with concavity, uniquely characterizes Shannon entropy.) Such axioms may have interesting consequences for informational S&C. Examining the structure of S&C under such axioms represents an intriguing direction for future work.
Under this interpretation, Definition 2.4.1 can be restated:
Definition 2.4.2** (Substitutes via generalized entropies).**
For any decision problem , let the generalized entropy function where is the corresponding expected score function. Signals are (weak, moderate, strong) substitutes for if and only if, for all on the subsets lattice and all on the (subsets, discrete, continuous) lattice with A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B\preceq A^{\prime}\preceq A,
[TABLE]
Intuitively, Definition 2.4.2 says this: Consider the expected amount of information about that is revealed upon learning , given that some information will already be known. Use the generalized entropy to measure this information gain. Then substitutes imply that, the more information one has, the less information reveals. On the other hand, complements imply that, the more information one has, the more information reveals.
Example 2.4.1*.*
Revisiting Example 2.2.1, where was a uniform bit and , imagine predicting against the scoring rule. Our previous observations imply that here the generalized entropy function is Shannon entropy . We have and , which already shows that and are weak substitutes.
If we instead revisit Example 2.2.2, where with uniformly random bits, and again consider predicting according to the scoring rule, then we see that , while , already proving that and are weak complements.
2.4.2 Bregman divergences
Given a convex function , the Bregman divergence of is defined as
[TABLE]
In other words, it is the difference between and the linear approximation of at , evaluated at . (See Figure 3.) Another interpretation is to consider the proper scoring rule associated with , by Fact 2.3.1, and note that , the difference in expected score when reporting one’s true belief versus lying and reporting . The defining property of a convex function is that this quantity is always nonnegative. This can be observed geometrically in Figure 1 as well; there .
This notion is useful to us because, it turns out, all marginal values of information can be exactly characterized as Bregman divergences between beliefs.
Lemma 2.4.1**.**
In a decision problem, for any signals , the marginal value of given is
[TABLE]
where is the associated expected score function and is the distribution of given .
Proof.
[TABLE]
because , so . ∎
Definition 2.4.3** (Substitutes via divergences).**
For any decision problem , let be the Bregman divergence of the corresponding expected score function . Signals are (weak, moderate, strong) substitutes if and only if, for all on the subsets lattice and all on the (subsets, discrete, continuous) lattice with A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B\preceq A^{\prime}\preceq A,
[TABLE]
This can be interpreted as a characterization of S&C where serves as a distance measure of sorts (although it is not in general a distance metric). The characterization says that, if we look at how “far” the agent’s beliefs move upon learning , on average, then for substitutes this distance is decreasing in how much other information is available to the agent. But for complements, the more information the agent already has, the farther she expects her beliefs to move on average upon learning .
Example 2.4.2*.*
For the scoring rule, is exactly the KL-divergence or relative entropy between distributions on and . If we recall Example 2.2.1, in which was a random bit and , we can consider the decision problem of prediction against the scoring rule. In this case, the prior , while the posteriors , , and the same for . Hence . But the posteriors conditioned on both signals are the same, e.g. . Hence .
This already shows that and are weak substitutes. And in fact, if , then this argument extends to show that and are substitutes in any decision problem (as they should be), because given , an update on moves the posterior belief a distance [math].
3 Game-Theoretic Applications
3.1 Prediction markets
A prediction market is modeled as a Bayesian extensive-form game. The market’s setting is specified by a strictly proper scoring rule and an information structure with prior and event , and set of signals . We assume these signals are “nontrivial” in that, given all signals but , the distribution of changes conditioned on .
An instantiation of the market is specified by a set of traders, each trader observing some subset of the signals (call the resulting signal ), and an order of trading , where at each time step , it is the turn of agent to trade. We assume that no trader participates twice in a row (if they do, it is without loss to delete one of these trading opportunities).
The market proceeds as follows. First, each trader simultaneously and privately observes , updating to a posterior belief. Then the market sets the initial prediction , which we assume to be the prior distribution on . We will also refer to a market prediction as the market prices. Then, for each , trader arrives, observes the current market prediction , and may update it to (“report”) any .
After the last trade step , the true outcome of is observed and each trader receives payoff . Thus, at each time , trader is paid according to the scoring rule applied to , but must pay the previous trader according to the scoring rule applied to . The total payment made by market “telescopes” into .
At any given time step , trader is said to be reporting truthfully if she moves the market prediction to her current posterior belief on . In other words, she makes the myopically optimal trade.
The natural solution concept for Bayesian games is that they be in Bayes-Nash equilibrium, where for every player, her (randomized) strategy — specifying how to trade at each time step as a function of her signal and all past history of play — maximizes expected utility given the prior and others’ strategies.
Because this is a broad class of equilibria and can in general include undesirable equilibria involving “non-credible threats”, it is often of interest in extensive-form games to consider the refinement of perfect Bayesian equilibrium. Here, at each time step and for each past history, a player’s strategy is required to maximize expected utility given her beliefs at that time and the strategies of the other players. (Note the difference to Bayes-Nash equilibrium in which this optimality is only required a priori rather than for every time step.) Here, at any time step and history of play, players’ beliefs are required to be consistent with Bayesian updating wherever possible. (It may be that one player deviates to an action not in the support of her strategy; in this case other players may have arbitrary beliefs about the deviator’s signal.)
To be clear, every perfect Bayesian equilibrium is also a Bayes-Nash equilibrium. Hence, we note that an existence result is strongest if it guarantees existence of perfect Bayesian equilibrium. Meanwhile, a uniqueness or nonexistence result is strongest if it refers to Bayes-Nash equilibrium.
Distinguishability criterion.
For most of our results, we will need a condition on signals equivalent or similar to those used in prior works (Chen et al., 2010; Ostrovsky, 2012; Gao et al., 2013) in order to ensure that traders can correctly interpret others’ reports. Formally, we say that signals are distinguishable if for all subsets and realizations , of the signals such that, for some , ,
[TABLE]
We believe it may be possible to relax this criterion and/or interpret such criteria within the S&C framework, and this is a direction for future work.
Our notation in prediction markets.
Note that, for the proper scoring rule with associated convex , along with the prior , we have the associated “signal value” function :
[TABLE]
In other words, is the expected score for reporting the posterior distribution conditioned on the realization of .
A second key point is that, to a trader whose current information is captured by a signal , the set of strategies available to that trader can be captured by the space of signals on the continuous lattice. This follows because any strategy is a randomized function of her information, so the outcomes of the strategy can be labeled as outcomes of a signal .
3.1.1 Substitutes and “all-rush”
We now formally define an “all-rush” equilibrium and show that it corresponds to informational substitutes. The naive definition would be that each trader reports truthfully at their first opportunity, or (hence) at every opportunity. This turns out to be correct except for one subtlety. Consider, for example, the final trader to enter the market. Because all others have already revealed all information, this last trader will be indifferent between revealing immediately or delaying. Similarly, consider three traders and the order of trading . If trader truthfully reports at time , then trader is not strictly incentivized to report truthfully at time . She could also delay information revelation until time .
Definition 3.1.1**.**
An all-rush strategy profile in a prediction market is one where, if the traders are numbered in order of the first trading opportunity, then each trader reports truthfully at some time prior to ’s first trading opportunity (with the final trader reporting truthfully prior to the close of the market).
Before presenting the main theorem of this section, we give the following useful lemma (which is quite well known):
Lemma 3.1.1**.**
In every Bayes-Nash equilibrium, every trader reports truthfully at her final trading opportunity.
Proof.
Consider a time at which trader makes her final trade. Fix all strategies and any history of trades until time ; then ’s total expected payoff from all previous time steps is fixed as well and cannot be changed by any subsequent activity. Meanwhile, ’s unique utility-maximizing action at time is to report truthfully, by the strict properness of the scoring rule. If does not take this action, then her entire strategy is not a best response: She could take the same strategy until time and modify this last report to obtain higher expected utility. Therefore, in Bayes-Nash equilibrium, reports her posterior on her final trading opportunity. ∎
Theorem 3.1.1**.**
If signals are distinguishable and are strict, strong substitutes, then for every set of traders and trading orders, every Bayes-Nash equilibrium is all-rush.
Proof.
Let the traders be numbered in order of their first trading opportunity and let be the signal of trader . Before diving in, we develop a key idea. In equilibrium, we can view the market prediction at time as a random variable. Then, construct a “signal” capturing the information contained in . This can be pictured as the information conveyed by to an “outside observer” who knows the prior distribution and the strategy profile, but does not have any private information. Furthermore, if traders have participated thus far, then is an element of the continuous signal lattice with C^{(t)}\preceq B_{1}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}\cdots\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B_{k}, because is a well-defined, possibly-randomized function of B_{1}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}\cdots\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B_{k}. Finally, if all participating traders have been truthful, then is a member of the subsets signal lattice , as it exactly reveals the subset of the signals held by those traders.
Now, let be ’s final trading opportunity prior to ’s first trading opportunity. We prove by backward induction on that, in BNE and for any participant participating at some time , the following holds: if is an element of the subsets lattice , then reports truthfully at . Now, suppose we have successfully proven this claim by backward induction; let us finish the proof. The claim implies that really is in and really is truthful at all such time steps for the following reason: is the null signal and is an element of , so trader participates truthfully, which implies that , which implies that participates truthfully, and so on. So in any BNE, all participants play all-rush strategies.
Now let us prove the statement. For the base case , the trader participating at the final time step is truthful by Lemma 3.1.1.
Now for the inductive step, consider any for some . If is ’s final trading opportunity, then by Lemma 3.1.1, in BNE reports truthfully at . If , then there is nothing to prove.
Otherwise, let be ’s next trading opportunity after . By inductive hypothesis, is truthful at time and thereafter. We compute ’s expected utility for any strategy, and show that if is not truthful at , she can improve by deviating to the following strategy: Copy the previous strategy up until , report truthfully at , and make no subsequent updates.
At , ’s strategy can be described as reporting truthfully according to C^{(t^{\prime})}=C^{(t^{\prime}-1)}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B_{i}. For this trade, obtains expected profit , and obtains no subsequent profit once her information is revealed. Meanwhile, consider ’s strategy at time , which induces some signal C^{(t)}\preceq B_{i}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}C^{(t-1)}. For this trade, obtains expected profit at most . This follows because a trade conveying signal obtains at most .555Although we don’t explicitly use it here, this implies that in equilibrium, every , that is, the price at time equals the posterior distribution on conditioned on all information that has been revealed so far, including at time .
Let be ’s total expected utility at time and greater. Once reports truthfully at , she expects to make no further profit in equilibrium. So
[TABLE]
Now suppose is in and is not reporting truthfully at . This implies that . By strong, strict substitutes, gives higher marginal benefit to than to :
[TABLE]
So
[TABLE]
But can achieve this by deviating to being truthful at time , then not participating at any subsequent times. (This follows because if is truthful at time , she reveals the signal C^{(t-1)}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B_{i}, which is exactly the same as C^{(t)}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B_{i}.) This deviation does not affect ’s utility from any previous times, so it is a strategy with higher total expected utility. So ’s only BNE strategy can be to be truthful at . ∎
Theorem 3.1.2**.**
If signals are distinguishable and are not strong substitutes, then there is a trading order such that no perfect Bayesian equilibrium is all-rush.
Proof.
The assumptions imply that there signals on the subsets lattice and some on the continuous lattice with and
[TABLE]
Then in particular, we can consider the scenario with two traders where “Alice” has signal (i.e. she observes the corresponding subset of signals) and Bob has , with a trading order Alice-Bob-Alice. In perfect Bayesian equilibrium (PBE), Bob must be truthful at his trading opportunity according to his beliefs even if Alice deviates from her strategy. By distinguishability, Alice can infer his signal from this truthful report, so in any PBE, Alice is truthful and correct in predicting at the second opportunity. Hence the two traders’ expected utilities sum to the constant amount \mathcal{V}(A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B)-\mathcal{V}(\bot), even when Alice deviates. If Alice reports truthfully at her first opportunity (the all-rush strategy), then Bob’s expected utility is \mathcal{V}(A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B)-\mathcal{V}(A). But if Alice reports according to , then Bob’s expected utility is at most \mathcal{V}(A^{\prime}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B)-\mathcal{V}(A^{\prime}), which by assumption of non-substitutes is strictly smaller. This implies that Alice prefers the deviation, so truthful reporting (and thus all-rush) could not have been an equilibrium. ∎
3.1.2 Complements and “all-delay”
We begin by defining an “all-delay” strategy profile, analogous to all-rush.
Definition 3.1.2**.**
An all-delay strategy profile in a prediction market is one where, when the traders are numbered in order of their final trading opportunity, each trader reveals no information until after trader ’s final trading opportunity.
Theorem 3.1.3**.**
If signals are distinguishable and are strict, strong complements, then for every set of traders and trading order, every perfect Bayesian equilibrium is an all-delay equilibrium.
Proof.
The ideas will be substantially the same as in Theorem 3.1.1, but the deviation argument is somewhat trickier. In the substitutes “rush” case, an agent could deviate to immediate truth-telling and ignore all subsequent consequences. Now, we will need agents to deviate to delaying all information revelation, relying on their opponents’ response to ensure this becomes profitable later. This is also the reason that we restrict to perfect Bayesian equilibrium.
The proof is by backward induction. We show that in any perfect Bayesian equilibrium: at each time , if is on the subsets lattice, then players play an all-delay strategy from time onward. (As in the proof of Theorem 3.1.1, let be the signal induced by the random variable in equilibrium. Because in PBE strategies are well-defined in every subgame, is also well-defined off the equilibrium path.) As in Theorem 3.1.1, this will prove the statement, because is on the subsets lattice, corresponding to , so the first trader plays all-delay at time , implying that is on the subsets lattice, etc.
For the base case, at , this is the trader’s final opportunity, so she is truthful by Lemma 3.1.1, which constitutes an all-delay strategy from onward.
Now consider any trading time and participant trading at time .
First suppose is ’s final trading opportunity; then by Lemma 3.1.1, she reports truthfully at this time. By induction, traders play all-delay at all times after , so this shows that they play all-delay from time onward.
If is not a trader’s final trading opportunity, but is after ’s final trading opportunity, then there is nothing to prove for this time step. So suppose trader is trading at time with ’s final opportunity coming at some . The inductive assumption implies that in any subgame starting at time in PBE, does not make any update until some . It also implies that no other trader participates between and . Finally, it implies that all traders participating between time and (exclusive) report truthfully666In a trading order such as , it is possible that reports something nontrivial at time , then reports truthfully at time . But does not participate at time , so ’s multiple reports WLOG telescope into a single truthful report.. Let denote the join of their signals. Then ’s total utility from time onward is
[TABLE]
Now suppose for contradiction that reveals some nontrivial information at time , i.e. . Then can deviate to revealing nothing at time , reporting according to , and being truthful at time . In this case, by assumption of PBE, others continue to best-respond. By inductive assumption, in any subgame of a PBE (which itself must be in PBE), others who participate between and therefore continue to report truthfully, implying that is still revealed between time and . Now, strong, strict complements imply
[TABLE]
So
[TABLE]
But this is ’s utility for the deviation above (note that C_{t}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B_{i}=C_{t-1}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B_{i}). Since the deviation is profitable, this gives a contradiction, implying that (and all players) must play all-delay starting from time in any PBE. ∎
Theorem 3.1.4**.**
If signals are distinguishable and are not strong complements, then there is a set of traders and trading order such that no perfect Bayesian equilibrium is all-delay.
Proof.
Analogous to the substitutes case (Theorem 3.1.2). The assumptions imply that there are subsets-lattice signals and continuous-lattice signal such that
[TABLE]
Then in particular, we can consider the scenario where “Alice” has signal and Bob has , with a trading order Alice-Bob-Alice. In PBE, Bob is truthful even if Alice deviates. By distinguishability, Alice can infer his signal from this truthful report, so in any PBE, Alice is truthful and correct in predicting at the second opportunity. So utilities have the constant sum \mathcal{V}(A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B)-\mathcal{V}(\bot) even when Alice deviates. If Alice reports nothing at her first opportunity, then Bob’s expected utility is . But if Alice deviates to reporting to , then Bob’s expected utility is at most \mathcal{V}(A^{\prime}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B)-\mathcal{V}(A^{\prime}), which is strictly smaller. This implies that Alice prefers the deviation, so truthful reporting (and thus all-rush) could not have been an equilibrium.
Hence, in perfect Bayesian equilibrium, trader cannot play all-delay. ∎
Discussion.
These results show that informational S&C are in a sense unavoidable in the study of settings such as prediction markets. However, the result raises many interesting questions for future work. Two major questions are: How can we identify structures that are substitutes or complements? and How can we design markets to encourage substitutability?
We give some initial steps toward answering these questions in Section 5.
3.2 Other game-theoretic applications
We will now examine a few game-theoretic contexts in which our results have immediate applications or implications. Instead of developing full formal proofs and theorem statements, we focus on illustrating the intuition of how to extend our results and the conceptual lens of informational S&C to those settings.
3.2.1 Crowdsourcing and contests
In the study of crowdsourcing from a theoretical perspective, the “crowd” is a group of agents who hold valuable information and the goal is to design mechanisms that elicit this information. Specifically, here we are interested in “wisdom of the crowd” settings where the total information available to the crowd is greater than that of the most-informed individual, and the goal is to aggregate this information.
Before describing how informational S&C apply in such settings, we would like to contrast with approaches to crowdsourcing that models each user’s contribution as a monolithic submission that has some endogenous quality, such as DiPalantino and Vojnovic (2009); Archak and Sundararajan (2009); Chawla et al. (2015) There, it is impossible to integrate or aggregate user contributions and the problem is to incentivize and select one of the highest possible quality. In these models, information plays no role and the model is equally well-suited to incentivizing production of a high-quality good of which only one is required; sometimes this is explicit in the motivation or model of the literature (Cavallo and Jain, 2012).
Here, we consider cases where users have heterogeneous information and we would like to aggregate it into a final form that is more useful than any one user. The question is how users behave strategically in revealing this information in the contest.
Collaborative, market-based contests for machine learning.
Abernethy and Frongillo (2011) proposes a mechanism for machine learning contests with a prediction market structure. This mechanism was later extended to elicit data points and to more general problems in Waggoner et al. (2015). While these mechanisms have appealing structure, seeming to align participants’ incentives with finding optimal machine-learning hypotheses, the authors did not give results on equilibrium performance or behavior of strategic agents. Here, we briefly describe the framework of this mechanism and how our results can apply.
In a machine learning problem, we are given a hypothesis class . There is some true underlying distribution of data, which is initially unknown. In our setting we assume there is a prior belief on this distribution , which distributed as a random variable . The goal is to select a hypothesis with minimum risk on the true data distribution. Here, for a loss function on hypothesis and a datapoint drawn from .
In the contest mechanism of Abernethy and Frongillo (2011); Waggoner et al. (2015), the mechanism selects an initial market hypothesis . As in the prediction market model of Section 3.1, participants iteratively arrive and propose a new hypothesis at each time . At the end of the contest, the mechanism draws a test data point from the true distribution and rewards each participant by their improvement to the loss of the market hypothesis, i.e. if updated the hypothesis at time from to , then is rewarded for that update.
We observe that prediction markets are a special case of this framework: Each corresponds to some fixed observation, for instance, for a given the data point drawn is always the same . The risk is therefore always equal to for the proper scoring rule used in the market. Thus, the above framework captures prediction markets as a special case. However, we now show that prediction markets capture the essential strategic features of this setting.
Now, notice that in expectation over the test data , this reward is equal to . Furthermore, we can define the utility of the designer to be , that is, the negative of the risk of that hypothesis on that data distribution. Hence, by the revelation principle, there is some proper scoring rule that is payoff-equivalent to . A prediction market with proper scoring rule is strategically identical to the above contest. Thus, with a few small caveats, our above results apply: in a Bayesian game setting where traders have signals and a common prior on the distribution of signals and , substitutes characterize “all-rush” equilibria with immediate aggregation, while complements characterize “all-delay” equilibria.
The caveats are (1) that the scoring rule obtained by the revelation principle will not in general be strictly proper if two beliefs about map to the same optimal hypothesis ; and (2) it is not guaranteed that traders can infer others’ information from their trades without a condition analogous to the distinguishability criterion of Section 3.1. While these hurdles are surmountable, our purpose here is only to mention the key ideas for how the above results on prediction markets will generalize.
This connection immediately presents several questions for future work: When, in a machine-learning setting, should we expect contest participants to have substitutable or complementary information? In particular, if agents hold data sets and the goal is to elicit these data sets using this structure (as explored in Waggoner et al. (2015)), when should we expect data sets to be substitutable? Furthermore, how can we design loss functions so as to encourage substitutability and hence early participation? This last question is discussed in Section 5.4.
Question-and-answer forums.
Jain et al. (2009) propose a model for analyzing strategic information revelation in the context of question-and-answer forums. Initially, some question is posed. Participants have private pieces of information as in the prediction market model, and they arrive iteratively to post answers. Unlike in the prediction market model, rather than arriving multiple times, participants may only post a single answer; however, they may be strategic about when they post this answer. By waiting until later, a participant may be able to aggregate information from others’ answers, allowing her to post a better response. Unlike in the prediction market setting, participants cannot “garble” their information. However these are not essential differences as compared to the substitutes or complements cases of prediction markets, where in equilibrium participants do not want to garble or participate multiple times, but instead fully reveal at the time that is optimal for them (as early or as late as possible, respectively).
In the model of Jain et al. (2009), the asker of the question has a valuation function over subsets of the pieces of information. Jain et al. (2009) does not justify how such a valuation function may arise, but we can now justify this modeling decision because any decision problem faced by the asker gives rise to some such valuation function. In one case of Jain et al. (2009), the asker draws a uniform “stopping threshold” on [\mathcal{V}(\bot),\mathcal{V}(A_{i}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}\cdots\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}A_{n})] (using our notation), and selects as the “winning answer” the one whose information raises her value above this threshold. For instance, if the first two users to post are with signals , and then the third user posts with signal , and we have \mathcal{V}(A_{i}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}A_{j})<t\leq\mathcal{V}(A_{i}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}A_{j}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}A_{k}), then user is declared the winner.
From this model, the expected reward of a participant, which is an indicator for being declared the winner, is exactly proportional to the marginal value of her information to the information collected so far. Thus, substitutes imply that participants’ dominant strategy is to rush to participate as early as possible, while complements imply that it is dominant to wait as long as possible. This follows from diminishing (increasing) marginal value of information.
And indeed, Jain et al. (2009) identify substitutes and complements conditions on the information which are exactly diminishing and increasing marginal returns, with the main result as stated above. (The paper also considers several other methods of selecting the winner with more complex features, which we will avoid discussing here for simplicity.)
We would like to emphasize that, while the above discussion intentionally highlights the similarities between that work and this one, the authors do not provide any endogenous model of the information, e.g. whether it be probabilistic and if so how it is structured, nor of the utility of the asker of the question and how this utility might arise or be related to the structure of the information. Without such models, it does not justify why a structure might satisfy their substitutes or complements conditions (nor when/if one could expect the conditions to hold).
Our work provides answers to all of these questions. Information may be modeled as Bayesian signals and the asker may face any decision problem. This gives rise to a valuation function over signals that can capture the model in Jain et al. (2009). Furthermore, under this model, that papers’ substitutes and complements conditions used in Jain et al. (2009) are subsumed by those proposed here. Hence, we are able to bring this work under the same umbrella as prediction markets, the crowdsourcing contests discussed above, and the algorithmic and structural results to be discussed later. An example of substitutes for any of these problems is an example for all of them.
4 Algorithmic Applications
Here, we investigate the implications of informational substitutes and complements on the construction and existence of efficient algorithms for information acquisition. We first define a very general class of problems, SignalSelection, to model this problem, and show positive results corresponding to substitutes and negative results in general. These results are obtained by showing tight connections to maximization of (submodular) set functions. We also investigate an adaptive or online variant of the problem with similar results.
To focus on the problem of information acquisition, we abstract out the complexity of interacting with the decision problem and prior. This differentiates these results from prior work on information acquisition, but the overall approach – utilizing submodular set functions – is common (even pervasive) in the literature (see Krause and Golovin (2012)). So we view the contribution of these results as offering a unification or explanation for successful approaches in terms of informational substitutes.
4.1 The SignalSelection problem
Definition 4.1.1**.**
In the problem SignalSelection, one is given a decision problem and information structure on signals ; and also a family of feasible subsets of signals . The goal is to select an approximately optimal , i.e. to
[TABLE]
For instance, suppose that an agent has a budget constraint of and each piece of information has a price tag; how to select the set that maximizes utility subject to the budget constraint? (This is also known as a knapsack constraint.)
The SignalSelection problem is not yet well-defined because we have not described how the input is represented. In general, the decision problem may be hard to optimize, and the prior distribution may have support . Because we want to abstract out the complexity of SignalSelection independently of the difficulty of these problems, we will assume an efficiently-queryable input. The outline for our approach is as follows, pictured in Figure 4.
- •
For positive results, we require an input representation that allows efficient computation of for some subset of signals. In Section 4.1.1, we will discuss several such representations. The most natural of these we call the oracle model.
- •
For negative results, we will reduce a hard problem – maximization of arbitrary monotone set functions given a value oracle – to SignalSelection. The reduction will produce instances of SignalSelection having a very concise and tractable input representation: The signals will be independent uniformly random bits, the event will be the vector , and the decision problem will be immediately computable, just requiring transparent calls to the value oracle of the original maximization problem. This implies that any algorithm that can solve SignalSelection, under any “reasonable” model of input (particularly the oracle model and others we describe), can solve the instances produced by our reduction.
Note that an oracle-based approach is very general because, if we do have an instance where the input is concise and given explicitly, for example, the decision-problem optimizer is given as a small circuit, then we can just run our algorithms treating this input as an oracle, evaluating it when necessary.
We next discuss input representations for positive results, including describing the oracle model. We will then give our positive and negative results.
Monotone set function maximization.
Our results will involve relating complexity of SignalSelection to that of maximizing some where is a finite ground set. The fact that is increasing, i.e. more information always helps, implies that we restrict to monotone increasing : If , then . Recall that submodular correspond to substitutable items, while supermodular correspond to complements.
In set function maximization, the input is often given as a value oracle that, when given a subset , returns in one time step . When is submodular, it is known that polynomial-time constant-factor approximation algorithms exist for many types of constraints. For instance, there are efficient -approximation algorithms under the knapsack constraint described above (Sviridenko, 2004; Krause and Guestrin, 2005a) and more general matroid constraints (Calinescu et al., 2011). On the other hand, in general set function maximization is known to be difficult information-theoretically, requiring exponentially-many oracle queries to obtain a nontrivial approximation factor even when restricting to monotone supermodular functions; we give an example in Proposition 4.1.5.
4.1.1 The oracle model and computing
Here, we investigate the computation of , the value function. We restrict attention to evaluating at a set of signals, i.e. the join of a subset of signals. The reason is that this case is sufficient for our positive results and is most compelling for SignalSelection. Furthermore, no difficulty arises in how the input signal is represented, as it can always be given by a subset of .
We begin with a case where the decision problem is specified by an oracle, but the prior is given explicitly. Because may be exponentially large in , the number of signals, we will later introduce an oracle model for as well.
Proposition 4.1.1**.**
For any decision problem and set of signals , given an oracle for computing the associated convex , we can compute in time polynomial in , and . (This is the size of the problem in general, as the prior distribution ranges over this many outcomes .)
Proof.
We assume an oracle that computes for any distribution on . Note that , so this is equivalent to assuming an oracle for the utility of the optimal decision for a given distribution on .
We need to calculate
[TABLE]
Here, the expectation is over all realizations of the set of signals , and is the posterior distribution on conditioned on that set of realizations.
There are at most terms in the sum, and each posterior can be computed as follows:
[TABLE]
for each in the support of . This can be computed in time polynomial in the products of the support sizes. ∎
In general, the running time of Proposition 4.1.1 is exponential in , the number of signals. This is unavoidable in general as the input itself may be this large (the prior distribution ranges over exponentially many events). Thus, it is natural to suppose that the input is given as an oracle in some fashion, making the input succinct. We define a natural model, give the associated positive result, and discuss possible variants or weakenings.
Definition 4.1.2**.**
In the oracle model for representing a decision problem and prior , one is given:
An oracle computing the prior probability of any realization of any subset of signals. 2. 2.
Access to independent samples from the prior distribution. 3. 3.
An oracle computing, for a distribution on , the expected optimal utility obtainable given belief , namely .
Proposition 4.1.2**.**
In the oracle model, we can approximate to arbitrary (additive) accuracy with arbitrarily high probability in time polynomial in , , and .
Proof.
For any given , we would like to approximate
[TABLE]
up to an additive error with probability . We can abstract this problem as computing , with is distributed as where is drawn from the prior. Letting over all , we can apply a standard Hoeffding bound: The average of i.i.d. realizations of is within of the true average with probability as long as exceeds .
To see that we can in fact sample : We simply draw one sample from the prior, giving us . We compute the posterior conditioned on this sample as follows: For each outcome of , we have
[TABLE]
This requires two calls to the prior computation oracle; then a call to the oracle for completes the calculation. The running time analysis simply observes that each signal realization requires only bits to represent, and there are of them; similarly for outcomes of . ∎
One would like to make weaker assumptions. However, dropping either the assumption of independent samples or the oracle seems problematic. Without samples, evaluating seems difficult because this is a sum over exponentially many terms, and we cannot a priori guess which terms are “large” or where to query the oracle for the prior.
With independent samples but no oracle for marginal probabilities, naïve approaches break down because of the difficulty of accurately estimating conditional probabilities . For instance, it may be that each outcome is very unlikely, so that one cannot draw enough samples to accurately estimate the desired conditional probability using the ratio . It is also of note that any distribution over the possible outcomes of the signals and of , including the prior itself, in general has size , which is exponential in . So one must avoid writing down such distributions; and any small sketch seems to quickly lose accuracy in estimating the conditional probability, which is computed from probabilities on events (and again, these probabilities may all be exponentially small even while conditional probabilities are large).
We give two further examples of how to overcome this difficulty. The first is to assume that the prior is tractable in some way; in our case, sparse. The second is to push the difficulties mentioned into an oracle of a different sort.
Proposition 4.1.3**.**
Suppose we are given access to an oracle for explicitly given the prior ; and suppose that the prior is sparse, supported on possible outcomes . Then we can compute in time polynomial in and .
Proof.
We can now assume that the prior is explicitly given as part of the input. The expectation in the definition of is now a sum that ranges over only at most terms. For each term, corresponding to some subset , we can efficiently look up , as well as (for each ) , allowing us to compute the conditional probability . (Recall that our notation is simply the vector of these probabilities, ranging over outcomes of .) These are all the ingredients we need to evaluate each term in the sum, which is . ∎
Proposition 4.1.4**.**
Suppose we are given access to the following:
A decisionmaking oracle that, given a subset of signal realizations, returns an optimal decision ; and 2. 2.
an oracle for evaluating the utility of decision when nature’s event ; and 3. 3.
access to independent samples of from the prior distribution.
Suppose that is bounded. Then we can approximate to arbitrary (additive) accuracy with arbitrarily high probability in time polynomial in and and .
Proof.
Again, given , we wish to approximate , which is equal to
[TABLE]
where is the optimal decision conditioned on observing signals . Again, assuming that lies in a bounded range of size , the same Hoeffding bound applies: By drawing i.i.d. samples from the prior and then calling the oracles to obtain and , we can approximate this expectation to arbitrary additive error with arbitrarily high probability, for a suitable (polynomial-sized) . ∎
This range of results gives some evidence that in general, if the decision problem has some sort of succinct representation outside of the oracle model, then we may still hope to compute . In fact, the problem we will construct for our negative result, Theorem 4.1.2, will fit the model of Proposition 4.1.4, where it is trivial to find and evaluate the optimal decision.
4.1.2 Positive and negative results via reductions
Positive results.
As a corollary of the above reductions, we are able to reduce SignalSelection to set function maximization, netting positive results especially in the case of substitutes.
Theorem 4.1.1**.**
If signals are weak substitutes, there are polynomial-time approximations for SignalSelection under a variety of constraint families in the oracle model or with succinct input as in Propositions 4.1.3 or 4.1.4. These include cardinality constraints (select at most signals), budget constraints (given a price associated with each signal, select a subset with total price at most ), and matroid constraints.
Proof.
Given an instance of SignalSelection with signals , we construct a monotone increasing set function via . If signals are weak substitutes, then is submodular on the signal lattice, and hence is submodular (as well as monotone). We can therefore apply known algorithms for submodular maximization, in particular, Nemhauser et al. (1978); Sviridenko (2004); Calinescu et al. (2011). Now, there is a subtlety: Under some of the models we propose, particularly the oracle model, we do not compute exactly but instead can only guarantee arbitrarily high accuracy with arbitrarily high probability. However, it is well known (e.g. Kempe et al. (2003)) and proven (e.g. Krause and Guestrin (2005a)), that these algorithms still give guarantees when we have a high-probability, high-accuracy guarantee on evaluations of . The key point is that we can still evaluate, with arbitrary accuracy, the gradient or marginal contributions of each element777Recent work (in preparation) considers the question of how much accuracy in evaluating is required, showing negative results when (roughly) accuracy is worse than ; but under the oracle model we can evaluate with error an arbitrary inverse polynomial with only an exponentially-small probability of error (simply via a Hoeffding bound), in which regime it is well-known that this problem does not arise.. ∎
Discussion of approximate oracles.
One would like a robustness guarantee of the following sort: Even if we do not have an oracle that exactly optimizes the decision problem, suppose we do have an oracle returning, say, a decision whose expected utility is within a constant factor of optimal. Then give a good algorithm for SignalSelection in the substitutes case, with an appropriately-decreased guarantee.
Unfortunately, it seems that this kind of result is unlikely without deeper investigation and further work. To see the challenge, imagine that an adversary is allowed to design the oracle subject to a constraint of some approximation ratio. Then our problem essentially reduces to submodular maximization with noisy or approximate value oracles, where the noise may be adversarially chosen subject to this approximation constraint. Unfortunately, recent work on these kinds of problems have shown them to be difficult in general (Singer and Vondrák, 2015; Balkanski et al., 2015; Hassidim and Singer, 2016). This seems like a very difficult barrier, but perhaps future work can leverage the structure of decision problems in some way to make progress in this direction.
Negative results.
We show that in general, SignalSelection is as difficult as optimizing general monotone set functions subject to constraints. In fact, this holds even for an easy special case of SignalSelection where all signals are independent uniformly-random bits and the decision problem is trivial to optimize (the solution is essentially to list all of the outcomes of signals you have observed).
To do so, we give a reduction in the opposite direction: Given , we construct a decision problem and prior distribution such that . Hence, any algorithm for SignalSelection gives an algorithm for optimizing . In terms of input representation, while may require exponential space to represent explicitly, our reduction is essentially as useful as one could hope for in this respect, creating a trivial wrapper around a value oracle for .
It seems that any reasonable algorithm one might propose for optimally selecting sets of signals should be able to handle such a tractable input. Hence, this is a strong negative result that, in general, optimization over signal sets is just as hard as over item sets.
Theorem 4.1.2**.**
For every monotone increasing set function , with , there exists a decision problem and a set of signals such that for all .
Furthermore, in the construction, it is trivial to compute and represent any posterior conditioned on any realizations of any subset of signals; and trivial (given a single evaluation of ) to compute the expected optimal utility .
Proof.
Each signal will be a uniform independent bit, and the event of nature will consist of the vector of realized s (a binary string of length ). Intuitively, the idea is that having observed , regardless of whether its realization is [math] or , corresponds to having item in the set , while not having observed (hence having a uniform belief over ) corresponds to .
The decision problem will look like a scoring rule, but it will be for predicting the mean of rather than predicting a probability distribution over . This is good news in terms of the representation size: predicting the mean of requires only reporting a vector in , while a general probability distribution over all outcomes of has support size up to .
Formally, it turns out that the scoring rule characterization applies equally well to constructing proper rules for predicting the mean of a random variable such as . Specifically, given any convex function , one can construct a scoring rule . Using the notation , we have the key scoring rule property that is maximized at , where it equals . A quick proof: Given , define where is a subgradient at . We have as desired. Now, note that , where is the Bregman divergence of ; since Bregman divergences are nonnegative, this proves that is maximized at .
Hence, the roadmap is as follows.
Construct a function . 2. 2.
Verify that is convex. This implies that there is a decision problem (namely, predicting the mean of ) where the expected utility for predicting when one’s true expectation is equals . 3. 3.
Verify that, for any subset of and for any set of realizations , we have . 4. 4.
Note this implies that, for any , we have . 5. 5.
Check that, given a value oracle for , we can efficiently compute any quantities of interest (the posterior distribution on , the posterior expectation of , , ).
(1)
The construction of is recursive on the dimension . It is ugly, being discontinuous at its boundary. However, drawing some pictures should convince the reader that can be “smoothed” to a more reasonable, continuous convex function. For a base case of , on we let on the interior where , and . (That is, evaluated at the set consisting of the item.) Note that, because is monotone increasing, i.e. , is convex. The discontinuity implies that, for the associated proper scoring rule for the mean , we must have whenever lies at an endpoint and is in the interior. But again, any smoothing of to be continuous will remove this property.
On , we let for in the interior of the hypercube. That is, is constant on its interior. Furthermore, we have , where is the prior, hence .
Now we define on its boundary. Consider any face of the -dimensional hypercube. Each face corresponds to a particular setting of some , either to [math] or , by the coordinate whose value is constant on that face. For instance, corresponds to the face consisting of the set of where ’s th coordinate equals one. On both of the faces corresponding to , is defined to be , where is with the th coordinate removed, and the function is defined recursively as follows. Consider the set function with . Then let be the result of our construction applied to . This completes the definition of .
Verbally, for each face of the hypercube, we have fixed some to be in the set passed to , and considered the resulting submodular function on the remainder of . The value of on the interior of that face will be , by the recursive construction. To picture and convince ourselves that it is well-defined, consider the intersection of the faces corresponding to, say, and . This is a lower-dimensional face consisting of all points on the hypercube whose th bit equals and th bit equals [math]. On the interior of this face (i.e. no other bits are equal to [math] or ), has value . And so on all the way “out” to the corners of the hypercube, where ; at all of these, .
(2)
We prove is convex on the hypercube by induction on , with the base case already observed above. For the inductive step, note again the key point: by monotonicity of , if is on the boundary of the hypercube and is in the interior then . Consider any two points , and break into cases. If both points lie in the interior, then because is constant there, for any . If one point lies in the interior and one on the boundary, then any convex combination of the two lies in the interior. is constant in the interior and weakly larger on the boundary, so the convexity inequality is satisfied. If the points lie in different faces, then again any convex combination lies in the interior. Finally, if the points lie in the same face, then coincides with some on and , and is convex by inductive hypothesis.
(3)
We now verify that , when applied to the expected value of given the realizations of signals corresponding to , is equal to . Consider any set of realizations and let . If , then by construction and . Otherwise, is the vector where each entry is equal to if and otherwise; this follows from the i.i.d. distribution of the . Hence, lies in the interior of the (low-dimensional) face of the hypercube corresponding to the realizations , hence . Here the notation is the function obtained from by fixing ; is the corresponding recursively constructed function on ; and is obtained by removing all coordinates.
(4)
Use the notation . Since step (3) holds for all realizations of a given set of signals, in particular (where is the scoring rule corresponding to ):
[TABLE]
(5)
Given any set of signals , one can immediately compute by a call to the value oracle for . Given their realizations , the posterior distribution is simply uniform on those with (and of course has each with probability one for ). This induces the posterior distribution over (which can thus be concisely represented, even though there are possible outcomes in general), as well as the posterior expectation of . Evaluating at an arbitrary point can be done quickly: Let be the subset of coordinates on which is equal to either [math] or ; then , as lies on the interior of a face corresponding to . This requires just a single call to the oracle for . Evaluating can thus be done in time polynomial in as well; the only additional step required is picking a subgradient of at each point , and there is only one choice at each point (as noted above, unless is smoothed, this construction does require if lies on a face and in the interior). Finally, the optimal decision for a given expectation of is just that expectation. ∎
Explicit negative results.
First, our reduction has implications even for the substitutes case. Monotone submodular maximization to a better factor than , even under a simple cardinality constraint, requires an exponential number of value-oracle calls (Nemhauser and Wolsey, 1978) and, even given a concise explicit input, is NP-hard (Feige, 1998). Because our reduction preserves marginal values, a submodular set function reduces via Theorem 4.1.2 to weak substitutes. This implies that our approximation cannot be improved without a stronger assumption than substitutability (or e.g. a polynomial-time algorithm for SAT).
Corollary 4.1.1**.**
Even when signals are weak substitutes with a cardinality constraint, achieving a strictly better approximation than to SignalSelection requires an exponential number of oracle queries.
Without the substitutes/submodularity condition, it is well known that maximizing general monotone set functions is difficult given only a value oracle, although such negative results are not always explicit in the literature. For instance and for concreteness, one can even restrict to monotone supermodular functions and the simple problem of maximizing subject to a cardinality constraint (select any set of at most elements, for some given ). In this case, there does not seem to be a negative result in the literature despite this hardness being well-known (see (cstheory.se, 2016)), so we give an explicit one here to illustrate the challenge.
Proposition 4.1.5**.**
For any algorithm for monotone supermodular maximization subject to a cardinality constraint, if the algorithm makes a subexponential number of value queries, then it cannot have a nonzero approximation ratio.
Proof.
We construct a simple family of supermodular functions. Let be the cardinality constraint, i.e. maximum cardinality of any feasible set. Let if and otherwise let . Pick one special set of size , uniformly at random from such sets, and let . Hence the optimal solution to the problem is to pick with solution value .
This function is supermodular: Given any element , the marginal contribution of to is [math] if , then either or [math] for , then either or for , then for . Because this marginal contribution is increasing, is supermodular. Now any algorithm making subexponentially (in ) many queries cannot, except with vanishing probability, guess which set of size is , so it will with probability tending to select some other set of size , which has solution value [math]. ∎
Corollary 4.1.2**.**
No algorithm for SignalSelection accepting the oracle model of inputs, or any of the other models discussed, can guarantee a nonzero approximation in general even for cardinality constraints and even if signals are assumed to be weak complements, unless it makes an exponential number of queries to the oracles.
4.1.3 Adaptive SignalSelection
We now define an adaptive version of the problem and show that substitutability implies positive results here as well. This problem has already been studied in very similar settings with a very similar approach by Asadpour et al. (2008); Golovin and Krause (2011). There is a significant difference in that these works aimed to maximize a set function over items in tandem with observations about those items. Our model is a significant generalization in that it considers arbitrary kinds of observations and an arbitrary decision problem. However, the goal of this section is not to claim originality or generality of solution, but only to demonstrate that this problem can also be viewed through the lens of substitutability.
Definition 4.1.3**.**
In Adaptive SignalSelection, one is given a decision problem and information structure on signals ; and also a family of feasible subsets of signals . An algorithm for this problem is a policy that first selects a signal to inspect; then observes the realization of ; then depending on that realization, selects a signal to inspect, observing the realization of , and so on. The algorithm must guarantee that the total set of signals selected is in . After ceasing all inspections, the algorithm outputs some decision . The goal is to maximize over the distribution of signals and as well as any randomness in the algorithm.
We note that the definition of SignalSelection abstracted out the process of deciding an action given the realizations of the signals, as we assumed that it was known how to optimize the utility function. Here, it seems cleaner to integrate the choice of the decision with the algorithm; but this is not the fundamental difference between the two settings. The fundamental difference is adaptivity: In Adaptive SignalSelection, the algorithm may observe the outcome of the first selected signal before deciding which signal to select next, and so on.
Here we need a somewhat stronger substitutes condition.
Definition 4.1.4**.**
A set of signals on a subsets lattice are pointwise weak substitutes if, for each and in , and for each of the outcomes in the support of the prior, letting be the posterior distributions conditioned on and respectively,
[TABLE]
In other words, the left side is the marginal value of given the observation , and the right is its marginal value given observation .
Verbally, signals are pointwise substitutes if having observed more information never increases the expected marginal value of a signal. The key difference is that for “regular” substitutes, the condition can be written as follows:
[TABLE]
In other words, signals are substitutes if expecting to observe more information never increases the expected marginal value; they are pointwise substitutes if observing more information never increases expected marginal value.
Our goal here is to illustrate that techniques from submodular maximization extend naturally. We focus on the simple cardinality constraint case and show that the greedy algorithm for monotone submodular maximization (Nemhauser et al., 1978) also gives good guarantees in this adaptive setting, when we have a strong substitutes condition.
Theorem 4.1.3**.**
When signals are pointwise weak substitutes, Algorithm 1 (Adaptive Greedy) obtains a -approximation algorithm for Adaptive SignalSelection under cardinality constraints in the oracle model.
Proof.
The usual proof essentially goes through. We utilize notation from Algorithm 1 (adaptive greedy). The expected performance of , the set constructed at time , is (we use this notation as shorthand for evaluation at the join of all signals in ).
Let be the expected performance of the optimal policy. We will upper-bound for each by the performance of a hypothetical algorithm that first selects , then selects all of the signals selected by the optimal policy. For a fixed , such a hypothetical algorithm can obtain, in expectation, at most the performance of plus times the maximum expected marginal contribution of any signal to . This follows from pointwise substitutes: as the algorithm acquires more information, the expected marginal contribution of a signal does not increase. So, taking the expectation over this condition, which holds for each realization of :
[TABLE]
This is exactly the inequality that produces the approximation ratio in submodular set function case. The inequality implies by induction that , and this approximation ratio is always better than . ∎
5 Structure and Design
In this section, we give some initial investigations into the structure of substitutes and complements. We focus on two kinds of questions: Understanding what types of information structures may be generally substitutes or complements for a broad class of decision problems; and understanding how to design a proper scoring rule or other decision problem so as to impose substitutability on a given signal structure.
In terms of contributions of this paper to these problems and starting points for future work, it is also worth recalling the relatively diverse set of equivalent definitions of informational S&C derived in Section 2.4. There, we showed that substitutes can be defined in terms of submodularity, generalized entropies, or divergences.
5.1 Universal substitutes and complements
An ideal starting point would be characterizing information structures that are always substitutable or complementary; we term these “universal”. We note that Börgers et al. (2013) investigated a two-signal variant of this problem, with a definition essentially corresponding to weak substitutes on those two signals, with some results of the same flavor.
Intuitively, there are “trivial” cases that satisfy this universality criterion, at least for weak S&C. An example of trivial weak substitutes is the case where . Here, after observing any one signal, all of the others do not change the posterior belief at all. An example of trivial weak complements is the case where each is an independent uniformly random bit and , the XOR of all the bits. Here, any subset of signals does not change the posterior belief at all compared to the prior, but the final signal completely determines .
Definition 5.1.1**.**
Given an information structure with prior , we term universal weak substitutes if they are weak substitutes for every decision problem. Universal moderate/strong substitutes and weak/moderate/strong complements are defined analogously. We term the signals trivial substitutes if for every realization of in the prior’s support, for all and trivial complements if for all realizations in the support, for all . We term them somewhat trivial if the prior is a mixture distribution that is equal to a trivial structure with some probability, and some other arbitrary other structure with the remaining probability.
Proposition 5.1.1**.**
If are universal weak substitutes, then they are somewhat trivial. Furthermore, their “trivial” component is more informative than the nontrivial component, in the following sense. Let be the convex hull of , and let be the convex hull of . If are universal substitutes, then for all .
Proof.
Consider any information structure that does have for all . We give a decision problem for which it sometimes satisfies strictly increasing marginal value. To do so, by Corollary 2.3.1, it is enough to construct a convex function with appropriate structure, for which we then know a decision problem (namely, a scoring rule, Fact 2.3.1) exists. The idea is pictured in Figure 5(c).
Let be the convex hull of , that is, of all possible posterior beliefs conditioned on one signal. Let be zero for in the convex hull of that set of beliefs (note that the prior must be in the convex hull) and be increasing outside of this convex hull. Then for any one signal , we must have as . But by assumption, there exist posterior beliefs for multiple signals that fall outside this convex hull. This follows because each for any , so unless for all , there are cases where some posterior belief falls outside the convex hull mentioned. (And if always, then repeat the argument for triples of signals , and so on; by nontriviality, the argument will succeed at some point.)
Now for these posterior beliefs falling outside the convex hull, they occur with positive probability and have a positive value of , hence the marginal value of additional signals because strictly positive at some point, while the marginal value of the first signal was [math].
To see that this implies a mixture containing trivial substitutes, note that the convex hulls for all , so in particular the corners of these convex hulls must be points where is equal, in the case where , to every posterior belief conditioned on any number of signals. ∎
Theorem 5.1.1**.**
All universal moderate substitutes are trivial. (Hence, the same holds for universal strong substitutes.)
Proof.
Any universal moderate substitutes must be universal weak substitutes as well; so we know via the proof of Proposition 5.1.1 that any candidates must have a mixture with trivial substitutes. However, we can let be a signal determining whether that component of the mixture has occurred. Then one implication of moderate substitutes is that, conditioned on , all signals are weak substitutes. Therefore, universal substitutes implies that even conditioned on , the prior is a mixture containing a trivial component. Repeating the argument gives that the entire prior must consist only of trivial components. ∎
Proposition 5.1.2**.**
Consider a binary event prior probability , and signals with posterior probabilities , and so forth. Suppose that:
* and ; and* 2. 2.
.
Then are universal moderate complements.
Proof.
The idea is that posterior beliefs on multiple signals usually tend to lie “farther” from the prior than those on single signals, because at least in some cases, more information leads to more certain (hence extreme) beliefs. Convex functions already tend to give larger marginal value to more extreme cases. Hence, convexity of the decision problem encourages complementarity; and it cannot discourage complementarity very much because a convex function can only be “so flat”. This is pictured in Figure 5. By moving the posteriors all very far from the prior, compared to the posteriors , we get increasing marginal returns.
Pick any convex on and let all probability distributions on be represented as scalars . This includes the prior and posteriors such as . It suffices to show that . Let be the minimizer of on . Then . Now we claim that . This implies , and after taking the analogous case reversing and and expectations, we get that the desired inequality must hold. To prove the claim, consider the case . Then , and we want to show . This holds by definition of a subgradient as the right side is at most the slope of a line connecting and . The analogous argument holds for the case . ∎
Corollary 5.1.1**.**
An example of universal complements are the signals each independently equal to with probability and (the XOR operation).
We note that these may be universal complements for larger ranges of as well, and a very interesting question for future work is to characterize the set of universal complements. This seems to require more work particularly for with a larger number of outcomes. For binary which are equal to a deterministic function of , it seems possible to relate complementarity to the sensitivity of a function in Boolean analysis.
5.2 Identifying complements
Our main result here is to identify the following very broad class of complements.
Proposition 5.2.1**.**
Independent signals are strong complements in any decision problem where has a jointly convex Bregman divergence .
Proof.
is termed jointly convex if it is a convex function on the domain (as opposed to the case where it is convex in each argument separately, for instance). Assume signals are independent; consider any on the subsets signal lattice (incomparable to each other) and any on the continuous lattice with . By Lemma 2.4.1, showing strong substitutes is equivalent to showing that
[TABLE]
Use independence and rewrite to this convenient form:
[TABLE]
Note that if were not independent of , then the inner expectations would require conditioning the distributions of and on the outcome of . Now, it suffices to prove the following for all :
[TABLE]
Now, since is jointly convex, Jensen’s inequality will imply this fact if we can just show that and . We prove the first equality; the second is exactly analogous but easier.
[TABLE]
We used Bayes rule and the law of total probability, then eventually used the fact that (because and are independent conditioned on ) and that and are independent of . ∎
A corollary is that independent signals are complements for the scoring rule and the quadratic scoring rule, as their divergences (-divergence and distance squared) are jointly convex.
On the other hand, some form of this restriction on the decision problem is needed:
Claim 5.2.1**.**
If is not convex in its second argument, then independent signals are not necessarily complements.
Proof.
We consider a binary event and where is a probability that . A counterexample is to let for and for larger . Consider any decision problem associated with this (for instance, predicting against a proper scoring rule derived from ). The two signals and are independent uniform random bits, and is equal to the binary OR of the bits. In this case, one can check that , but and \mathcal{V}(A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B)=\frac{3}{16}. Thus in particular \mathcal{V}(A)+\mathcal{V}(B)\geq\mathcal{V}(A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B)+\mathcal{V}(A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B), so they are not complements. Note that may be modified to be strictly convex while preserving the counterexample. Also note that the sharp “kink” in the graph of at forces to be non-convex in its second argument (one can take the first argument to be the prior on , , choosing the subgradient ). ∎
5.3 Identifying substitutes
It was previously known (Chen et al., 2010) for prediction markets under the scoring rule that conditionally independent signals induced players to reveal their information as early as possible. The result in a different guise was shown in an algorithmic context, namely, that for conditionally independent signals, . Both of these turn out to correspond to the fact that signals are always substitutable in the context of the scoring rule:
Theorem 5.3.1**.**
For the scoring rule, signals that are conditionally independent upon the event are strong substitutes.
Proof.
Suppose signals are conditionally independent on . Let on the subsets signal lattice and on the continuous lattice with A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B\preceq A^{\prime}\preceq A. We are to show that \mathcal{V}(A^{\prime}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B)-\mathcal{V}(A^{\prime})\geq\mathcal{V}(A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B)-\mathcal{V}(A). Recall that, using the entropy characterization, by the properties of the entropy function.
[TABLE]
Similarly, \mathcal{V}(A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B)-\mathcal{V}(A)=H(B|E,A)-H(B|A). Now because A^{\prime}\succeq A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B, by conditional independence, . Meanwhile, by conditional independence, , which completes the proof. ∎
However, this fact is apparently quite special to the scoring rule.
Proposition 5.3.1**.**
Conditionally independent signals are not necessarily substitutes for the quadratic scoring rule (which has a jointly convex Bregman divergence).
Proof.
The quadratic scoring rule has expected score function , and therefore . Let be binary with and let each of be i.i.d. conditioned on , each equal to with probability and equal to the bit-flip of otherwise.
One can check that for cases where is large compared to , for instance and , we have \mathcal{V}(A)+\mathcal{V}(B)\leq\mathcal{V}(A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}B)+\mathcal{V}(A\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\wedge}}}}B) (recall that ). Hence substitutability is strongly violated. For instance, an agent observing would prefer to report second in a prediction market after an agent observing , even when both agents are constrained to report truthfully.
Intuitively, what happens in this information structure is that neither the realization of nor of on its own is enough to change the rather strong prior on . However, sometimes, observing both and (if they are both [math]) can cause a large change in beliefs about , which means that observing both can sometimes be very valuable. ∎
5.4 Designing to create substitutability
We now briefly consider the question of designing a decision problem or scoring rule so as to enforce substitutability of information. In a game-theoretic setting such as prediction markets, one would like to design mechanisms where information is aggregated quickly; as we have seen, this is essentially equivalent to making information more substitutable.
In an algorithmic setting, the decision problem with which one is faced may be difficult to optimize due to non-substitutability of information. One would like to construct a “surrogate” decision problem for which the information at hand is substitutable, then use algorithms for (Adaptive) SignalSelection to approximately maximize that surrogate, with some guarantee for the original problem. This is the approach of submodular surrogates in the literature (Chen et al., 2015). We do not directly consider this problem in this paper, but we hope that these techniques may yield insights or tools that are useful in these problems for future work.
An important conclusion of this paper is that substitutability, or lack thereof, tends to arise from a combination of two factors:
The distances between beliefs due to the information structure. If updating on additional information tends to spread beliefs farther apart, then that information tends to be complementary. If it tends to make smaller changes in beliefs, it tends to be substitutable. 2. 2.
The curvature of the expected utility function associated to the decision problem. Highly-curved regions correspond to high marginal value of information to beliefs in those regions.
We illustrate some of this intuition, with an eye toward design of substitute-encouraging , in Figure 6. By designing a that is highly curved close to the prior, then gradually less curved farther from the prior, one increases marginal value of information near the prior and decreases it (relatively) further away; this increases substitutability.
A formalization of the problem.
Given an information structure consisting of a prior and signals , (when) can we construct a decision problem such that the signals are substitutes? Noting that a trivial decision problem, e.g. with one action, technically satisfies substitutes, we will seek decision problems that satisfy a “nontriviality property”.
Definition 5.4.1**.**
Substitutes are somewhat strict if the marginal value is always diminishing on the corresponding lattice, and sometimes strictly diminishing. Analogously, complements are somewhat strict if marginal value is always increasing and sometimes strictly increasing.
Unfortunately, our results on universal S&C in Section 5.1 essentially imply that this is not always achievable.
Proposition 5.4.1**.**
For some information structures, it not possible to design a decision problem giving rise to weak, somewhat strict substitutes; the same holds for complements. However, if the information structure does not contain a mixture of trivial substitutes, then it is possible to design a decision problem under which signals are weak, somewhat strict complements.
Proof.
We showed in Corollary 5.1.1 that nontrivial universal complements exist; for these structures, for every decision problem, they are weak complements, which implies that they cannot be somewhat strict substitutes.
Meanwhile, in Proposition 5.1.1, we showed that unless the information structure contained a mixture with trivial substitutes, one can construct a decision problem satisfying weakly and sometimes strictly increasing marginal value. ∎
A crucial direction for future work is to better characterize for what information structures it is always possible to design for substitutability.
.
6 Discussion, Conclusion, and Future Work
6.1 Contributions and discussion
This paper makes two types of contributions. One type is to propose definitions of informational substitutes and complements and provide evidence that these definitions are natural, useful, and tractable. The other type is to prove concrete results for game-theoretic and algorithmic problems. These two contributions are interrelated: The concrete results rely on the definitions and characterizations of S&C, while evidence in favor of the definitions comes from their utility in proving the concrete results.
We begin with a summary of the contributions in terms of concrete results, then summarize the evidence in favor of our proposed definitions of informational S&C. We then discuss future work in a variety of directions.
Contributions independent of S&C
The first application in this paper was to identify essentially necessary and sufficient conditions for the two types of prediction market equilibria that have been most studied: The “good” case where all traders rush to reveal information as soon as possible, and the “bad” case where all traders delay all revelation as long as possible. This result broadly generalizes previously-known special cases Chen et al. (2010); Gao et al. (2013). These conditions corresponded respectively to our definitions of informational substitutes and complements. Our other results regarding these definitions have implications and applications for the market setting. We gave some tools and approaches for identifying and designing substitutability, which (we showed) corresponds to the main goal of a market designer: encouraging immediate information revelation.
We also gave some additional game-theoretic applications in other settings involving strategic information revelation and aggregation. We hope that future work can explore more connections.
The second main application was to the algorithmic problem of information acquisition, which we formalized as the SignalSelection problem. This is a very natural and general problem, capturing any sort of information acquisition scenario. We showed that substitutes imply efficient approximation algorithms via submodular maximization, offering a unifying lens or perspective on a variety of literature utilizing this approach. We also saw matching hardness in general when substitutability is not present. These results give hope that substitutable structure may be leveraged in future algorithmic investigations of related problems.
Evidence for informational S&C
Informational substitutes require much more background and work to define than their counterparts for goods, as discussed in Section B. This leads to many ways in which the definition can “get it wrong”. However, there are several reasons to believe that the definitions proposed in this paper are substantially on the right track. The three goals we address are showing that they are natural, useful, and tractable. The evidence for these is summarized below; however, there is room for further investigation and discussion on the subtleties of the proposed definitions. As additional future applications are investigated, it may be found that tweaks in the definition improve it along these axes.
Natural.
The following evidence suggests that the definitions of informational S&C presented in this paper are natural.
- •
They are defined in terms of sub- and super-modular lattice functions, sharing this property with widely accepted definitions of S&C for items. The weak, moderate, and strong versions of the definitions correspond to lattices have natural interpretations both game-theoretically – strategies that are binary, deterministic, or randomized – and algorithmically, optimizing over subsets, deterministic “poolings”, or randomized “garblings”.
- •
They can be alternatively characterized information-theoretically, by diminishing (increasing) amount of information revealed by a signal.
- •
They can be alternatively characterized geometrically, by the distance a belief is moved by a posterior update on one signal given the other.
- •
S&C respectively characterize “all-rush” and “all-delay” equilibria in prediction markets. Hence, our definitions (or very close variants) seem unavoidable when studying equilibria of these basic models of strategic play.
- •
The algorithmic complexity mirrors the now-familiar story in the case of items: approximately optimizing over substitutes can be done efficiently (at least in the case of weak substitutes), while for the general case or complements the problem is difficult.
Useful.
The usefulness of the definitions, so far, is reflected by the results summarized above. In particular, they allowed resolving an open problem on strategic aggregation in prediction markets that was previously solved only for particular cases.
Tractable.
Our definition of informational S&C refers to a very general setting of decision problems and information structures. It is not initially clear that a definition that broadly captures S&C can also be amenable to analysis or intuition. This particularly presents a challenge for our definitions because they depend on both the decision problem and the information structure, each of which can be a very complicated object.
We presented a convexity-based approach, namely, studying the convex expected-utility function , and showed that it both captures geometric intuitions for understanding S&C and gives useful analytic tools. One of these is prediction and the theory of proper scoring rules, which gives a way to construct a decision problem from any such convex . We used these tools to give some example broad classes of informational S&C as well as positive and negative results on designing decision problems to encourage substitutability or complementarity. We also gave intuitive definitions of informational S&C from the perspectives of information theory (generalized entropies) and geometry (divergences). These found some formal applications already in this paper, e.g. a convexity condition on the divergence definition of substitutes implies that all independent signals are substitutes.
6.2 Future work: game theory
An immediate direction is to extend our results to broader models of financial markets. We believe that analogous results are likely to hold. It would also be ideal, albeit esoteric, to understand the distinction between when Bayes-Nash equilibria and perfect Bayesian equilibria exist or are essentially unique. In a Bayes-Nash equilibrium that is not perfect Bayesian, some trader essentially makes a “threat” that is “not credible”. Can such scenarios exist when signals are substitutes or complements?
Many broader questions about S&C have a direct implication on markets via our results, and we outline some of these directions below.
For more general Bayesian games, the implications of our results are not yet clear. With multiple players, for instance, it is no longer true that more information always helps. It may be most tractable to look at special cases such as auctions or signaling games, although those seem to have a significant component of strategic substitutes as well. One natural direction is common-value auctions, where in one case (Milgrom and Weber, 1982) informational substitutes have been explicitly used.
6.3 Future work: algorithms
There are many potential algorithmic questions raised by these definitions. One is to consider existing problems, such as algorithmic Bayesian persuasion (Dughmi and Xu, 2016) or signalling in auctions, through the lens of substitutability. However, such problems are already significantly more complex because of the interaction between the players, which is largely absent or abstracted out so far in this work. We hope that further investigation will uncover the connections to such settings. Meanwhile, one might ask whether, for instance, variants of SignalSelection on the discrete or continuous lattice are well-motivated in any settings and, if so, whether they are tractable.
More specifically and technically, one might ask whether the oracle model presented in this paper the best way to represent a decision problem, or if there is some other natural alternative beyond those we presented. Perhaps it is possible to obtain positive results with weaker or substantially different assumptions on the oracles.
Another group of algorithmic questions relates to better understanding the definitions themselves. One concrete question is the following: Given a decision problem, perhaps in the oracle model (or some other well-justified model), determine whether signals are substitutes, complements, or neither. Or more strongly, given a decision problem and two signals , find
[TABLE]
An algorithm for this problem and multi-signal generalizations would help identify substitutes or complements, and would more generally solve e.g. the problem of how to trade in general prediction markets where signals are neither substitutes nor complements, but something in between. Many more possible algorithmic questions are likely to arise as investigation continues.
6.4 Future work: structure of S&C
A straightforward direction for future work is to identify further classes of substitutes and complements, including natural classes of decision problems and information structures that together produce S&C. This question also bears directly on the design of prediction markets.
The second straightforward direction is how to “design for substitutability”. We believe that our results give a good start in this direction, but many questions remain. For instance, can we characterize all universal substitutes and complements? (This characterizes cases where we cannot design for substitutability or complementarity.) And, in cases where we can make signals strict substitutes, can we design some natural and computationally efficient method for doing so? We hope that the geometric intuitions in this paper provide a starting point for addressing such questions.
Finally, one could be interested in exploring variants of the definitions taking the same approach but with some piece of the puzzle substantially altered. Concretely, one could imagine a very risk-averse agent with an adversarial view of nature and some more combinatorial representation of information. Can one formulate analogous definitions in cases like these?
Appendix A Other Related Work
We first survey related work on substitutes and complements for information (and in general). Then, we discuss work relating to information aggregation in markets and other game-theoretic settings. Finally, we describe algorithmic work on information acquisition (particularly in “submodular” settings).
A.0.1 Substitutes and complements
The notion that pieces of information may exhibit substitutable or complementary features is certainly not a new intuition; but up until this work, it seems to have remained mainly an intuition. There seem to be few attempts at formalizing a general definition or even special cases. The only work we know of in this direction is Börgers et al. [2013], which inspires our approach but also has significant drawbacks and limitations. We extensively discuss Börgers et al. [2013] in in Section B, where we contrast it with our definitions.
Two works in (algorithmic) game theory touching on informational S&C are Jain et al. [2009] and Milgrom and Weber [1982]; however, these do not propose general definitions. They are discussed below.
Somewhat related is the game-theoretic notion of strategic substitutes and complements [Bulow et al., 1985]. Roughly, these concepts refer to cases where a change in action by one player in a game results in a response from another that is similar to the first player’s (in the case of complements) or offsetting (in the case of substitutes). This notion seems relatively unrelated to our definitions of informational S&C. For one, our definitions focus on the case of a single decisionmaker or single optimization problem. Also, strategic S&C can be defined in complete-information games, where there are no signals or information of any kind. However, perhaps future work can discover classes of games in which the notions are more closely related.
Valuations for items.
In contrast to the lack of literature on informational S&C, in the case of valuation functions for items, substitutability and complementarity have been put on firm formal foundations, with strong connections between substitutes and existence of equilibria in markets for goods or matching markets [Kelso Jr and Crawford, 1982, Roth, 1984, Gul and Stacchetti, 1999, Hatfield and Milgrom, 2005, Ostrovsky, 2008].
In computer science, the literature on optimization has produced strong positive results leveraging substitutable structure. The main example of this is submodularity, which has been connected to computational tractability throughout theoretical computer science and machine learning [Krause and Golovin, 2012]. Submodularity, in addition to having nice algorithmic properties, is also recognized as a natural model of substitutes in (algorithmic) game theory [Lehmann et al., 2001].
This paper draws parallels to such research because our definition of informational substitutes turns out to correspond formally to submodular valuation functions. We show market equilibrium results of a similar flavor, but for “information markets”; and we also show algorithmic results of a similar flavor for the problem we call SignalSelection, which is the problem of selecting an optimal set of signals subject to constraints.
However, we emphasize that informational S&C pose challenges that do not arise in the item setting:
- •
Items are modeled as having an a priori innate value. Information is not; its value must arise from context.
- •
Items are modeled as being atomic or indivisible, with no inner structure. In contrast, information, modeled as e.g. random variables, is defined by inner structure: the probability distributions from which it is drawn.
- •
The relationship between items is completely determined by the valuation function in the context of interest. Concretely, when modeling a set function , it is usually not the case in the model that items , , and have a special relationship that has an impact on allowable forms of . In contrast, in a value-of-information setting, the value of observing a triple of signals cannot be completely arbitrary; it must depend somehow on correlations between these signals.
We give an in-depth description of how our definitions overcome these challenges in Section B.
A.0.2 Information in markets
The “efficient markets hypothesis” (EMH) refers to a large set of informal conjectures about how quickly information is revealed and incorporated into the prices of financial instruments in markets. For our purposes, a “financial instrument” may be formalized as a security with an uncertain value, which will be revealed after the close of the market; each share purchased of that security may then be redeemed for a payout equal to this revealed true value of the security. Concretely, one can picture a binary security that has value if a certain event occurs and [math] otherwise.
Fama [1970] discussed formalizations of the EMH with varying levels of strength. Kyle [1985] defined a formal model of financial traders in markets, involving both informed traders and “noise” traders who are uninformed and essentially trade randomly; this is the current most common model of such trading in the economics literature. However, formal progress on this question was very slow until Ostrovsky [2012] showed that, in equilibrium of this model, information is always eventually aggregated under a certain condition on securities. Formally, this means that the price of a security converges to its ex post expected value conditioned on the information held by all traders. Ostrovsky [2012] considered both finite-round and infinite-round markets (with and without discounting), and considered prediction markets (described below) as well as Kyle’s model. One subtlety that may be worth pointing out is that, in an infinite-round market without discounting, it is not known that a Bayes-Nash equilibrium always exists, while this is known for the other cases. Ostrovsky [2012] showed in all cases that aggregation occurs in any equilibrium, under his “separability” condition on securities. The condition essentially ensures that, if information is not yet aggregated, then some participant has information that can be used to make a “useful” trade, i.e. one that makes money and (therefore) intuitively makes “progress” toward aggregation.
This showed that information is eventually aggregated; but how is it aggregated? It would be ideal if traders rush to reveal their information, but very bad if they “delay” as long as possible.
Such questions are difficult to address in the economic model of financial markets of Kyle [1985], Ostrovsky [2012]. Research on the dynamics of strategic trading has made some progress in the model of prediction markets. These are simplified financial markets in which there are no uninformed “noise” traders and in which participants generally interact one-at-a-time with a centralized market maker, who sets prices via a transparent mechanism and subsidizes the market. Specifically, research on strategic play focuses, as does this paper, on the market scoring rule design of prediction markets [Hanson, 2003], which are based on proper scoring rules (see e.g. Savage [1971], Gneiting and Raftery [2007]). However, we note that market scoring rule markets are equivalent, in a strategic sense, to more traditional-looking “cost function” based prediction markets Abernethy et al. [2013].
The following was known about equilibrium in such markets prior to this work, in addition to Ostrovsky [2012]. Chen et al. [2007] studied the scoring rule and a particular type of information structure among the traders, namely, that each trader’s “signal” (information) was distributed independently of all others’ conditional on the true value of the security. (For a simple example of such a structure, suppose that the true value of the security is distributed randomly in some way; then each trader observes this true value plus independent noise.) In this conditionally-independent case, Chen et al. [2007] showed that the “ideal” outcome does indeed occur in equilibrium: Traders all rush to reveal their information as early as possible.
Subsequently, Dimitrov and Sami [2008] also studied the scoring rule but considered other signal structures, particularly independent signals (that is, unconditionally independent). For a simple example, suppose each trader observes an i.i.d. random variable, and the true value of the security equals the sum of all the traders’ observations. Dimitrov and Sami [2008] showed that, in this case, the “ideal” outcome does not occur. However, when assuming discounting and an infinite number of trading rounds, Dimitrov and Sami [2008] showed that information is “eventually” aggregated. This result was generalized to any scoring rule (not just ) by Ostrovsky [2012]. Chen et al. [2007] and Dimitrov and Sami [2008] were combined and extended in Chen et al. [2010].
Then, Gao et al. [2013] revisited the scoring rule in finite-round markets and considered the information structure where all traders signals are unconditionally independent. In this case, Gao et al. [2013] showed that the “worst possible” outcome occurs in equilibrium: Traders all delay as long as possible before making any trades based on their information. This casts doubt on the efficient markets hypothesis and suggests, taken in tandem with Chen et al. [2007], that structure of information is crucial to determining strategic behavior.
A.0.3 Other game-theoretic settings.
In almost any Bayesian extensive-form game, the question of information revelation is relevant. While we hope that future work may expand the set of topics to which informational substitutes and complements may apply, in this section, we will focus on the most closely related works, those that directly touch on S&C, or those for which we show results in Section 3.2.
The model of prediction markets is in some sense the simplest model of strategic information revelation in dynamic settings. Thus, it is natural that settings such as crowdsourcing contests are closely related. One such model of crowdsourcing and machine-learning contests appears in Abernethy and Frongillo [2011], Waggoner et al. [2015]. In that framework, participants iteratively provide data sets or propose updates to a central machine-learning hypothesis, being rewarded for the improvement they make to performance on a test set of data. A prediction market can be seen as a special case. Those works did not address strategic equilibria of the mechanisms they proposed.
Another related setting is the model of question-and-answer forums in Jain et al. [2009]. That paper introduced a model where an asker has some value function for “pieces of information”, which are not modeled directly. Participants can strategically choose when to reveal information. Jain et al. [2009] identified “substitutes” and “complements” cases in which participants rush (respectively, delay) to provide answers. However, Jain et al. [2009] did not provide any endogenous model for asker utility or information; the information was modeled almost as discrete items without structure. Hence, it was not clear from that work under what circumstances (if any) pieces of information would satisfy their substitutes and complements conditions.
In Section 3.2, we describe implications of our work for results in the above two settings.
More broadly, there are large literatures dealing with signalling in games [Spence, 1973], or the more recent Bayesian persuasion literature [Kamenica and Gentzkow, 2011, Gentzkow and Kamenica, 2011, Dughmi and Xu, 2016]. While the models and questions in this area are related, to our knowledge there is no immediate connection. It may be that future work uncovers connections of informational S&C to this field, but the literature on persuasion and signalling in games does not seem to have developed notions of informational substitutes nor tools for addressing the applications considered in this paper.
Another significant line of work has considered signalling in auctions, e.g. Milgrom [1981], Milgrom and Weber [1982]. The only paper in this literature that we know to explicitly formalize a notion of substitutable information is Milgrom and Weber [1982], which considers a common-value auction with two bidders, one informed and one uninformed, with two signals, each a real-valued random variable with some positive affiliation with the item’s value. The authors define a notion of substitutes specific to their context and show that it implies intuitive properties for this asymmetric-information auction setting. In future work, it would be interesting to see if there is a formal connection of our definitions to their setting.
A.0.4 Algorithms for information acquisition.
The value of information to a decision problem was formally introduced by Howard [1966] and is also closely related to the classic problem in statistics of Bayesian experimental design [Lindley, 1956]. Given this perspective, it is natural to consider the problem of acquiring information under constraints. This problem has historically been investigated from many different angles, e.g. [Mookerjee and Mannino, 1997]. It is known to be very computationally difficult in some general settings [Krause and Guestrin, 2009].
A successful recent trend in this area is to leverage submodular structure to apply efficient approximation algorithms. For instance, an approximation ratio of is obtained by the greedy algorithm for maximizing a monotone submodular function subject to a cardinality constraint. This algorithm or related submodular maximization algorithms were utilized by Krause and Guestrin [2005b], Guestrin et al. [2005], and a variety of literature since; see Krause and Golovin [2012] for a survey. In cases where the information is acquired not in a batch but adaptively over time, based on the information observed so far, the problem (and/or solution) is known as adaptive submodularity [Asadpour et al., 2008, Golovin and Krause, 2011].
Appendix B Defining S&C: intuition, challenges, and historical context
In this section, we describe the intuition, justification, and historical context behind our proposed definitions of informational substitutes and complements (S&C). The formal definitions are presented more concisely starting in Section 2, to which the reader may skip if uninterested in background. We focus on the substitutes case; the complements case, where not mentioned, is analogous.
The only prior attempt at definitions of S&C of which we are aware is Börgers et al. [2013], which will be introduced shortly. The setting and approach in that paper are very appealing, and they inspire our approach in this paper. However, there are also key drawbacks that motivate us to diverge from their approach in several ways. We will describe in context the drawbacks and our approach to overcoming them.
B.0.1 Defining the value of information
Defining informational S&C turns out to be significantly more work than defining substitutes and complements for valuation functions over items, starting from the very beginning: How does value arise in the first place? It is generally accepted to model a valuation function over a set of goods as some , without justifying how arises (perhaps the items are yummy, shiny, or have other desirable qualities). However, outside of curiosity, it seems that information’s innate value is more questionable; and furthermore, should depend on its probabilistic structure. For instance, two signals may be independent or may be highly correlated. How does this relate to their value?
A solution arises from the observation that information’s value is often determined by the use to which the information may be put. As in Howard [1966], Börgers et al. [2013], for us the value of information arises from its utility in the context of a decision problem. We consider a Bayesian model of information in which there is a prior distribution on the event of interest and on the possible pieces of information, called signals. In a decision problem, the agent observes some signals and then makes a decision , after which the outcome is revealed and the agent’s utility is . Thus (following Börgers et al. [2013]), we define a value function on signals, for a given decision problem and prior , by the expected utility of the agent given that she first observes , then takes the optimal action based on that information. Formally,
[TABLE]
Thus, one can compare, for instance, the value for observing signal alone versus the value for observing signal alone. Note that the marginal value for observing signal , given that the agent will already observe signal , is \mathcal{V}^{u,P}(A_{2}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}A_{1})-\mathcal{V}^{u,P}(A_{1}), where the notation A_{2}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}A_{1} means to observe both signals (this notation will be explained shortly).
B.0.2 The approach of Börgers et al.
Börgers et al. [2013] proposes the following definition: Given an event , two signals and are substitutes if for every decision problem (and associated value function ),
[TABLE]
where A_{1}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}A_{2} denotes observing both signals, while denotes not observing any signal and deciding based on the prior.
This definition has two properties that might seem attractive, but turn out to be fatal in many cases of interest: (a) it does not depend on the particular decision problem, but only on how , , and are correlated; (b) it depends only on the values , , of both, and of neither, and does not depend on any internal structure of and . We explain why these properties are problematic and how our definition will differ.
a. Lack of dependence on the decision problem.
The problem here is that in a majority of cases, two signals can turn out to be either substitutes or complements depending on the decision problem at hand. For example, whether two weather observations are substitutes or complements depends on what decision is being made. Temperature and dew point might be considered complements when deciding whether to bring one’s umbrella.888Accepting the proposition that knowledge of both gives a much better prediction of rain than knowledge of either alone. But when deciding, for instance, how warmly to dress, these two measurements might be considered substitutes since, given one of them, the other gives relatively less information about the comfort level of warm or cool clothing. For another example: To a trader deciding whether to invest in a technology index fund (that is, a stock whose value follows that of the general tech sector), the share prices of two given tech companies may be substitutable information, since each gives some indication of the current value of tech stocks. But to a trader deciding which of these two specific companies to invest in, these prices may be complements, since the decision can be made much more effectively with both pieces of information than with either alone.
The definition of Börgers et al. [2013] cannot capture such scenarios because it requires two signals to be substitutes for every possible decision problem. Our solution is to define S&C relative to both the particular information structure and the particular decision problem.
b. Lack of dependence on the internal structure of the signals.
The other concern with the definition of Börgers et al. [2013] is that it only depends on “extreme” values: \mathcal{V}^{u,P}(A_{1}),\mathcal{V}^{u,P}(A_{2}),\mathcal{V}^{u,P}(A_{1}\mathrel{\text{\raisebox{1.07639pt}{\scalebox{0.8}{\vee}}}}A_{2}), and . Hence, it ignores the internal structure of and , which can lead to incongruous predictions. For example, suppose that and are substitutes while and are complements. Now consider the signals and . For some decision problems, it may be that the signals are slightly more important and so and seem to be substitutes. For other decision problems, it may be that the signals are slightly more important and and seem to be complements. This is formalized in Example 2.2.3.
To see why this could be problematic for a predictive or useful theory, suppose that an agent will be able to observe , and a seller wishes to sell to that agent the opportunity to observe as well. As just argued, one might have defined and to be “substitutes” or “complements” depending on very small fluctuations in the decision problem. But the seller, by “hiding” or “forgetting” either the or the component of his signal, can force the signals to become either substitutes or complements as she desires. A definition that does not account for internal structure may get such examples completely wrong, e.g. classifying the signals as substitutes when the seller can make them behave as complements.
We will introduce definitions that account for the internal structure of signals.
B.0.3 Our approach: dependence on context and internal structure
Context.
As mentioned above, we will allow the definitions of S&C to depend on the particular decision or value function . That is, while Börgers et al. [2013] defined a particular information structure to be substitutes on pairs of signals if satisfied a condition for all , we will define a pair to be substitutes if satisfies a similar condition. This will turn out to be crucial in all of our applications.
The potential drawback is that it might be difficult to say anything general about when signals are substitutes or complements; it might seem that one must take things completely on a case-by-case basis.
We make two counterpoints. First, a universal approach may be the wrong goal or “too much to hope for”. For instance, in the case of items, there is no such optimistic analogue; one does not consider items that are always substitutes for every valuation function. Despite this, there are many successful theories leveraging substitutable goods. These approaches start by assuming a context (e.g. valuation functions) for which the goods are substitutes; similarly, we can consider a set of signals and only those decision problems for which they are substitutes.
Second, we later give some evidence that we need not take things completely case-by-case. We seek classes of signals that can be considered substitutes or complements in a broad class of decision problems. For example, we show that if signals are independent, then they are complements for any decision problem satisfying a smoothness condition. Our work also gives intuition for which kinds of signals are more likely to be substitutable or are substitutable in more contexts. And indeed, one of the exciting questions raised by our work is how the context of a decision problem and internal structure combine to produce substitutable or complementary features.
Probabilistic structure.
We will allow definitions of S&C to depend on the internal structure of signals. But how? Two signals may be related in complex ways by correlations with each other and with the event of interest. Therefore, a more natural analogy than substitutability of two items may be substitutability of two subsets of items. Consider Lehmann et al. [2001], which studied valuation functions over sets of items. There, the authors identified a natural “no complementarities” condition where two sets of items, and , could only be substitutes if all “pieces” of those subsets were substitutes: no subset of set could be complementary to the set , and vice versa. This turned out to be exactly a requirement that the valuation function be submodular: that it exhibit diminishing marginal value.
We would also like to capture diminishing marginal value. The challenge that arises is, what is a marginal “unit” of information? The answer actually may vary by application.
In some applications, a “marginal unit” may be an entire signal: Given the current subset of , one can either add another , or not. This would be appropriate for cases where our above arguments about internal structure may not apply. For example, perhaps the seller in an auction does not have the ability, for whatever reason, to process her signals in any way; she can just choose between allowing each of them to be either broadcast or kept private. In this paper, we utilize this notion, which will correspond to “weak substitutes”, in the context of discrete optimization problems where an algorithm must choose between acquiring different signals. In many contexts, it is impossible to acquire partial signals, so this is the natural marginal unit. While they may be useful, they also are subject to the criticisms given above; in many contexts that allow “pieces” of signals, they may not behave as substitutes or may even behave as complements. 2. 2.
Sometimes, a “marginal unit” may be some partial information about a signal, in the form of a “fact” about its realization. For instance, imagine a commuter learns something about the barometer reading but not the exact reading; e.g., whether it is above or below , or the measurement rounded to the nearest integer. This application arises when considering pure strategies in a game, or deterministic “post-processings” of a signal in algorithmic contexts. The effect of such processing is always to “coarsen” a signal by pooling multiple outcomes together under one announcement. In the barometer example, all realizations of the signal below map to the same result, and all realizations above map to the other result; similarly when rounding to the nearest integer. If a set of signals exhibits diminishing marginal value with respect to this notion, we will term them “moderate substitutes”. We actually do not provide an application for moderate substitutes in this paper, but expect them to be useful in contexts such as those just described. 3. 3.
Finally, a “marginal unit” may be partial information in the form of a noisy “signal about the signal”. For instance, the commuter may learn the barometer reading plus Gaussian noise. To see that this notion may be much more fine-grained than the previous one, imagine starting from the binary barometer example, where the commuter learns whether or not the barometer is below ; and now imagine that, with some probability , this observation is “flipped” from the correct one. When , the commuter can be certain that she learns correctly which outcome is the case (above or below ). But as , the commuter learns less from the signal. If a set of signals exhibits diminishing marginal value even with respect to such partial information – for instance, diminishing marginal value as decreases from to [math] – then we term them “strong substitutes”. In applications where, for instance, the barometer observation is controlled by a strategic agent whose strategy consists of a “garbling” of that observation, this will be a useful notion of marginal information.
We formalize these marginal units of information using lattices of signals: sets of signals with a partial order corresponding to “informativeness” and satisfying some natural conditions. While our proposed uses for them here are quite new, the lattices we use, or closely related concepts, are relatively classical. For weak substitutes, we consider the lattice of subsets of signals, with partial order given by set containment; this corresponds directly to subsets of goods and the notion of substitutes is essentially the same.
For moderate substitutes, we utilize a variant of Aumann’s classic model of information in Bayesian games [Aumann, 1976], in which signals correspond to partitions on a ground state of the world. To our knowledge, although it is known that Aumann’s signals form a lattice (because the space of partitions do), they have not been used to formalize marginal units of information. One difference in the variant we propose is that the ground states only determine the signals, not the event or any other pieces of information; this makes our model much more useful for formalizing marginal pieces of information because the ground states only contain information about the signals.
For strong substitutes, we extend Aumann’s model to capture randomized “garblings” of signals. Although this is not the model normally used in that context, the idea and intuition is extremely similar to Blackwell’s criterion or partial ordering on signals [Blackwell, 1953]. One major difference is that in our model, there is a particular event of interest and signals are ordered according to informativeness about that event, rather than pure informativeness. Also, the use of Aumann’s partition model allows our signals to form a lattice.
B.0.4 Capturing “diminishing” marginal value
Luckily, once we have placed a lattice structure on signals, we can apply a now-classic criterion for diminishing marginal value: submodularity. Often, submodularity is a condition for functions on subsets, e.g. , which is submodular if an element ’s “marginal contribution” to , , is decreasing as elements are added to . This is a widely-used model for substitutability of discrete, indivisible items [Lehmann et al., 2001]. The same goes for supermodularity, increasing marginal value, and complementary items.
The final piece of our puzzle will be to utilize submodularity and supermodularity, and extensions, to capture diminishing marginal value and increasing marginal value respectively. This is formalized in Section 2.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1Abernethy et al. [2013] Jacob Abernethy, Yiling Chen, and Jennifer Wortman Vaughan. Efficient market making via convex optimization, and a connection to online learning. ACM Transactions on Economics and Computation , 1(2):12, 2013.
- 2Abernethy and Frongillo [2011] Jacob D Abernethy and Rafael M Frongillo. A collaborative mechanism for crowdsourcing prediction problems. In Advances in Neural Information Processing Systems (NIPS) , pages 2600–2608, 2011.
- 3Archak and Sundararajan [2009] Nikolay Archak and Arun Sundararajan. Optimal design of crowdsourcing contests. In Proceedings of the International Conference on Information Systems (ICIS) , 2009.
- 4Asadpour et al. [2008] Arash Asadpour, Hamid Nazerzadeh, and Amin Saberi. Stochastic submodular maximization. In Proceedings of the 4th International Workshop on Internet and Network Economics (WINE) , pages 477–489. Springer Berlin Heidelberg, 2008. ISBN 978-3-540-92185-1. doi: 10.1007/978-3-540-92185-1_53 .
- 5Aumann [1976] Robert J. Aumann. Agreeing to disagree. Annals of Statistics , 4(6):1236–1239, 1976.
- 6Babaioff et al. [2012] Moshe Babaioff, Robert Kleinberg, and Renato Paes Leme. Optimal mechanisms for selling information. In Proceedings of the 13th ACM Conference on Electronic Commerce , pages 92–109. ACM, 2012.
- 7Balkanski et al. [2015] Eric Balkanski, Aviad Rubinstein, and Yaron Singer. The limitations of optimization from samples. ar Xiv preprint ar Xiv:1512.06238 , 2015.
- 8Blackwell [1953] David Blackwell. Equivalent comparisons of experiments. The Annals of Mathematical Statistics , 24(2):265–272, 1953.