Naive probability

TL;DR

This paper introduces a simple, rational model of probability that emphasizes low-resolution, coarse-grained reasoning about uncertain events.

Contribution

It presents a novel, low-resolution approach to modeling probability, contrasting with traditional high-precision methods.

Findings

Provides a rational framework for low-resolution probability modeling

Highlights the simplicity and potential applications of coarse probability models

Suggests new directions for probabilistic reasoning with limited detail

Abstract

We describe a rational, but low resolution model of probability.

Tables

Table 1. Table 1: Likeliness of cause of death

Cause of death	Default	Reykjavík	Istanbul	trip
in hospital	4	4	5	4
by accident (non-ski)	4	4	4	5
at home in bed	4	1	1	0
in war	1	0	0	1
by homicide	1	1	1	1
by suicide	2	2	2	1
by forces of nature	1	4	1	2
by ski accident	1	2	1	1

Equations

$$l(A)=\underset{i}{⨁}l(B_i)\otimes l(B_i\to A)$$

$$l(A)=⨁l(B_i)={\oplus }_{i=0}^6{\oplus }_{l(B_j)=i}i$$

$$l(\text{Snowbird}\to \text{travelling})$$

$$l(\text{Snowbird}\to \text{skiing})$$

$$l(\text{Snowbird}\to \text{snowing})$$

$$l(\text{Snowbird}\to \text{death})$$

$$l(p)=\min\big{\{}l(v_{i}\to v_{i+1}):\;i=1\ldots n-1\big{\}}$$

$$\min\big{\{}l(p):\;p\text{ is a path in $G$ from $a$ to $b$}\big{\}}$$

$$\max\big{\{}l(a),l(p):p\text{ is a path from $a$ to $b$}\big{\}}$$

Full text

1 Introduction

Historically, the theory of probability emerged from the efforts of Pascal and Fermat in the 1650s to solve problems posed by a gambler, Chevalier de Méré Rényi (1972); Devlin (2008), and reached its current form in Kolmogorov (1933). Remarkably, not even highly experienced gamblers can extract high precision probability estimates from observed data: one of de Méré’s questions concerned comparing the probabilities of getting at least one 6 in four rolls of one die ($p=0.5177$) and getting at least one double-6 in 24 throws of a pair of dice ($p=0.4914$). Four decades later, Samuel Pepys is asking Newton to discern the difference between at least two 6s when 12 dice are rolled ($p=0.6187$) and at least 3 6s when 18 dice are rolled ($p=0.5973$).

In this paper we make this phenomenon, the very limited ability of people to deal with probabilities, the focal point of our inquiry. These limitations, we will argue, go beyond the well understood limits of numerosity Dehaene (1997), and touch upon areas such as cognitive limits of deduction Kracht (2011) and default inheritance Etherington (1987). We will offer a model of the naive/commonsensical theory of probability. In Section 2 we discuss likeliness, which we take to be a valuation of propositions on a discrete (seven-point) scale. In Section 3 we turn to the inference mechanism supported by the naive theory, akin to Jeffreys-style probability updates. In Section 4 we briefly sketch the background theory and discuss what we take to be the central concern, learnability.

2 The likeliness scale

We use the term ‘likeliness’ for a valuation on a 7-point scale 0,…,6 which only roughly corresponds to a discretized notion of probability (we avoid the more natural-sounding ‘likelihood’ as this already has a well-established technical sense). 0 is assigned to impossible events, $l(e)=0$, and 6 to necessary ones. Note that in this regard $l$ corresponds better to everyday usage in that zero probability events ($p(e)=0$) do occur, and $p(e)=1$ guarantees only that the event $e$ has measure zero exceptions of occurring. $l(e)=2$ means unlikely: an example would be traffic accidents. $l(e)=1$ means conceivable, events that are unlikely in the extreme, but not forbidden by physical law. An example would be being struck by a meteorite.

There is a duality between $x$ and $6-x$ as in Łukasiewicz $L_{7}$, so $l(e)=4$ is assigned to likely events such as travelling without an accident and $l(e)=5$ to typical or expected ones. Almost all lexical knowledge falls in this last category: chairs are by definition furniture that support a seated person, and if a particular instance collapses under ordinary weight we say it failed (whereas we don’t conclude that my car failed when I get in a traffic accident – alternative hypotheses such as driver error are readily entertained). Events that are neither likely nor unlikely are assigned the value 3.

Clearly, using exactly $7$ degrees is somewhat arbitrary, but it is evident that using only $3$ (say impossible, unknown, possible) would be a gross oversimplification of how people deal with probability, and using a very fine scale would create illusory precison that goes beyond people’s actual abilities. With $7$, we stick to a relatively small but descriptive enough scale. Even if one could argue that, say on cognitive grounds, $5$ or $9$ degrees would be better, the overall methodology would be the exact same, and everything below could be easily modified and worked out with that scale. Altogether, our choice of having a $7$-degree scale is more of an illustration than a commitment, albeit one well supported by practical exerience with semantic differentials Osgood et al. (1957).

The commonsensical valuation, which is our object of study here, differs from probabilities in several respects. The most important from our perspective is lack of additivity. At this point, it is worth emphasizing that the theory of likeliness valuation is not intended as a replacement of the standard (Kolmogorov) notion of probability, which we take to be the correct theory of the phenomena studied under this heading, but rather as an explanatory theory of how the naive worldview accounts for these phenomena. The fact that as a computational device the standard theory is superior to the naive theory is no more a reason to abandon study of the naive theory than the superiority of eukaryotes is reason to abandon study of prokaryotes.

In this regard, our work differs significantly from studies like Pearl (2009) or Spohn (2012), which investigate causalty and degrees of belief in a conservative framework, taking the preservation of all results of the standard theory for granted. To quote Spohn (2012) “Probability theory is indeed my paradigm; by all means, we must not fall below its standards.” Since our goal is to study the commonsensical notion, we are operating in a different paradigm, and by lack of additivity we don’t just mean lack of $\sigma$-additivity, but something that is already visible on finite sums. Consider the Law of Total Probability, that $p(A)$ can be computed as $\sum_{n}p(A|B_{i})p(B_{i})$ where the $B_{i}$ provide a (typically finite) partition of the event space. The equivalent formulation with likelinesss normed to 1 would be

[TABLE]

Here we retain the assumption that likeliness is a valuation in a semiring where addition $\oplus$ and multiplication $\otimes$ are defined, but instead of conditional probability we will speak about relevant implication $\rightarrow$ having a valuation of its own. The semiring of greatest interest is the one familiar from $n$-valued logic, where $\otimes$ is min, and $\oplus$ is max. We note that we allow two types of propositions only: standalone sentences $A$ and sentences in the form of an implication $A\to B$, cf. Section 3.

To put lack of additivity in sharp relief, consider the following commonsensical example: all men are mortal. If we take $A$ to be eventual death, we have $l(A)=6$. If we ask people to elicit causes of death $B_{i}$, they will produce a handful of causes such as cancer or heart attack that they consider likely ($l=4$); some like accidents of infections they consider neither very likely nor very unlikely ($l=3$); some like autoimmune diseases or freezing to death they consider less likely ($l=2$); and some they consider conceivable but extremely unlikely such as murder/suicide or terrorism ($l=1$). Needless to say, such valuations are not precisely uniform across people, but they do have high intrasubjective consistency (as measured e.g. by $\kappa$ statistics). Since $l(B_{i}\rightarrow A$) is by definition 6, we are left with an enumeration of causes:

[TABLE]

The problem here is that no amount of heaping on more of less likely causes will increase the $\oplus$ above the valuation of its highest term. The phenomenon is already perceptible at the low end: if we collect all conceivable causes of death from lightning strike to shark attack, we have ‘death by (barely) conceivable causes’ which itself is unlikely, not just conceivable.

In actual mortality tables, this phenomenon is reflected in the proliferation of categories like ‘unknown’, ‘unspecified’, and ‘other’, which take up the slack. Depending on the depth of tabulation, the catchall category typically takes up between .5% and 5% of the total data, which corresponds well to the lack of sensitivity below 1% observed in the de Méré and Pepys examples we started with.

Another obvious difference between the standard and the naive theory is the way extremely low or extremely high probability events are treated. When we want to draw the line between impossible and conceivable events, we don’t rely on a single numerical cutoff. Nevertheless, to get a rough idea of the probability values corresponding to the various likeliness values, we take the proverbial ‘one in a billion chance’ as marking, in some fuzzy sense, the impossible/conceivable boundary. Using $\log_{10}$ odds scale (for a justification of base 10, see Jaynes (2003) Ch. 4.2), this gives $-9$, so the next natural order of magnitude Gordon and Hobbs (2017) is $-9/\sqrt{10}\approx-2.846$. Converting back from log odds to probabilities, this brings us to $p=0.001423$, which we can take to mark the conceivable/unlikely boundary. The next natural order of magnitude brings us to log odds $-0.9$, corresponding to $p=0.1118$, which marks the unlikely/neutral boundary (see Fig. 1).

By symmetry, in this reckoning everything between $p=0.1118$ and $p=0.8882$ is considered $l=3$, neither particularly likely nor particularly unlikely. Likely events are between $p=0.8882$ and $p=0.9986$, while typical events are above that limit though still with a one in a billion chance of failure. As at the low end, the naive theory lacks the resolution to distinguish such failure rates from necessity (total absence of failure).

We should emphasize here that it is the overall logic of the scheme that we are vested in, not the particular numbers. For example, if we assume an initial threshold of one in a million instead of one in a billion, the limits will be at $0.0125$ and $0.2008$ (and by symmetry at $0.7992$ and $0.9875$), as shown by the blue curve in Fig. 1, but the major characteristics of the system, such as the ‘neither likely nor unlikely’ category takes up the bulk of the cases, or that $l=2$ cases are noticeable, whereas $l=1$ cases are barely detectable, remain unchanged.

It should be emphasized that such limits, however we set them, are not intended as a crisp characterization of human classification ability, the decision boundaries are fuzzy. Returning to lack of additivity, there may well be several likeley causes of death beyond cancer and heart attack, but no closed list of such is sufficient for accounting for the fact that eventual death is necessary. For this, we need a slack variable that lifts the $\oplus$ of the likely $l=4$ causes to $l=5$ or $l=6$, which we find in $B_{n}$ ‘death by other causes’. We note that historically old age was seen as a legitimate cause of death, and only very recently (since the 1980s) do coroner’s reports and obituaries find it necessary to list the failure of a specific organ or subsystem as the cause of death.

Finally, in contradistinction to the standard theory, $\oplus$ can extend only to a handful of terms, especially as the terms are implicitely assumed independent. By the above reckoning, it takes less than 80 unlikely causes to make one neutral, and less than 8 neutral to make a likely one. The geometry of the likeliness space is tropical Maclagan and Sturmfels (2015), with the naive theory approximating the log odds (max) semiring.

3 Naive inference (likeliness update)

We have two types of propositions: stand alone sentences $A$ and sentences in the form of an implication $A\to B$. A context is a (finite) collection of propositions, which can be represented by a directed graph: nodes of the graph denote propositions $A$ and edges of the graph denote implications $A\to B$. The likeliness function is an evaluation acting on the graph: both vertices and edges can have numeric values between [math] and $6$, [math] representing impossibility, $6$ representing necessity.

Values $l(A\to B)$ belong to the inner model (for details see Section 4), therefore they are hardly subject to change. Take the following example as an illustration. Snowbird is a ski resort in Utah. Say, for a typical European, Snowbird is related to travelling, skiing, and snowing with the likelinesss

[TABLE]

Such likelinesses express typicality of these relations. Skiing is related to some extent, say, to ski-accident, and ski-accident to death. Take the example below (for the sake of example we differentiate between ski-accidents and accidents; the latter excludes accidents occurring while skiing).

In a typical scenario one does not have any likeliness of the implication Snowbird$\to$death inside the inner model. However, naive inference works: Snowbird typically implies skiing; skiing is likely to imply ski-accident; finally, it is neither likely nor unlikely that ski-accident results in death. Therefore, one may say [visiting] Snowbird is neither likely nor unlikely to result in death, i.e.

[TABLE]

In a similar manner, one could obtain the likeliness $l($skiing$\to$death$)=3$ by saying that skiing is likely to ensure a ski-accident, while it is neither likely nor unlikely that ski-accident results in death.

In virtue of the examples above we give a formal model. Let assume we have a finite directed graph $G=(V,E)$ and an evaluation $l:E\to\{0,\ldots,6\}$. We would like to evaluate edges of the complete graph on $V$ that are not in $E$. Pick two vertices $a,b\in V$, $a\neq b$ and suppose $(a,b)\notin E$. Let $p=(v_{1},\ldots,v_{n})$ be a path in $G$ from $a=v_{1}$ to $b=v_{n}$. We write

[TABLE]

The value $l(p)$ expresses how likely the inference $a\to b$ is in case we are relying on the chain of already evaluated implications belonging to the path $p$. Then the value $l(a\to b)$ is obtained as

[TABLE]

In the example above vertices of the graph did not have likelinesss. Suppose we get new information about John: he is likely to be in Snowbird, i.e. $l($Snowbird$)=4$. What consequences can we draw? Being a typical European, if John is in Snowbird, then he must be travelling and it is really typical that people travel to Snowbird to ski. The information that $l($Snowbird$)=4$ propagate via the edges of the graph: the likeliness of those propositions that are related to Snowbird (that is, they are connected by an edge in the graph to Snowbird) will be updated given new information: $l($travelling$)$ and $l($skiing$)$ become $4$. In the formal model, given the value $l(a)$ and a path $p=(v_{1},\ldots,v_{n})$ from $a=v_{1}$ to $b=v_{n}$, using the definition of $l(p)$ in equation (3) we can update the likeliness $l(b)$ of $b$ as

[TABLE]

This process of updating iterates: neighbours of just updated vertices get updates in the next round, etc. Supposing the graph is connected, all vertices are assigned with likelinesss as shown of Fig. 3.

Let us now suppose that we learn that John died abroad. The first column of Table 1 describes the default likelinesses we assign to various causes of death, with subsequent columns showing the updates based on whether we learn ($l=6$) that the death took place in Reykjavík, Istanbul, or on a tourist trip, destination unspecified. Some rows are easy to explain: for example death at home in bed is considered likely, but if we know that John was on a tourist trip the implication is that he is not at home, and the likeliness is demoted to 1. Not 0, because there are extremely unlikely but not inconceivable scenarios whereby he fell in love with the place, bought a home, and resettled there, cf. Jaynes (2003) 5.2.2. This is a scenario that is, perhaps, worth considering if we know only that John went to Reykyavík or Istanbul and tourism was merely an inferred, rather than explicitly stated, goal of the trip, but if we know it was a tourist trip and nothing more (last column) this is logically incompatible with being at home.

The same logic is operative in the next row (war): since we know there is no war in Reykjavík or Istanbul the likeliness is demoted to 0, but for a generic trip it is not, since we do know that there are war zones on the globe and John may have visited one of these.

We obtain that death by ski accident is less likely in Reykjavík (2) than in Snowbird (3) not because skiing is inherently more safe in Iceland, but simply because one can travel to Reykjavík for many reasons, and the likeliness that one goes skiing there is 3, perhaps 4, whereas to ski in Snowbird is typical (5). In connection of Reykjavík we are much more likely to think of death by forces of nature, as there are many natural dangers nearby, from volcanoes to geysirs and sneaker waves, indeed this class rises to the top category (4).

This line also illustrates the nonmonotonic nature of the calculus: in general we consider death by forces of nature conceivable but unlikely in the extreme (1), knowing that John went to a tourist trip increases this to 2, but further learning that he went to Istanbul, not particularly known as a natural danger zone, demotes this back to 1.

4 Learning

As we stated at the outset, our goal is to characterize the naive view of probability, an undertaking closely tied to naive physics Hayes (1979), folk psychology Ravenscroft (2016), etc. Here we offer a brief, informal outlook of the entire theory, but deal in detail only with the question of how the adult system of mental representation, called the ‘inner model’ above, is formed in regards to probabilities. We introduce terminology by paraphrase, describing the intended meaning before offering more formal definitions. Our goal is to stay close to the standard meaning of these terms, but we do not intend to fully recreate every aspect of the theories where they originate.

One of the central questions for linguistics, both mathematical/computational and theoretical, is to characterize how text and meaning are related. The standard answer, provided by Montague Grammar, is to define a homomorphism from structured/disambiguated text to formulas of intensional logic. For naive theory, we use a simpler model composed of (hyper)graphs (Quillian, 1969; Collins and Loftus, 1975). For the basic building blocks of our model, the vertices of a graph, we assume a large number (about $10^{5}$) of ur-objects, roughly one per morpheme or word. In addition to these, we will have a few technical elements such as the empty node $\cdot$, three directed connectives ‘0’ (is, isa); ‘1’ (subject); and ‘2’ (object). Our theory of types is rather skeletal, especially when compared to situation theory Barwise and Perry (1983); Devlin (1991), with which we share a great deal of motivation, especially in regards to common-sense reasoning about real world situations. When we say that a node is (defeasably) typed as location or person, this simply means that a 0-edge runs from the node in question to the location or person node.111An initial implementation is available at https://github.com/kornai/4lang, and a parser translating English and Hungarian text to this style of model is described in Recski (2016, 2018). For further details on knowledge representation by hypergraphs see Kornai (2019).

There can be various (n-place) relations obtaining between objects but, importantly, relations can also hold between things construed as objects, such as geometrical points with no atomic content, e.g. the corner of the room is next to the window’, complex motion predicates, e.g. ‘the flood caused the breaking of the dam’, and so on. Arguments of relations will be called matters, but they need not be material. We use edges of type 1 and 2 to indirectly anchor such higher relations, so the subject of causing will have a 1-edge running from the vertex cause to the vertex flood, and the object, the breaking of the dam, will have a 2-edge running from cause to the head of the construction where dam is subject of burst. For ditransitive and higher arity relations, which are tangential to our main topic here, see Kornai (2012). Valuations are partial mappings from graphs (both from vertices and from edges) to some small linear order $L$ of scores. There is no analogous ‘truth assignment’ because in the inner models that are central to the theory, everything is true by virtue of being present. On occasion we may be able to reason based on missing signifiers, the dog that didn’t bark, but this is atypical and left for later study. Here we use $L=\{0,\ldots,6\}$ for probability scores, but similar scales are standardly used in the measurement and modeling of all sorts of psychological attitudes since Osgood et al. (1957). Of particular interest is the activity valuation taking values in $A=\{-1,0,1,2\}$, where -1 means ‘blocked’, 0 means ‘inactive’, 1 means ‘active’, and 2 means ‘spreading’. These are used to keep track of the currently active part of the graph and implement the spreading activation model of Quillian (1969); Nemeskey et al. (2013).

Learning, therefore, requires three kinds of processes: the learning of nodes, the learning of edges, and the learning of valuations. We discuss each in turn.

Learning new vertices

We assume a small, inborn set of nodes roughly corresponding to cardinal points of the body schema Head and Holmes (1911) and cardinal aspects of the outside world such as the gravity vertical Campos et al. (1970), to which further nodes are incrementally adjoined. This typically happens in one shot, a single exposure to a new object like a boot is sufficient to set up a permanent association between the word and the object, likely including sensory snapshots from smell to texture and a prototypical image Rosch (1975). The association is effected by relations such as spatial on ‘boot on foot’ and the more abstract teleological for ‘boot for excursion’.

On rare occasions, children may learn abstract nodes, such as color, based on explicit enumerations ‘red isa color, blue isa color,…’, but on the whole we don’t have much use for post hoc taxonomic categories like footwear. Rather, we assume that seeing the boot on a foot, and having already acquired the notion of shoe, the child simply adds an edge ‘boot isa shoe’ to their preexisting representation, a matter to which we now turn.

Learning new edges

Again, we assume a small, inborn set of edges (0,1,2), and an inborn mechanism of spreading activation. The inborn edges are learned by a direct mechanism: once the edge ‘boot on foot’ is activated, this spreads to nodes associated to boot (initially, none) and to foot. For the sake of the example, let us assume that the child already knows about shoes. If not, we could start by describing the earlier learning process, whereby shoe gets associated to foot (which was posited as part of the body schema the child is born with) without altering our main point, that learning is always incremental attachment to previously learned nodes. Now, since foot is activated, this spreads to shoe, and the clild adds the new 0-edge between boot and foot.

The matter is a bit more complex when the association to be learned is not one of the primitive ones 0,1,2, but a contentful edge like for. If the parents are skinheads, the association ‘boot for excursion’ may never get formed, since the parents wear the boots on all occasions. But if the boots are only worn for excursions (or construction work, or any other specific occasion already identified as such by the child) we will see the boot and the excursion nodes jointly activated, which will prompt the creation of a new link between the two.

Learning valuations

We assume that the activation mechanism is unlearned (innate), and we may assume that some valuations, in particular the sensory hurt/enjoy valuation, are at least partially innate, e.g. in regards to harm to body parts. But this still leaves open the question of how we know that forces of nature are a likely cause of death in Reykjavík but not in Istanbul? Surely this knowledge is not innate, and most of us have not studied mortality tables and statistics at this level of specificity, yet the broad conclusion, that death by natural forces is more likely in Reykjavík than in Istanbul, is present in rational thinking at the very least in a defeasible form (we will revise our naive notions if confronted with strong statistical evidence to the contrary).

Part of the answer was already provided in Section 3, where we described the mechanism to compute these values. Aside from very special cases, we assume that such valuations are always computed afresh, rather than stored. What is stored are simpler building blocks, such as ‘volcano near Reykjavík’, ‘volcano isa danger’ from which we can easily obtain ‘danger near Reykjavík’. A great deal of background information, such that danger is connected to death, must be pulled in to compute the kind of valuations we described in Table 1, but this does not alter the main point we are making here, that inner models are small information objects (the entire mental lexicon is estimated to be about 1.5MB, see Mollica and Piantadosi (2019)).

5 Conclusions

We have offered a rational reconstruction of the naive theory of probability. This theory is not as powerful computational device as the standard theory, and generally only leads to rough estimates of likeliness. However, it is better suited for studying human cognitive behavior, as it requires very little data, and extends to a broad range of cases where the statistical data undergirding the standard theory is unavailable.

Acknowledgments

Gyenis was supported by the Premium Postdoctoral Grant of the Hungarian Academy of Sciences and by the Hungarian Scientific Research Found (OTKA), contract number K115593.

Kornai was supported by the Hungarian Ministry of Innovation and Technology NRDI Office within the framework of the Artificial Intelligence National Laboratory Program (MILAB).