Equiprobable mappings in weighted constraint grammars

TL;DR

This paper compares MaxEnt and stochastic HG frameworks in weighted constraint grammars, showing MaxEnt's ability to distinguish all mappings and characterizing when stochastic HG produces equiprobable mappings, with a case study on Finnish stress.

Contribution

It provides a formal characterization of equiprobable mappings in stochastic HG and contrasts this with MaxEnt's discriminative power.

Findings

MaxEnt can distinguish any two different mappings.

Stochastic HG admits equiprobable mappings, characterized formally.

Comparison on Finnish stress illustrates differences in predictions.

Abstract

We show that MaxEnt is so rich that it can distinguish between any two different mappings: there always exists a nonnegative weight vector which assigns them different MaxEnt probabilities. Stochastic HG instead does admit equiprobable mappings and we give a complete formal characterization of them. We compare these different predictions of the two frameworks on a test case of Finnish stress.

Tables

Table 1. Table 1: Constraints for Finnish vowel harmony

*Int[+back]:	No vowel between [+back] and
	right word edge
*Int[-back]:	No vowel between [-back] and
	right word edge
Ident-Root:	Be faithful to /a, ä/ in roots
Ident:	Be faithful to /a, ä/

Table 2. Table 2: Constraints for foot structure in Finnish nouns

FtBin	Feet are disyllabic.
PkProm	No unstressed light syllables.
Align-L	All feet left.
*Rev	No trochees with sonority reversal.
*Flat	No trochees with a flat sonority profile.
H.X	No stress next to a heavy syllable.
WSP	No unstressed heavy syllables.
WSP/VV	No unstressed heavies with long vowel.

Equations

$$\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z})=\left[\begin{array}[]{c}C_{1}(\mathsf{x},\mathsf{z})-C_{1}(\mathsf{x},\mathsf{y})\\[-2.84544pt] \vdots\\[-2.84544pt] C_{k}(\mathsf{x},\mathsf{z})-C_{k}(\mathsf{x},\mathsf{y})\\[-2.84544pt] \vdots\\[-2.84544pt] C_{n}(\mathsf{x},\mathsf{z})-C_{n}(\mathsf{x},\mathsf{y})\end{array}\right]$$

$$\text{C}(\hat{\text{x}},\hat{\text{y}},\hat{\text{z}})\ge \sum _{i=1}^m{\lambda }_i\text{C}(\mathsf{x},\mathsf{y},{\mathsf{z}}_i)$$

$${\lambda }_1+\cdots +{\lambda }_m=1$$

$$\sum _{i=1}^m{e}^{{\mathbf{w}}^{\mathsf{T}}{\text{c}}_i}=\sum _{j=1}^{\hat{m}}{e}^{{\mathbf{w}}^{\mathsf{T}}{\hat{\text{c}}}_j}$$

$$\underset{P}{\underbrace{conv({\text{c}}_1\dots {\text{c}}_m)+{}_{+}^n}}=\underset{\hat{P}}{\underbrace{conv({\hat{\text{c}}}_1\dots {\hat{\text{c}}}_{\hat{m}})+{}_{+}^n}}$$

Full text

1 Introduction

This paper compares two frameworks for probabilistic constraint-based phonology: Stochastic Harmonic Grammar (SHG; Boersma and Pater, 2016)111 Boersma and Pater Boersma and Pater (2016) actually use the term “noisy HG” instead of “stochastic HG”. We prefer “stochastic HG” to stress the complete analogy with Boersma’s Boersma (1997, 1998) earlier framework of stochastic OT. Furthermore, we prefer to use “stochastic” to describe a property of the framework, reserving “noisy” to describe a property of the learning scenario (as opposed to noise-free). and Maximum Entropy (ME; Goldwater and Johnson, 2003; Hayes and Wilson, 2008). Recent literature has documented a few realistic quantitative patterns which seem to admit a better fit in ME than in SHG Smith and Pater (2017); Zuraw and Hayes (2017); Hayes (2017). These findings suggest that ME is a richer probabilistic framework than SHG (relative to the same constraint set). But how much richer? Can these anecdotal observations reported in the literature be systematized into a principled formal comparison between SHG and ME probabilistic typologies? This paper is part of a larger project trying to address this question. In particular, this paper compares ME and SHG from the perspective of their equiprobable mappings. That is phonological mappings which are always assigned the same probability and are therefore phonologically equivalent despite being distinguished by the constraint set.

Section 2 motivates this notion of equiprobability within phonological theory. Section 4 then shows that the ME typology is so rich that it admits no equiprobable mappings: for any two mappings distinguished by the constraints, there exists an ME grammar that distinguishes between them, namely assigns them different probabilities. This typological richness is peculiar to ME and does not extend to other implementations of probabilistic constraint-based phonology such as SHG. Indeed, Section 5 shows that the equiprobable SHG mappings are exactly those mappings which are indistinguishable by categorical Harmonic Grammars (HG; Legendre et al., 1990a,b; Smolensky and Legendre, 2006) and thus provides a complete characterization of SHG equiprobability.

These formal results are presented informally. A detailed proof of the ME result is provided in a final appendix. The proof of the SHG result is analogous and it is omitted for reasons of space (see the longer version of this paper available on the authors' websites). Our discussion rests on some earlier results on uniform SHG and ME probability inequalities from Anttila and Magri Anttila and Magri (2018), recalled in Section 3.

Is the richness of ME relative to SHG typologies an empirical advantage or a case of unmotivated overgeneration? Section 6 provides some preliminary evidence that the latter might be the case, by looking at the case of Finnish stress. We compute SHG equiprobable mappings using the formal characterization obtained in Section 5. We show that a large corpus of Finnish provides preliminary empirical support for these mappings indeed being equiprobable. Finally, we show that ME breaks up these equiprobabilities in a way that is phonologically counterintuitive.

2 Equiprobability

A typical phonological process applies uniformly to all forms that share some relevant property, but ignores the irrelevant ways in which they differ. For example, in Latin, stress targets heavy syllables, but ignores vowel quality; in English, aspiration targets voiceless stops, but ignores place of articulation; in Finnish, vowel harmony targets [$\pm$back], but ignores the number of syllables. This means that words with the same distribution of heavy and light syllables are stressed alike; voiceless stops are aspirated alike; and words of any length harmonize alike. These phonological equivalences are a key property of phonological systems.

Derivational phonology captures these equivalences straightforwardly: phonological rules are allowed to refer to only the shared property that defines a natural class, ignoring everything else. To illustrate, the Finnish vowel harmony rule can be simply written as $\mbox{V}\rightarrow[\alpha\mbox{back}]/\mbox{V}[\alpha\mbox{back}]\mbox{C}_{0}\_\!\_$. This rule directly encodes the fact that harmony targets [$\pm$back] but ignores any other properties such as, say, the number of syllables. Thus, the monosyllabic /maa/ country' and the disyllabic /kaava/ formula' trigger back harmony on the suffix /-nä/ `essive' in exactly the same way. In other words, they are equivalent for vowel harmony.

The situation is prima facie less obvious in constraint-based phonology. A candidate may contain multiple constraint violations, some relevant, some irrelevant, but all simultaneously visible and potentially interacting. Yet, categorical implementations of constraint-based phonology are well known to readily predict these desired phonological equivalences. To illustrate, consider an HG grammar for Finnish vowel harmony based on the constraints in Table 1, from Ringen and Heinämäki (1999). The back harmony mappings $\textsf{/maa-n\"{a}/}\rightarrow\textsf{[maana]}$ and $\textsf{/kaava-n\"{a}/}\rightarrow\textsf{[kaavana]}$ can be shown to be HG equivalent: no matter the weighting, no HG grammar succeeds on one but fails on the other.

How should phonological equivalence be extended from the categorical to the probabilistic setting? We submit that equiprobability provides an answer to this question. In fact, let us recall that a probabilistic phonological grammar is a function which assigns to each underlying representation (UR) $\mathsf{x}$ a probability distribution $\mathbb{P}(\mathsf{y}\,|\,\mathsf{x})$ over the corresponding set of candidate surface representations (SRs) $\mathsf{y}$. We consider two mappings $(\mathsf{x},\mathsf{y})$ and $(\widehat{\textsf{x}},\widehat{\textsf{y}})$ of the two URs $\mathsf{x},\widehat{\textsf{x}}$ to the two SRs $\mathsf{y},\widehat{\textsf{y}}$. We say that these two mappings are (uniformly) equiprobable provided there is no probabilistic grammar in the typology considered which assigns a different probability to those two mappings, namely such that $\mathbb{P}(\mathsf{y}\,|\,\mathsf{x})\not=\mathbb{P}(\widehat{\textsf{y}}\,|\,\widehat{\textsf{x}})$. To illustrate, the equivalence between the two mappings $\textsf{/maa-n\"{a}/}\rightarrow\textsf{[maana]}$ and $\textsf{/kaava-n\"{a}/}\rightarrow\textsf{[kaavana]}$ is captured in a probabilistic setting through the requirement that their probabilities $\mathbb{P}(\textsf{[maana]}\,|\,\textsf{/maa-n\"{a}/})$ and $\mathbb{P}(\textsf{[kaavana]}\,|\,\textsf{/kaava-n\"{a}/})$ always coincide. In other words, the probability of vowel harmony does not depend on the number of syllables.222 Note that this is quite different from the well-known case of Hungarian vowel harmony where suffixes show different degrees of back-front variation after stems with both back and neutral vowels depending on the number of neutral vowels; see, e.g., Hayes and Londe (2006), Hayes et al. (2009), and Zymet (2015). In our Finnish example, all the stem vowels are unambiguously back, yet our Proposition 1 below says that ME fails to guarantee that the suffix harmony is invariably back.

As we will see in Section 5, two mappings are equivalent according to categorical HG if and only if they are equiprobable in SHG. This result suggests that equiprobability is indeed the right extension of the notion of phonological equivalence from the categorical to the probabilistic setting. Surprisingly, we will see in Section 4 that ME instead allows for no equiprobable mappings and thus fails to capture the notion of phonological equivalence.

3 Formal background

Our characterization of ME and SHG equiprobability in sections 4-5 rests on some results from Anttila and Magri (2018; A&M) recalled here.

HG

A weight vector $\mathbf{w}=(w_{1},\dots,w_{n})$ assigns nonnegative weights $w_{1},\dots,w_{n}\geq 0$ to $n$ underlying phonological constraints $C_{1},\dots,C_{n}$. The phonological quality of a phonological mapping $(\mathsf{x},\mathsf{y})$ of a UR $\mathsf{x}$ and a candidate SR $\mathsf{y}$ is quantified by its harmony $H_{\mathbf{w}}(\mathsf{x},\mathsf{y})$. This quantity is defined as the weighted sum of the constraint violations multiplied by $-1$, namely $H_{\mathbf{w}}(\mathsf{x},\mathsf{y})=-\sum_{k=1}^{n}w_{k}C_{k}(\mathsf{x},\mathsf{y})$. Mappings with large harmony have small constraint violations. The HG grammar corresponding to a weight vector $\mathbf{w}$ maps a UR $\mathsf{x}$ to the candidate SR $\mathsf{y}$ such that the mapping $(\mathsf{x},\mathsf{y})$ has a larger harmony than the mapping $(\mathsf{x},\mathsf{z})$ corresponding to any other candidate $\mathsf{z}$ of $\mathsf{x}$. In this case, we say that $\mathsf{y}$ is the winner while any other candidate $\mathsf{z}$ is a loser.

HG thus has an intrinsic comparative nature: absolute numbers of violations are irrelevant, what matters is only the comparison between the violations of the loser and those of the winner. To bring out this intuition, we define the difference vector $\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z})$ for a UR $\mathsf{x}$, an intended winner candidate $\mathsf{y}$, and an intended loser candidate $\mathsf{z}$ as in (1). This vector has a component for each constraint $C_{k}$ defined as the difference between the number $C_{k}(\mathsf{x},\mathsf{z})$ of violations assigned by $C_{k}$ to the loser mapping $(\mathsf{x},\mathsf{z})$ minus the number $C_{k}(\mathsf{x},\mathsf{y})$ of violations assigned to the winner mapping $(\mathsf{x},\mathsf{y})$.

[TABLE]

SHG and ME are two probabilistic extensions of this underlying categorical HG model.

SHG

The SHG probability $\mathbb{P}_{\mathbf{w}}^{\mbox{SHG}}(\mathsf{y}\,|\,\mathsf{x})$ that a UR $\mathsf{x}$ is mapped to a SR $\mathsf{y}$ according to the weight vector $\mathbf{w}$ is the probability of sampling $n$ numbers $\boldsymbol{\epsilon}=(\epsilon_{1},\dots,\epsilon_{n})$ independently according to a distribution $\mathcal{D}$ in such a way that the HG grammar corresponding to the weight vector $\mathbf{w}+\boldsymbol{\epsilon}=(w_{1}+\epsilon_{1},\dots,w_{n}+\epsilon_{n})$ indeed maps $\mathsf{x}$ to $\mathsf{y}$. A&M prove the following Lemma 1 about uniform probability inequalities in SHG, namely inequalities which hold for every choice of the weight vector.

Lemma 1

Consider two mappings $(\mathsf{x},\mathsf{y})$ and $(\widehat{\textsf{x}},\widehat{\textsf{y}})$. Assume that the UR $\mathsf{x}$ comes with only a finite number $m$ of loser candidates $\mathsf{z}_{1},\dots,\mathsf{z}_{m}$ (besides the winner candidate $\mathsf{y}$) and that the mapping $(\mathsf{x},\mathsf{y})$ is possible in HG (namely, $\mathsf{y}$ beats the losers $\mathsf{z}_{1},\dots,\mathsf{z}_{m}$ relative to some nonnegative weight vector). The SHG probability inequality $\mathbb{P}_{\mathbf{w}}^{\mbox{SGH}}(\mathsf{y}\,|\,\mathsf{x})\leq\mathbb{P}_{\mathbf{w}}^{\mbox{SGH}}(\widehat{\textsf{y}}\,|\,\widehat{\textsf{x}})$ holds uniformly for every choice of the nonnegative weight vector $\mathbf{w}$ if and only if for every loser candidate $\widehat{\textsf{z}}$ of the UR $\widehat{\textsf{x}}$, there exist $m$ nonnegative coefficients $\lambda_{1},\dots,\lambda_{m}\geq 0$ (one for each loser candidate $\mathsf{z}_{1},\dots,\mathsf{z}_{m}$ of the UR $\mathsf{x}$) such that

[TABLE]

*namely the difference vector $\textbf{C}(\widehat{\textsf{x}},\widehat{\textsf{y}},\widehat{\textsf{z}})$ is at least as large (constraint by constraint) as the sum of the difference vectors $\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})$ each rescaled by a corresponding nonnegative coefficient $\lambda_{i}$.333 The two assumptions made by the lemma—-that the UR $\mathsf{x}$ comes with only a finite number of losers and that the mapping $(\mathsf{x},\mathsf{y})$ is possible in HG—-are non-restrictive. In fact, if a mapping $(\mathsf{x},\mathsf{y})$ is impossible in HG, then its SHG probability $\mathbb{P}_{\mathbf{w}}^{\mbox{SGH}}(\mathsf{y}\,|\,\mathsf{x})$ can be shown to be equal to zero for every choice of the nonnegative weight vector $\mathbf{w}$. The probability inequality $\mathbb{P}_{\mathbf{w}}^{\mbox{SGH}}(\mathsf{y}\,|\,\mathsf{x})\leq\mathbb{P}_{\mathbf{w}}^{\mbox{SGH}}(\widehat{\textsf{y}}\,|\,\widehat{\textsf{x}})$ thus holds uniformly, because its left hand side is always equal to zero. The assumption made by the lemma that the mapping $(\mathsf{x},\mathsf{y})$ is possible in HG is therefore non-restrictive. Furthermore, HG has the property that only a finite number of candidates of any given UR win according to some weights Magri (2019). All other candidates are redundant because impossible no matter how the weights are chosen. Since HG impossible mappings have zero SHG probability, the candidate set of any underlying form can always be assumed to be finite without loss of generality in SHG. The assumption made by the lemma that the UR $\mathsf{x}$ comes with only a finite number of losers is therefore non-restrictive.

$\Box$*

Lemma 1 admits the following geometric interpretation, which will be used below. Suppose there are only $n=2$ constraints and $m=4$ losers $\mathsf{z}_{i}$. The difference vectors $\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})$ which appear on the right hand side of (2) can therefore be represented as the four black dots in Fig. 1. The region $\{\sum_{i=1}^{m}\lambda_{i}\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})\,|\,\lambda_{i}\geq 0\}$ is the convex cone generated by these four difference vectors $\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})$, depicted in dark gray in Fig. 1a. The region in light gray singles out the points which are at least as large as some point in this cone. Condition (2) thus says that the difference vector $\textbf{C}(\widehat{\textsf{x}},\widehat{\textsf{y}},\widehat{\textsf{z}})$ belongs to this light gray region.

ME

The ME probability $\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\mathsf{y}\,|\,\mathsf{x})$ that a UR $\mathsf{x}$ is mapped to a SR $\mathsf{y}$ according to a nonnegative weight vector $\mathbf{w}$ is the exponential of the harmony $H_{\mathbf{w}}(\mathsf{x},\mathsf{y})$ of that mapping, normalized through a constant $Z=Z(\mathbf{w},\mathsf{x})$, namely $\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\mathsf{y}\,|\,\mathsf{x})=e^{H_{\mathbf{w}}(\mathsf{x},\mathsf{y})}/Z$. A&M show that also in ME uniform probability inequalities can be characterized in terms of difference vectors, as stated by Lemma 2 below. This ME Lemma is analogous to the SHG Lemma 1 above, but for two differences. The first difference is that condition (2) is only necessary in ME while it is also sufficient in SHG. The second difference is that ME requires the normalization condition (3) on the coefficients $\lambda_{i}$.

Lemma 2

Consider two mappings $(\mathsf{x},\mathsf{y})$ and $(\widehat{\textsf{x}},\widehat{\textsf{y}})$. Assume that the UR $\mathsf{x}$ comes with a finite number $m$ of loser candidates $\mathsf{z}_{1},\dots,\mathsf{z}_{m}$ (besides the winner candidate $\mathsf{y}$). If the ME probability inequality $\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\mathsf{y}\,|\,\mathsf{x})\leq\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\widehat{\textsf{y}}\,|\,\widehat{\textsf{x}})$ holds uniformly for every choice of the nonnegative weight vector $\mathbf{w}$, then for every loser candidate $\widehat{\textsf{z}}$ of the UR $\widehat{\textsf{x}}$, there exist $m$ nonnegative coefficients $\lambda_{1},\dots,\lambda_{m}\geq 0$ (one for each loser candidate $\mathsf{z}_{1},\dots,\mathsf{z}_{m}$ of the UR $\mathsf{x}$) which add up to 1

[TABLE]

and furthermore satisfy condition (2). $\Box$

The normalization condition (3) admits the following geometric interpretation. As seen above, the region $\{\sum_{i}\lambda_{i}\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})\,|\,\lambda_{i}\geq 0\}$ is the convex cone generated by the difference vectors $\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})$, represented by the dark gray region in Fig. 1a. The smaller region \{\sum_{i}\lambda_{i}\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})\,|\,\lambda_{i}\geq 0,\framebox{\sum_{i}\lambda_{i}=1}\}, which differs for the (boxed) normalization condition (3) on the coefficients $\lambda_{i}$, is instead the convex hull generated by the difference vectors $\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})$, represented by the smaller dark gray region in Fig. 1b. The effect of the normalization condition (3) is thus to shrink from the larger convex cone to the smaller convex hull. Finally, the region in light gray in Fig. 1b singles out the points which are at least as large as some point in this convex hull. Lemma 2 thus requires the difference vector $\textbf{C}(\widehat{\textsf{x}},\widehat{\textsf{y}},\widehat{\textsf{z}})$ to belong to this light gray region.

4 ME has no equiprobable mappings

Lemmas 1 and 2 say that ME differs from SHG because of the normalization condition (3). This apparently small technical difference has substantial phonological implications. Indeed, this Section shows that the normalization condition (3) makes the ME typology so rich that it can distinguish between any two mappings. In other words, equiprobability is impossible in ME. The reasoning is presented here informally, split up into three steps formalized in the final appendix.

Step 1

Let us suppose that the two mappings $(\mathsf{x},\mathsf{y})$ and $(\widehat{\textsf{x}},\widehat{\textsf{y}})$ are equiprobable in ME, namely that the ME probability identity $\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\mathsf{y}\,|\,\mathsf{x})=\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\widehat{\textsf{y}}\,|\,\widehat{\textsf{x}})$ holds for every choice of the nonnegative weight vector $\mathbf{w}$. Let $\mathsf{z}_{1},\dots,\mathsf{z}_{m}$ be the loser candidates of the UR $\mathsf{x}$. They define a light gray region as in Fig. 1b, namely the region of points which are at least as large as the points in the convex hull generated by the difference vectors $\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})$. Let us denote this light gray region as $\mbox{\em LGR}^{\mbox{ME}}(\mathsf{z}_{1},\dots,\mathsf{z}_{m})$. Analogously, let $\widehat{\textsf{z}}_{1},\dots,\widehat{\textsf{z}}_{\widehat{m}}$ be the loser candidates of the other UR $\widehat{\textsf{x}}$. They as well define the light gray region of points which are at least as large as the points in the convex hull generated by the difference vectors $\textbf{C}(\widehat{\textsf{x}},\widehat{\textsf{y}},\widehat{\textsf{z}}_{j})$. Let us denote this light gray region as $\mbox{\em LGR}^{\mbox{ME}}(\widehat{\textsf{z}}_{1},\dots,\widehat{\textsf{z}}_{\widehat{m}})$.

The probability identity $\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\mathsf{y}\,|\,\mathsf{x})=\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\widehat{\textsf{y}}\,|\,\widehat{\textsf{x}})$ is equivalent to the two reverse inequalities $\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\mathsf{y}\,|\,\mathsf{x})\leq\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\widehat{\textsf{y}}\,|\,\widehat{\textsf{x}})$ and $\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\mathsf{y}\,|\,\mathsf{x})\geq\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\widehat{\textsf{y}}\,|\,\widehat{\textsf{x}})$. By lemma 2 above, the former inequality requires each difference vector $\textbf{C}(\widehat{\textsf{x}},\widehat{\textsf{y}},\widehat{\textsf{z}}_{j})$ to belong to $\mbox{\em LGR}^{\mbox{ME}}(\mathsf{z}_{1},\dots,\mathsf{z}_{m})$. And the latter inequality requires each difference vector $\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})$ to belong to $\mbox{\em LGR}^{\mbox{ME}}(\widehat{\textsf{z}}_{1},\dots,\widehat{\textsf{z}}_{\widehat{m}})$. A simple convexity argument deduces from these two facts the identity $\mbox{\em LGR}^{\mbox{ME}}(\mathsf{z}_{1},\dots,\mathsf{z}_{m})=\mbox{\em LGR}^{\mbox{ME}}(\widehat{\textsf{z}}_{1},\dots,\widehat{\textsf{z}}_{\widehat{m}})$ between the two light gray regions.

Step 2

To proceed, let us suppose for concreteness that $m=4$ and that the light gray region $\mbox{\em LGR}^{\mbox{ME}}(\mathsf{z}_{1},\mathsf{z}_{2},\mathsf{z}_{3},\mathsf{z}_{4})$ is the one plotted in light gray in Fig. 1b. The difference vectors corresponding to the two losers $\mathsf{z}_{1}$ and $\mathsf{z}_{2}$ are extreme points (or vertices) of this light gray region. In the sense that they crucially contribute to shape it: if these two points were shifted even slightly in any direction, the corresponding light gray region would change. The identity between the two light gray regions established in step 1 thus entails that the two light gray regions share the same set of extreme points. In conclusion, the two difference vectors corresponding to losers $\mathsf{z}_{1}$ and $\mathsf{z}_{2}$ which are extreme points of the light gray region in figure Fig. 1b must be shared by the two equiprobable mappings considered. Since these difference vectors are shared by the two equiprobable mappings, they can be ``peel off'' the two sides of the ME probability identity.

Step 3

We are thus left with the difference vectors corresponding to the other two losers $\mathsf{z}_{3}$ and $\mathsf{z}_{4}$ in Fig. 1b. These latter two vectors are not extreme points of the original light gray region but rather sit in the interior of the light gray region. Indeed, they can be shifted around without affecting the shape of the light gray region. Yet, once the two losers $\mathsf{z}_{1}$ and $\mathsf{z}_{2}$ have been ``peeled off'' at step 2, we can repeat the reasoning in steps 1 and 2 ignoring the two losers $\mathsf{z}_{1}$ and $\mathsf{z}_{2}$ and instead considering only the other two losers $\mathsf{z}_{3}$ and $\mathsf{z}_{4}$.

Thus, we construct the convex hull of the difference vectors corresponding to just these two remaining losers $\mathsf{z}_{3}$ and $\mathsf{z}_{4}$. This convex hull is the segment which connects the two corresponding dots. Next, we construct the light gray region of points which are at least as large as some point in that segment, as depicted in Fig. 2. Now the difference vectors corresponding to the two losers $\mathsf{z}_{3}$ and $\mathsf{z}_{4}$ are extreme points of the new light gray region. We can therefore repeat the reasoning in steps 1-2 and conclude that these two difference vectors as well must be shared by the two equiprobable mappings considered. And so on.

The reasoning informally sketched above leads to the following Proposition 1, which is the first main result of this paper. It says that two mappings are equiprobable in ME if and only if they share all difference vectors. This entails in particular that the two mappings must have the same number of loser candidates. In other words, the ME typology is so rich that the only case where ME fails to come up with at least one weight vector which assigns different probabilities to the two mappings $(\mathsf{x},\mathsf{y})$ and $(\widehat{\textsf{x}},\widehat{\textsf{y}})$ is when the two mappings are the same mapping, in the sense that they are indistinguishable by the constraints, as they have the same difference vectors.444 To illustrate, suppose that the constraint set only consists of the two constraints NoVoicedObstruent and Ident(voice). The mappings $(\mathsf{x},\mathsf{y})=(\textsf{/mab/},\textsf{[map]})$ and $(\widehat{\textsf{x}},\widehat{\textsf{y}})=(\textsf{/bam/},\textsf{[pam]})$ will always have the same ME probability, because they and their losers have the same constraint violation profiles.

Proposition 1

Two mappings $(\mathsf{x},\mathsf{y})$ and $(\widehat{\textsf{x}},\widehat{\textsf{y}})$ are equiprobable in ME if and only if the corresponding sets of difference vectors coincide. $\Box$

5 SHG allows for equiprobable mappings

The preceding Section has shown that ME is so rich that it can distinguish between any two different mappings. Crucially, this typological richness is peculiar to ME, not intrinsic to probabilistic constraint-based phonology. In this section, we illustrate this point with the case of SHG. As in the preceding section, the discussion is kept informal. The formalization rests on the same convex geometric tools used for ME in the final appendix. The details are omitted here for reasons of space (see the longer version of this paper available on the authors' website).

Let us consider two mappings $(\mathsf{x},\mathsf{y})$ and $(\widehat{\textsf{x}},\widehat{\textsf{y}})$. Again, let $\mathsf{z}_{1},\dots,\mathsf{z}_{m}$ be the loser candidates of the UR $\mathsf{x}$. They define a light gray region as in Fig. 1a, namely the region of points which are at least as large as the points in the convex cone generated by the difference vectors $\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})$. Let us denote this light gray region as $\mbox{\em LGR}^{\mbox{SHG}}(\mathsf{z}_{1},\dots,\mathsf{z}_{m})$. This region is different from (and larger than) the light gray region $\mbox{\em LGR}^{\mbox{ME}}(\mathsf{z}_{1},\dots,\mathsf{z}_{m})$ considered above for ME, because the latter ME region is restricted through the normalization condition (3) and therefore defined in terms of convex hulls rather than convex cones. Analogously, let $\widehat{\textsf{z}}_{1},\dots,\widehat{\textsf{z}}_{\widehat{m}}$ be the loser candidates of the other UR $\widehat{\textsf{x}}$ and let $\mbox{\em LGR}^{\mbox{SHG}}(\widehat{\textsf{z}}_{1},\dots,\widehat{\textsf{z}}_{\widehat{m}})$ be the corresponding SHG light gray region.

Again as in the case of ME, Lemma 1 says that the uniform SHG probability identity $\mathbb{P}_{\mathbf{w}}^{\mbox{SHG}}(\mathsf{y}\,|\,\mathsf{x})=\mathbb{P}_{\mathbf{w}}^{\mbox{SHG}}(\widehat{\textsf{y}}\,|\,\widehat{\textsf{x}})$ entails that the two SHG light gray regions coincide, namely that $\mbox{\em LGR}^{\mbox{SHG}}(\mathsf{z}_{1},\dots,\mathsf{z}_{m})=\mbox{\em LGR}^{\mbox{SHG}}(\widehat{\textsf{z}}_{1},\dots,\widehat{\textsf{z}}_{\widehat{m}})$. Yet, these SHG light gray regions have different geometric properties than the ME light gray regions. As a result, in the case of SHG the identity between the two light gray regions tells us much less about the difference vectors that generate them than in the case of ME.

To see that concretely, let us consider for instance the SHG light gray region in Fig. 1a. The loser candidates $\mathsf{z}_{2},\mathsf{z}_{3}$ and $\mathsf{z}_{4}$ have difference vectors which sit in the interior of this light gray region. These losers thus contribute nothing to shape the light gray region: their difference vectors can be shifted around without affecting the shape of the region. Identity of the light gray regions thus tells us nothing about identity of these difference vectors which sit in the interior.

Interestingly, the loser candidates whose difference vectors sit in the interior of the SHG light gray region can be characterized phonologically as those losers which are HG redundant given the rest of the losers. In the sense that, for every nonnegative weight vector $\mathbf{w}$, if the HG harmony of the winner $\mathsf{y}$ is larger than that of the nonredundant losers, then it is in particular larger than the harmony of the redundant losers. In other words, these redundant losers carry no interesting phonological content as they do not in any way affect the weight vectors consistent with the mapping $(\mathsf{x},\mathsf{y})$.

The case of the loser $\mathsf{z}_{1}$ in Fig. 1a is instead different. Its difference vector sits on the border of the light gray region and therefore contributes to its shape. Yet, its position is not completely determined by the shape of the region. In fact, the shape of the region is not affected if this difference vector is slid closer to or further away from the origin. Equivalently, the shape of the region is not affected if the difference vector corresponding to the nonredundant loser $\mathsf{z}_{1}$ is rescaled by a nonnegative constant $\lambda\geq 0$. This means that the identity of the two SHG light gray regions does not entail identity of the difference vectors which generate them, not even for those difference vectors which sit on the boundary of the regions and therefore correspond to nonredundant losers. The identity of the two SHG light gray regions only entails that the difference vectors of the nonredundant losers are one the rescaling of the other. This informal reasoning leads to the following Proposition, which is our second main result.

Proposition 2

Two mappings $(\mathsf{x},\mathsf{y})$ and $(\widehat{\textsf{x}},\widehat{\textsf{y}})$ are equiprobable in SHG if and only if each nonredundant difference vector $\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})$ is a rescaling of some nonredundant difference vector $\textbf{C}(\widehat{\textsf{x}},\widehat{\textsf{y}},\widehat{\textsf{z}}_{j})$, namely $\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})=\lambda\textbf{C}(\widehat{\textsf{x}},\widehat{\textsf{y}},\widehat{\textsf{z}}_{j})$ for some $\lambda\geq 0$; analogously, each nonredundant difference vector $\textbf{C}(\widehat{\textsf{x}},\widehat{\textsf{y}},\widehat{\textsf{z}}_{j})$ is a rescaling of some nonredundant difference vector $\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})$. $\Box$

Interestingly, this characterization of SHG equiprobability coincides with the characterization of equivalence in categorical HG obtained by A&M. We conclude that two mappings are equiprobable in SHG (namely are always assigned the same probability) if and only if they are equivalent in categorical HG (namely no HG grammar succeeds on one but fails on the other).

6 Equiprobability in Finnish stress

This section brings the preceding formal results to bear on Finnish word stress.

The phonological system

The basic generalizations about Finnish word stress can be stated as follows Carlson (1978); Hanson and Kiparsky (1996); Elenbaas (1999); Elenbaas and Kager (1999); Karvonen (2005): (a) primary stress falls on the initial syllable; (b) secondary stress falls on every other syllable after that, (c) except that a light syllable is skipped if the syllable after that is heavy, unless the heavy syllable is final. Examples are íl.moit.tàu.tu.mì.nen registering' and íl.moit.tàu.tu.mi.sès.ta from registering'.

However, the skipping clause turns out to be a coarse approximation of the actual facts. Skipping is sometimes optional and we find variable stress in cases like pró.fes.so.rìl.la$\sim$pró.fes.sò.ril.la `professor-ade', where the basic rule fails at the second variant. This optional pattern turns out to depend on two additional conditions that affect the outcome in a gradient manner Anttila (2012): (a) low vowels (/a, ä, o, ö/) attract stress and high vowels (/e, i, u, y/) repel stress; (b) stress is avoided next to a heavy syllable.555 The categories “low” and “high” are morphophonemic, not phonetic. In Finnish, low vowels alternate morphophonologically with rounded mid vowels (a $\sim$ o, ä $\sim$ ö) and the unrounded high vowel alternates with the unrounded mid vowel (i $\sim$ e). For this reason we consider o, ö low and e high.

In addition to native speaker intuitions about syllable prominence, empirical support for these soft conditions can be obtained from the optional rule of Stop Deletion Keyser and Kiparsky (1984) which deletes singleton stops in extrametrical syllables Anttila (2012). In particular, the /t/ in the partitive suffix /-tA/ is deleted vs. retained depending on the location of secondary stress feet. Given the input /professori-i-tA/ `professor-pl-par' we have two possible foot structures: (pró.fes.so)(rèi.ta) where /t/ falls inside a foot and is retained vs. (pró.fes)(sò.re)ja where /t/ falls outside a foot and is deleted. The metrical free variation is thus reflected in segmental free variation. This provides a valuable diagnostic for foot structure, especially because the segmental variation is present even in the written standard language readily available in large quantities.

The constraints necessary for deriving the foot structure in Finnish nouns are shown in Table 2. These constraints were applied to 48 types of partitive plural nouns, systematically varying stem length, syllable weight, and vowel sonority. All in all, the test set contains 4 types of three-syllable stems, 12 types of 4-syllable stems, and 32 types 5-syllable stems (stem types are briefly denoted as ``(a), (b), …'' in what follows).

SHG

We computed the uniform probability inequalities predicted by SHG for this Finnish stress test case using $\mathbb{C}$o$\mathbb{G}$e$\mathbb{T}$o Magri and Anttila (2019), a suite of tools for studying constraint-based typologies of categorical and probabilistic phonological grammars based on their underlying rich convex geometry. The key observation is that SHG predicts seven blocks of equiprobable mappings, shown in Table 3. These blocks are furthermore organized into two chains of uniform probability inequalities. The predicted probabilities increase from left to right. The symbol ``$\leq$" between two boxes means that the candidates in the box on the left are predicted to have a probability at most as large as the candidates in the box on the right.

To evaluate the empirical accuracy of the equiprobabilities predicted by SHG, we examined Finnish /t/-deletion in a corpus of approximately 9 million nouns (tokens) harvested from Finnish internet sites on April 12, 2005. The percentages reported in Table 3 represent the token frequency of /t/-retention vs. /t/-deletion variants for each phonologically distinct stem type. The corpus data are consistent with the equiprobability prediction in five out of the seven blocks, namely those in black. These blocks turn out to be empirically nearly categorical, with almost all stems undergoing either /t/-deletion or /t/-retention, consistently with the equiprobability prediction.

However, the two red blocks in Table 3 bundle together the stem types (c)-(f) despite them showing rather different empirical frequencies, providing prima facie evidence against SHG's equiprobability prediction. The stem types are illustrated by /symposiumi/ symposium', /polyamidi/ polyamide', /liirumlaarumi/ nonsense', and /inkunaabeli/ incunable'. The stems differ in the weight and quality of the preantepenultimate and antepenultimate syllables (heavy vs. light, $[+\mbox{low}]$ vs. $[-\mbox{low}]$), which results in constraint violation differences, yet HG predicts that all four should undergo /t/-deletion/retention at identical rates. In order to reconcile SHG's equiprobability predicitions with corpus frequencies, we make the following observations. First, the difference between types (d) /liirumlaarumi/ and (f) /inkunaabeli/ is not statistically significant ($\chi^{2}$ = 2.9849, df = 1, p = 0.08404). Second, type (c) contains only two stems: /symposiumi/ symposium' and /imperiumi/ empire', both potentially syllabifiable as four-syllable stems, e.g., im.pe.ri.u.mi $\sim$ im.pe.riu.mi Anttila and Shapiro (2017), which is consistent with their unexpectedly high /t/-deletion rate. This leaves us with type (e) /polyamidi/ `polyamide' (N = 69), again with an unexpectedly high deletion rate for which we have no plausible explanation. We conclude that by and large our Finnish corpus data support SHG's equiprobability predictions.

ME

One might wonder whether ME with its ability to make fine-grained distinctions might actually offer a more principled solution to the difficulties just discussed. This turns out not to be the case. On the retention side, ME predicts the uniform probability inequalities in the top row of Table 4. For example, the retention probability of /polyamidi/ is predicted to be at most as high as that of /liirumlaarumi/, no matter the choice of the weight vector. That seems initially promising: these inequalities are in fact exactly what we observe in the data. Puzzlingly, on the deletion side, ME reverses the probabilities, yielding the uniform probability inequalities in the bottom row of Table 4. For example, the deletion probability of /polyamidi/ is predicted to be at most as high as that of /liirumlaarumi/. This is exactly the opposite of what we observe in the data. We submit there is simply no way to reconcile ME's predictions with the corpus data. Such counterintuitive probability reversals appear in other blocks as well.

7 Summary and conclusions

We have shown that ME predicts typologies so rich that ME grammars can distinguish between any two different mappings and therefore admit no equiprobable mappings (Proposition 1). This richness does not extend to other implementations of probabilistic constraint-based phonology, such as SHG (Proposition 2), revealing a fundamental difference between the two frameworks.

We have then applied these results to the test case of Finnish word stress. Our corpus data provide preliminary evidence in favor of SHG's equiprobability predictions. In the two blocks where SHG appeared to run into problems, ME did not help refine the analysis empirically, but instead split the SHG equiprobable stem types apart in a counterintuitive fashion. Our study thus provides some preliminary empirical support in favor of SHG, which permits equiprobable mappings, against ME, which does not.

Acknowledgements

For very useful discussion, we would like to thank the participants of the workshop Analyzing typological structure: from categorical to probabilistic phonology (Stanford, September 2018; website). The research reported in this paper has been supported by a Collaborative Projects Grant from the France-Stanford Center for Interdisciplinary Studies (project title: The Mathematics of Language Universals) as well as by a JCJC grant from the Agence Nationale de la Recherche (project title: The mathematics of segmental phonotactics).

Appendix A Proof of Proposition 1

We write $\textbf{c}_{i}$ and $\widehat{\textbf{c}}_{j}$ as shorthands for the difference vectors $\textbf{C}(\mathsf{x},\mathsf{y},\mathsf{z}_{i})$ and $\textbf{C}(\widehat{\textsf{x}},\widehat{\textsf{y}},\widehat{\textsf{z}}_{j})$ corresponding to the losers $\mathsf{z}_{i}$ and $\widehat{\textsf{z}}_{j}$. The ME probability inequality $\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\mathsf{x},\mathsf{y})=\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\widehat{\textsf{x}},\widehat{\textsf{y}})$ can be made explicit as in (4) through some elementary manipulations. As usual, $\textbf{a}^{\sf T}\textbf{b}$ denotes the scalar product of a and b.

[TABLE]

Once the ME probability identity $\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\mathsf{x},\mathsf{y})=\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\widehat{\textsf{x}},\widehat{\textsf{y}})$ is made explicit as in (4), it is obvious that it holds uniformly for every weight vector $\mathbf{w}$ when the two sets of difference vectors coincide, namely $\{\textbf{c}_{1},\dots,\textbf{c}_{m}\}=\{\widehat{\textbf{c}}_{1},\dots,\widehat{\textbf{c}}_{\widehat{m}}\}$. To complete the proof of Proposition 1, we thus only have to prove the reverse. We split the proof into three steps, corresponding to those in Section 4.

Step 1. We start from the assumption that the ME probability identity $\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\mathsf{x},\mathsf{y})=\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\widehat{\textsf{x}},\widehat{\textsf{y}})$ holds uniformly. This means in particular that the probability inequality $\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\mathsf{x},\mathsf{y})\leq\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\widehat{\textsf{x}},\widehat{\textsf{y}})$ holds uniformly. The necessary condition for this uniform ME inequality provided by Proposition 2 can be rewritten as the inclusion (A). As usual, + is the set of nonnegative real numbers and $A+B=\{\textbf{a}+\textbf{b}\,|\,\textbf{a}\in A,\textbf{b}\in B\}$ is the vector sum of two sets $A$ and $B$ of n. The region on the right hand side of (A) is the light gray region in Fig. 3.b. {exe} \ex $\displaystyle\{\widehat{\textbf{c}}_{1},\dots,\widehat{\textbf{c}}_{\widehat{m}}\}\subseteq conv(\textbf{c}_{1},\dots,\textbf{c}_{m})+_{+}^{n}$

The set $conv(\textbf{c}_{1},\dots,\textbf{c}_{m})+_{+}^{n}$ on the right hand side of (A) is convex because the two sets $conv(\textbf{c}_{1},\dots,\textbf{c}_{m})$ and ${}_{+}^{n}$ are both convex and the sum of two convex sets is convex (Boyd and Vandenberghe, 2004, Section 2.3.2). The inclusion (A) thus extends from the points $\widehat{\textbf{c}}_{1},\dots,\widehat{\textbf{c}}_{\widehat{m}}$ to their convex hull $conv(\widehat{\textbf{c}}_{1},\dots,\widehat{\textbf{c}}_{\widehat{m}})$, yielding the inclusion $conv(\widehat{\textbf{c}}_{1},\dots,\widehat{\textbf{c}}_{\widehat{m}})\subseteq conv(\textbf{c}_{1},\dots,\textbf{c}_{m})+_{+}^{n}$. Finally, by adding ${}_{+}^{n}$ at both sides, the latter inclusion entails $conv(\widehat{\textbf{c}}_{1},\dots,\widehat{\textbf{c}}_{\widehat{m}})+_{+}^{n}\subseteq conv(\textbf{c}_{1},\dots,\textbf{c}_{m})+_{+}^{n}$. Analogously, the reverse probability inequality $\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\widehat{\textsf{x}},\widehat{\textsf{y}})\leq\mathbb{P}_{\mathbf{w}}^{\mbox{ME}}(\mathsf{x},\mathsf{y})$ requires the reverse inclusion $conv(\textbf{c}_{1},\dots,\textbf{c}_{m})+_{+}^{n}\subseteq conv(\widehat{\textbf{c}}_{1},\dots,\widehat{\textbf{c}}_{\widehat{m}})+_{+}^{n}$, yielding (5).

[TABLE]

Step 2. This identity (5) says in particular that the two sets $P$ and $\widehat{P}$ on its left and right hand side have the same set of extreme points, namely $ext(P)=ext(\widehat{P})$. The set $ext(P)$ of extreme points of the set $P$ is nonempty. In fact, a set which is closed, convex, nonempty, and does not contain a line admits at least an extreme point (Bertsekas, 2009, Proposition 2.1.2). Indeed, $P$ is closed, because $conv(\textbf{c}_{1},...,\textbf{c}_{m})$ is compact, ${}_{+}^{n}$ is closed, and the sum of a compact set with a closed set is closed (Bertsekas, 2009, Section 1.3). Furthermore, $P$ is convex, because $conv(\textbf{c}_{1},...,\textbf{c}_{m})$ and ${}_{+}^{n}$ are both convex and the sum of two convex sets is convex. Finally, $P$ is obviously nonempty and it does not contain a line.

The set $ext(P)$ of extreme points of the set $P$ is a subset of the set of difference vectors $\{\textbf{c}_{1},\dots,\textbf{c}_{m}\}$. In fact, the set of extreme points of the finitely generated polyhedron $conv(\textbf{c}_{1},\dots,\textbf{c}_{m})$ is a subset of $\{\textbf{c}_{1},\dots,\textbf{c}_{m}\}$ (by the Krein-Milman theorem). The set of extreme points of the pointed cone ${}_{+}^{n}$ only consists of the zero vector $\mathbf{0}$. And the set $ext(A+B)$ of extreme points of the vector sum $A+B$ of any two polyhedra $A$ and $B$ is a subset of the vector sum $ext(A)+ext(B)$ of the two sets $ext(A)$ and $ext(B)$ of extreme points of $A$ and $B$, namely $ext(A+B)\subseteq ext(A)+ext(B)$ (Bertsimas and Tsitsiklis, 1997, exercise 2.22). Analogously, the set $ext(\widehat{P})$ of extreme points of the set $\widehat{P}$ is a nonempty subset of the set $\{\widehat{\textbf{c}}_{1},\dots,\widehat{\textbf{c}}_{m}\}$.

In conclusion, the two sets of difference vectors $\{\textbf{c}_{1},\dots,\textbf{c}_{m}\}$ and $\{\widehat{\textbf{c}}_{1},\dots,\widehat{\textbf{c}}_{m}\}$ share the vectors in the nonempty set $\Omega=ext(P)=ext(\widehat{P})$. Without loss of generality, we assume that these shared vectors are those corresponding to the first $h\geq 1$ losers, so that $\{\textbf{c}_{1},\dots,\textbf{c}_{m}\}=\Omega\cup\{\textbf{c}_{h+1},\dots,\textbf{c}_{m}\}$ and $\{\widehat{\textbf{c}}_{1},\dots,\widehat{\textbf{c}}_{m}\}=\Omega\cup\{\widehat{\textbf{c}}_{h+1},\dots,\widehat{\textbf{c}}_{\widehat{m}}\}$.

Step 3. The terms on the left and the right hand side of the ME probability identity (4) which correspond to the shared difference vectors in $\Omega$ cancel out. The ME probability identity thus reduces to $\sum_{i=h+1}^{m}e^{\mathbf{w}^{\sf T}\textbf{c}_{i}}=\sum_{j=h+1}^{\widehat{m}}e^{\mathbf{w}^{\sf T}\widehat{\textbf{c}}_{j}}$, where the sums start at $h+1$ rather than at $1$. The claim follows by iterating the reasoning above, starting from the latter simplified ME probability identity.