When redundancy is useful: A Bayesian approach to 'overinformative' referring expressions

TL;DR

This paper introduces a Bayesian rational speech act model with non-deterministic semantics to explain why speakers often produce seemingly overinformative referring expressions, showing that such redundancy can be rational and beneficial.

Contribution

It presents a novel Bayesian model that accounts for overinformativeness in referring expressions using continuous semantics, unifying diverse phenomena under a rational framework.

Findings

Overinformativeness is rational and beneficial.

The model explains asymmetries in overmodification.

Redundancy in reference enhances communicative effectiveness.

Abstract

Referring is one of the most basic and prevalent uses of language. How do speakers choose from the wealth of referring expressions at their disposal? Rational theories of language use have come under attack for decades for not being able to account for the seemingly irrational overinformativeness ubiquitous in referring expressions. Here we present a novel production model of referring expressions within the Rational Speech Act framework that treats speakers as agents that rationally trade off cost and informativeness of utterances. Crucially, we relax the assumption that informativeness is computed with respect to a deterministic Boolean semantics, in favor of a non-deterministic continuous semantics. This innovation allows us to capture a large number of seemingly disparate phenomena within one unified framework: the basic asymmetry in speakers' propensity to overmodify with color rather than size; the increase in overmodification in complex scenes; the increase in overmodification with atypical features; and the increase in specificity in nominal reference as a function of typicality. These findings cast a new light on the production of referring expressions: rather than being wastefully overinformative, reference is usefully redundant.

Tables

Table 1. Table 1 : List of effects a theory of referring expression production should account for and paper section(s) in which they are treated.

Section	Effect	Description
2 & 3	Color/size asymmetry	More redundant use of color than size 222Reported by many <e.g.,¿Pechmann1989, Engelhardt2006, gatt2011, rubiofernandez2016
2 & 3	Scene variation	More redundant use of color with increasing scene variation 333Multiple replications reported <e.g.,¿Davies2013, Koolen2013
4	Color typicality	More redundant use of color with decreasing color typicality 444Multiple replications reported <e.g.¿sedivy2003a, Westerbeek2015, rubiofernandez2016
5	Basic level preference	Preference for basic level term when superordinate sufficient 555Originally reported by \citeARosch1976, dozens of replications.
5	Subordinate level use	Unnecessary use of subordinate level term 666Reported by \citeAJolicoeur1984

Table 2. Table 2 : Row-wise literal listener distributions P L 0 ​ ( o | u ) subscript 𝑃 subscript 𝐿 0 conditional 𝑜 𝑢 P_{L_{0}}(o|u) for each utterance u 𝑢 u in the size-sufficient context depicted in Figure 1(a) , under a deterministic Boolean semantics (left) or under a continuous semantics (right) with x size = .8 subscript 𝑥 size .8 x_{\text{size}}=.8 , x color = .99 subscript 𝑥 color .99 x_{\text{color}}=.99 . Bolded numbers indicate crucial comparisons between literal listener probabilities in correctly selecting the intended referent o small_blue subscript 𝑜 small_blue o_{\text{small\_blue}} in response to observing the sufficient small and the redundant small blue utterances.

	Boolean			continuous
	$o_{\text{big\_blue}}$	$o_{\text{big\_red}}$	$o_{\text{small\_blue}}$	$o_{\text{big\_blue}}$	$o_{\text{big\_red}}$	$o_{\text{small\_blue}}$
big	.5	.5	0	.39	.39	.22
small	0	0	1	.26	.26	.48
blue	.5	0	.5	.42	.16	.42
red	0	1	0	.21	.57	.21
big blue	1	0	0	.50	.23	.27
big red	0	1	0	.24	.52	.24
small blue	0	0	1	.27	.23	.50

Table 3. Table 3 : Column-wise pragmatic speaker distributions P S 1 ​ ( u | o ) subscript 𝑃 subscript 𝑆 1 conditional 𝑢 𝑜 P_{S_{1}}(u|o) for each object o 𝑜 o in the size-sufficient context depicted in Figure 1(a) , under a deterministic Boolean semantics (left) or under a continuous semantics (middle, right) with x size = .8 subscript 𝑥 size .8 x_{\text{size}}=.8 , x color = .99 subscript 𝑥 color .99 x_{\text{color}}=.99 , with β i subscript 𝛽 𝑖 \beta_{i} set to 1 (middle) or 30 (right). Bolded numbers indicate the relevant speaker probabilities for the minimal ( small ) and redundant ( small blue ) utterances when intending to communicate referent o small_blue subscript 𝑜 small_blue o_{\text{small\_blue}} .

	Boolean			continuous ($bet{a}_i=1$)			continuous (${\beta }_i=30$)
	$o_{\text{big\_blue}}$	$o_{\text{big\_red}}$	$o_{\text{small\_blue}}$	$o_{\text{big\_blue}}$	$o_{\text{big\_red}}$	$o_{\text{small\_blue}}$	$o_{\text{big\_blue}}$	$o_{\text{big\_red}}$	$o_{\text{small\_blue}}$
big	.25	.2	0	.17	.17	.09	0	0	0
small	0	0	.4	.11	.11	.20	0	0	.21
blue	.25	0	.2	.18	.07	.18	.01	0	0
red	0	.4	0	.09	.24	.09	0	.93	0
big blue	.5	0	0	.22	.10	.12	.99	0	0
big red	0	.4	0	.10	.22	.10	0	.07	0
small blue	0	0	.4	.12	.10	.21	0	0	.79

Table 4. Table 4 : Hypothetical semantic values for utterances (rows) as applied to objects (columns). Values where a Boolean semantics would return ‘true’ are bolded.

	yellow banana	brown banana	blue banana	other
banana	.9	.35	.1	.01
yellow banana	.99	.01	.01	.01
brown banana	.01	.99	.01	.01
blue banana	.01	.01	.99	.01
other	.01	.01	.01	.99

Table 5. Table 5 : Overview of the typicality norming studies for Exp. 2. Column ‘Items’ contains the number of unique utterance-object pairs that we elicited responses for.

Utterances	Example	Images	Participants	Trials	Items	Excluded participants
Adj Noun	yellow banana	object	174	110	484	14
Noun	banana	object	75	90	154	1
Adj	yellow	color patch	110	90	176	None

Table 6. Table 6 : Mean typicalities for banana items. Combinations where Boolean semantics would return ‘true’ are marked in boldface.

	Banana items			Other
Utterance	yellow	brown	blue
banana	.98	.66	.42	.05
yellow banana	.97	.30	.15	.05
brown banana	.22	.91	.15	.04
blue banana	.16	.15	.92	.06
yellow	.77	.05	.06	.09
brown	.11	.87	.01	.12
blue	.06	.06	.92	.07

Table 7. Table 7 : Marginal log likelihood for each model. Best model is in bold. Parentheses indicate number of free parameters.

		Semantics
		empirical	fixed	fixed plus empirical
Cost	empirical	-1390.3 (4)	-1295.1 (6)	-1251.4 (7)
	fixed	-1350.3 (4)	-1272.6 (6)	-1238.5 (7)
	none	-1352.2 (2)	-1223.0 (4)	-1188.4 (5)

Table 8. Table 8 : Overview of the best-performing models used for the three different production datasets Color/size (Exp. 1), Color typicality (Exp. 2), and Nominal choice (Exp. 3). Parameter names: x color subscript 𝑥 color x_{\textrm{color}} : semantic value of color; x size subscript 𝑥 size x_{\textrm{size}} : semantic value of size; β c ​ ( color ) subscript 𝛽 𝑐 color \beta_{c(\textrm{color})} : cost of color; β c ​ ( size ) subscript 𝛽 𝑐 size \beta_{c(\textrm{size})} : cost of size; β i subscript 𝛽 𝑖 \beta_{i} : weight on informativity; β F subscript 𝛽 𝐹 \beta_{F} : weight on cost (as estimated by utterance frequency); β L subscript 𝛽 𝐿 \beta_{L} : weight on cost (as estimated by utterance length); β t subscript 𝛽 𝑡 \beta_{t} : weight on elicited typicality values; β fixed subscript 𝛽 fixed \beta_{\textrm{fixed}} : interpolation weight between fixed type-level and empirical semantic values

	Color/size	Color typicality	Nominal choice
Semantic values	at type-level (inferred)	at type-level (inferred) + object-level (elicited)	at object-level (elicited)
Size of lexicon $ℒ(u,o)$	8 (all combinations of size and color)	814 (1 for each utterance-object pair)	51 (1 for each utterance-object pair)
Set of alternatives	8 contextually available feature combinations (size, color)	8 or 9 contextually available feature combinations (type, color)	3 target alternatives (level of reference: sub, basic, super)
Cost	type-level (color and size)	none necessary	empirical (length and frequency)
Free parameters	$x_{\text{color}}$, $x_{\text{size}}$, ${\beta }_{c\left(\text{color}\right)}$, ${\beta }_{c\left(\text{size}\right)}$, ${\beta }_i$	$x_{\text{color}}$, $x_{\text{type}}$, ${\beta }_i$, ${\beta }_t$, ${\beta }_{\text{fixed}}$	${\beta }_F$, ${\beta }_L$, ${\beta }_i$, ${\beta }_t$

Table 9. Table 9 : Exp. 1 items and the colors they appeared in.

Object	Colors	Object	Colors
avocado	black, green	balloon	pink, yellow
belt	black, brown	bike	purple, red
billiard ball	orange, purple	binder	blue, green
book	black, blue	bracelet	green, purple
bucket	pink, red	butterfly	blue, purple
candle	blue, red	cap	blue, orange
chair	green, red	coat hanger	orange, purple
comb	black, blue	cushion	blue, orange
flower	purple, red	frame	green, pink
golf ball	blue, pink	guitar	blue, green
hair dryer	pink, purple	jacket	brown, green
napkin	orange, yellow	ornament	blue, purple
pepper	green, red	phone	pink, white
rock	green, purple	rug	blue, purple
shoe	white, yellow	stapler	purple, red
thumb tack	blue, red	tea cup	pink, white
toothbrush	blue, red	turtle	black, brown
wedding cake	pink, white	yarn	purple, red

Table 10. Table 10 : Model coefficients, standard errors, and p-values. Significant p-values are bolded.

	Coef $\beta$	SE($\beta$)	$p$
Intercept	$-1.85$	$0.34$	<.0001
Scene variation	$4.29$	$1.16$	<.001
Sufficient property	$2.72$	$0.60$	<.0001
Scene variation : Sufficient property	$0.88$	$2.12$	<0.68
Sufficient property : Typicality gain	$9.43$	$2.68$	<.001

Table 11. Table 11 : List of domains and associated superordinate category, target stimuli, and mean length (standard deviation) in characters of actually produced subordinate level utterances in Exp. 3.

Domain	Super	Targets	Mean sub length (sd)
bear	animal	black bear	9.9 (.14)
		polar bear	8.8 (.35)
		panda bear	5.5 (.2)
		grizzly bear	9 (.98)
bird	animal	eagle	4.9 (.1)
		parrot	6.1 (.13)
		pigeon	5.9 (.22)
		hummingbird	10.1 (.5)
candy	snack	MnMs	4.4 (.49)
		skittles	6.9 (.43)
		gummy bears	8.5 (.47)
		jelly beans	9.3 (.44)
car	vehicle	SUV	3 (0)
		minivan	5.7 (.27)
		sports car	9.8 (.23)
		convertible	11.1 (.2)
dog	animal	pug	3 (.08)
		husky	4.7 (.22)
		dalmatian	8.8 (.18)
		German Shepherd	13.1 (.82)
fish	animal	catfish	6.6 (.4)
		goldfish	7.9 (.22)
		swordfish	8 (.43)
		clownfish	9.1 (.38)
flower	plant	rose	4 (0)
		tulip	4.4 (.18)
		daisy	5.9 (.55)
		sunflower	9 (.11)
shirt	clothing	T-shirt	6.4 (.48)
		polo shirt	6.7 (.79)
		dress shirt	11 (0)
		Hawaii shirt	12.6 (.46)
table	furniture	picnic table	9.7 (.58)
		dining table	12 (0)
		coffee table	9.1 (.95)
		bedside table	8.3 (.68)

Equations

$$P_{L_0}(o|u)\propto \mathcal{L}(u,o).$$

$$\mathcal{L}(u,o)=\left\{\begin{array}[]{rl}1&\text{if }u\text{ is true of }o\\ 0&\text{otherwise}.\end{array}\right.$$

$$P_{S_1}(u|o)\propto e^{U(u,o)}$$

$$U(u,o)={\beta }_i\ln P_{L_0}(o|u)-{\beta }_{c}c(u)$$

$$\mathcal{L}(u_{\text{complex}},o)=\mathcal{L}(u_{\text{size}},o)\times \mathcal{L}(u_{\text{color}},o).$$

$$\mathcal{L}(u,o)\in [0,1]\subset \mathbb{R}$$

$$\text{scene variation}=\frac{n_{\text{diff}}}{n_{\text{total}}}$$

$$c(u)={\beta }_F\cdot p(u)+{\beta }_L\cdot l(u)$$

$$P_{L_1}(o|u)\propto P_{S_1}(u|o)\cdot P(o)$$

Full text

Keywords: language production; reference; overinformativeness; experimental pragmatics; Bayesian modeling

When redundancy is useful: A Bayesian approach to ‘overinformative’ referring expressions

1 Overinformativeness in referring expressions

Reference to objects is one of the most basic and prevalent uses of language. In order to refer, speakers must choose from a wealth of referring expressions at their disposal. How does a speaker decide whether to call an object the animal, the dog, the dalmatian, or the big mostly white dalmatian? The context within which the object occurs (other non-dogs, other dogs, other dalmatians) plays a large part in determining which features the speaker chooses to include in their utterance – speakers aim to be sufficiently informative to establish unique reference to the intended object. However, speakers’ utterances exhibit what has been claimed to be overinformativeness: referring expressions are often more specific than necessary for establishing unique reference, and they are more specific in systematic ways.

This paper is concerned with developing a unified quantitative account for these systematic patterns, which has so far proven elusive. We formalize our account as a computational model of referring expression production within the Rational Speech Act framework M\BPBIC. Frank \BBA Goodman (\APACyear2012); Goodman \BBA Frank (\APACyear2016); Franke \BBA Jäger (\APACyear2016), which treats speakers as boundedly rational agents who optimize the tradeoff between utterance cost and informativeness. Our key innovation is to relax the assumption that informativeness of utterances is computed with respect to a deterministic Boolean semantics. Under this relaxed semantics, certain terms may apply better than others to an object without strictly being true or false. This idea has its oldest modern precursor in fuzzy logic Zadeh (\APACyear1965). It is similar in spirit to recently proposed models of meaning in both computational semantics, which assign probabilities rather than truth conditions to sentences Bernardy \BOthers. (\APACyear2018), and in NLP, which treat word and sentence meanings as vectors of real numbers Pennington \BOthers. (\APACyear2014); Peters \BOthers. (\APACyear2018); Devlin \BOthers. (\APACyear2018).

As we will show, computing utterance informativeness with respect to these more graded meanings can explain a number of seemingly disparate phenomena. We restrict ourselves to definite descriptions of the form the (ADJ?)+ NOUN, that is, noun phrases that minimally contain the definite determiner the followed by a head noun, with any number of restrictive adjectives occurring between the determiner and the noun.111In contrast, we will not provide a treatment of pronominal referring expressions, indefinite descriptions, names, definite descriptions with post-nominal modification, or non-restrictive modifier uses, though we offer some speculative remarks on how the approach outlined here can be applied to these cases. This broad class of referring expressions subsumes two domains in language production that have been typically treated as separate. The choice of adjectives in (purportedly) overmodified referring expressions has been a primary focus of the language production literature Herrmann \BBA Deutsch (\APACyear1976); Pechmann (\APACyear1989); Nadig \BBA Sedivy (\APACyear2002); Sedivy (\APACyear2003); Maes \BOthers. (\APACyear2004); Engelhardt \BOthers. (\APACyear2006); Arts \BOthers. (\APACyear2011); Koolen \BOthers. (\APACyear2011); Rubio-Fernandez (\APACyear2016), while the choice of noun in simple nominal expressions has so far mostly received attention in the concepts and categorization literature Rosch (\APACyear1973); Rosch \BOthers. (\APACyear1976) and in the developmental literature on generalizing basic level terms (\citeNPXu2007; but see \citeNPdale1995 for a treatment of basic level terms in natural language generation).

In Section 1 we review several key overinformativeness phenomena across these literatures that have presented a puzzle for rational accounts of language use. In Section 2 we introduce the basic Rational Speech Act framework with deterministic Boolean semantics and show how it can be extended to a relaxed semantics. In Sections 3 - 5 we evaluate the relaxed semantics RSA model on data from interactive online reference game experiments that exhibit the phenomena introduced in Section 1: asymmetries in size and color modifier choice under varying conditions of scene complexity; typicality effects in the choice of color modifier; and choice of nominal level of reference. In each case, our model explains why seemingly overinformative modifiers or overly specific nouns can in fact be useful and informative; not doing so might lead the listener astray, or require them to invest too much processing effort. We wrap up in Section 6 by summarizing our findings and discussing the far-reaching implications of and further challenges for this line of work.

1.1 Production of referring expressions: a case against rational language use?

How should a cooperative speaker choose between competing referring expressions? Grice, in his seminal work, provided some guidance by formulating his famous conversational maxims, intended as a guide to listeners’ expectations about cooperative speaker behavior Grice (\APACyear1975). His maxim of Quantity, consisting of two parts, requires of speakers to:

Quantity-1: Make your contribution as informative as is required (for the purposes of the exchange). 2. 2.

Quantity-2: Do not make your contribution more informative than is required.

That is, speakers should aim to produce neither under- nor overinformative utterances. While much support has been found for the avoidance of underinformativeness Brennan \BBA Clark (\APACyear1996); R. Brown (\APACyear1958); Olson (\APACyear1970); Levinson (\APACyear1983); Engelhardt \BOthers. (\APACyear2006); Davies \BBA Katsos (\APACyear2013), speakers seem remarkably willing to systematically violate Quantity-2. For example, they routinely produce modifiers that are not necessary for uniquely establishing reference <e.g., the small blue pin instead of the small pin in contexts like Figure 1a;¿gatt2011, Gatt2014, Arts2011, Koolen2011 and routinely use a basic level term even when a superordinate level term would be sufficient <e.g., the dog instead of the animal in contexts like Figure 3;¿Rosch1976, hoffmann1983objektidentifikation, TanakaTaylor91_BasicLevelAndExpertise, Johnson1997, brown1958words.

These observations have posed a challenge for theories of language production, especially those positing rational language use (including the Gricean one): why this extra expenditure of useless effort? Why this seeming blindness to the level of informativeness requirement? Many have argued from these observations that speakers are in fact not economical Engelhardt \BOthers. (\APACyear2006); Pechmann (\APACyear1989). Some have appealed to a built-in preference for referring at the basic level from considerations of conceptual representation or perceptual factors such as shape Rosch \BOthers. (\APACyear1976); Rosch (\APACyear1973); Murphy \BBA Smith (\APACyear1982). Others have argued for salience-driven effects on willingness to overmodify Gatt \BOthers. (\APACyear2014); Westerbeek \BOthers. (\APACyear2015). In all cases, it is argued that informativeness itself cannot be the key factor in determining the content of speakers’ referring expressions. Here we revisit this claim and show that systematically relaxing the requirement of a deterministic Boolean semantics for referring expressions also systematically changes the informativeness of utterances. This results in a reconceptualization of what have been termed overinformative referring expressions as usefully redundant referring expressions. We begin by reviewing the phenomena of interest that a revised theory of definite referring expressions should be able to account for.

1.2 Overinformativeness in modified referring expressions

Most of the literature on overinformative referring expressions has been devoted to the use of overinformative modifiers in modified referring expressions. The prevalent observation is that speakers frequently do not include only the minimal modifiers required for establishing reference, but often also include redundant modifiers Pechmann (\APACyear1989); Nadig \BBA Sedivy (\APACyear2002); Maes \BOthers. (\APACyear2004); Engelhardt \BOthers. (\APACyear2006); Arts \BOthers. (\APACyear2011); Koolen \BOthers. (\APACyear2011). However, not all modifiers are created equal: there are systematic differences in the overmodification patterns observed for size adjectives (e.g., big, small), color adjectives (e.g., blue, red), material adjectives (e.g., plastic, wooden), and others Sedivy (\APACyear2003). Furthermore, these asymmetries interact with features of the context and world knowledge about the typicality of different properties.

Asymmetry in redundant use of color and size adjectives.

In Figure 1(a), distinguishing the object highlighted by the thick border requires only mentioning its size (the small pin). It is now well-documented that speakers routinely include redundant color adjectives (the small blue pin) which are not necessary for uniquely singling out the intended referent in these kinds of contexts Pechmann (\APACyear1989); Belke \BBA Meyer (\APACyear2002); Gatt \BOthers. (\APACyear2011). However, the same is not true for size: in contexts like Figure 1(b), where color is sufficient for unique reference (the blue pin), speakers overmodify much more rarely. Though there is quite a bit of variation in proportions of overmodification, this asymmetry in the propensity for overmodifying with color but not size has been documented repeatedly Pechmann (\APACyear1989); Sedivy (\APACyear2003); Gatt \BOthers. (\APACyear2011); Rubio-Fernandez (\APACyear2016); Westerbeek \BOthers. (\APACyear2015); Koolen \BOthers. (\APACyear2013).

Scene variation.

Speakers’ propensity to overmodify with color is highly dependent on features of the distractor objects in the context. In particular, as the variation present in the scene increases, so does the probability of overmodifying. For example \citeAKoolen2013 consistently found higher rates of overmodification with color adjectives in high-variation scenes (28-27%) compared to the low-variation ones (4-10%). Scene variation has been quantified in several different ways: the number of dimensions along which objects differ \citeAKoolen2013, the number of distractors present in a scene \citeAgatt2017, and whether objects are ‘simple’ or ‘compositional’ \citeADavies2013. A model of referring expression generation should ideally capture all of these types of variation in a unified way.

Feature typicality.

Overmodification with color has also been shown to be systematically related to the typicality of the color for the object. \citeAWesterbeek2015 have shown that the more typical a color is for an object, the less likely it is to be mentioned when not necessary for unique reference <see also¿sedivy2003a,rubiofernandez2016. For example, speakers never refer to a yellow banana in the absence of other bananas as the yellow banana (see Figure 2(a)), but they sometimes refer to a brown banana as the brown banana, and they almost always refer to a blue banana as the blue banana (see Figure 2(b)). Similar typicality effects have been shown for other (non-color) properties. For example, \citeAMitchell2013 showed that speakers are more likely to include an atypical than a typical property (either shape or material) when referring to everyday objects like boxes when mentioning at least one property was necessary for unique reference.

1.3 Overinformativeness in nominal referring expressions

Even in the absence of modifying adjectives, a referring expression can be more or less informative: the dalmatian communicates more information about the object in question than the dog (being a dalmatian entails being a dog), which in turn is globally more informative than the animal. Thus, this choice can be considered analogous to the choice of adding more modifiers – in both cases, the speaker has a choice of being more or less specific about the intended referent. A well-documented effect from the concepts and categorization literature is that speakers prefer to refer at the basic level Rosch \BOthers. (\APACyear1976); Tanaka \BBA Taylor (\APACyear1991\APACexlab\BCnt2). That is, in the absence of other constraints, even when a superordinate level term would be sufficient for establishing reference (as in Figure 3(b)), speakers prefer to say the dog rather than the animal. However, there are systematic exceptions: in some cases when the basic level would be sufficient, speakers prefer the subordinate term. For example, atypical birds like penguins are often referred to at the subordinate level rather than at the basic level bird Jolicoeur \BOthers. (\APACyear1984). Indeed, children may even use expectations about such referential preferences to infer narrower categories from atypical exemplars during word learning Emberson \BOthers. (\APACyear2019).

2 Modeling speakers’ choice of referring expression

To date, there is no theory to account for all of these different phenomena (see Table 1), and no model has attempted to unify the domains of modified and nominal referring expressions. Here we propose an explicit computational account of how multiple factors — including an utterance’s literal semantics, its contextual informativity, its cost relative to alternative utterances, and the typicality of an object or its features — interact in referring expression production. We argue that this model provides a principled explanation for the phenomena reviewed in the previous section and holds promise for being generalizable to many further production phenomena related to overinformativeness, which we discuss in relation to previous accounts in Section 6.

Our model is formulated within the Rational Speech Act (RSA) framework M\BPBIC. Frank \BBA Goodman (\APACyear2012); Goodman \BBA Frank (\APACyear2016).777All RSA models and Bayesian data analyses reported in this paper were implemented in the probabilistic programming language WebPPL Goodman \BBA Stuhlmüller (\APACyearelectronic) and can be viewed at https://github.com/thegricean/RE_production. All experimental materials and analysis scripts are available in the same repository. An interactive browser-based toy model is provided at http://forestdb.org/models/overinf.html. We proceed by first presenting the general production framework in Section 2.1, and show why the most basic model, as formulated by \citeNPfrank2012, does not produce the phenomena outlined above due to its strong focus on speakers maximizing the informativeness of expressions under a deterministic Boolean semantics. In Section 2.2 we introduce our crucial innovation: relaxing the semantics.

2.1 Basic RSA

The production component of RSA aims to soft-maximize the utility of utterances, where utility is defined in terms of the contextual informativeness of an utterance, given each utterance’s literal semantics. Formally, this is treated as a pragmatic speaker $S_{1}$ reasoning about a literal listener $L_{0}$, who can be described by the following formula:

[TABLE]

The literal listener $L_{0}$ observes an utterance $u$ from the set of utterances $U$, consisting of single adjectives denoting features available in the context of a set of objects $O$, and returns a distribution over objects $o\in O$. Here, $\mathcal{L}(u,o)$ is the lexicon that encodes deterministic lexical meanings such that:

[TABLE]

Thus, $P_{L_{0}}(o|u)$ returns a uniform distribution over all contextually available $o$ in the extension of $u$. For example, in the size-sufficient context shown in Figure 1(a), $U=\{\textrm{\emph{big}},\textrm{\emph{small}},\textrm{\emph{blue}},\textrm{\emph{red}}\}$ and $O=\{o_{\textrm{big\_blue}},o_{\textrm{big\_red}},o_{\textrm{small\_blue}}\}$. Upon observing blue, the literal listener therefore assigns equal probability to $o_{\textrm{big\_blue}}$ and $o_{\textrm{small\_blue}}$. Values of $P_{L_{0}}(o|u)$ for each $u$ are shown on the left in Table 2.

The pragmatic speaker in turn produces an utterance with probability proportional to the utility of that utterance:

[TABLE]

The speaker’s utility $U(u,o)$ is a function of both the utterance’s informativeness with respect to the literal listener $P_{L_{0}}(o|u)$ and the utterance’s cost $c(u)$:

[TABLE]

Two free parameters, $\beta_{i}$ and $\beta_{c}$ enter the computation, weighting the respective contributions of informativeness and utterance cost, respectively.888\citeAfrank2012 fixed $\beta_{i}=1$ and did not include cost in their formulation, because they assumed equal costs for all utterances. Subsequent work has demonstrated the importance of taking into account utterance cost in modeling interpretation phenomena like cost-based quantity implicatures Rohde \BOthers. (\APACyear2012); Degen \BOthers. (\APACyear2013) and M-implicature Bergen \BOthers. (\APACyear2016). We include it here because of the importance that cost has played in explanations of overinformative referring expressions, where it typically surfaces as the idea that speakers have different overall preferences for mentioning color vs. size modifiers Dale \BBA Reiter (\APACyear1995); Koolen \BOthers. (\APACyear2011); van Gompel \BOthers. (\APACyear2019). At this point we remain agnostic about the factors that contribute to an utterance’s cost $c(u)$. In later sections we allow cost to be a function of properties (e.g. color & size) mentioned in the utterance, or of an utterance’s empirical length and corpus frequency; our policy for these cases is to introduce free cost parameters for each linear component of the cost function. In order to understand the effect of $\beta_{i}$, it is useful to explore its effect when utterances are cost-free. In this case, as $\beta_{i}$ approaches infinity, the speaker increasingly only chooses utterances that maximize informativeness; if $\beta_{i}$ is 0, informativeness is disregarded and the speaker chooses randomly from the set of all available utterances; if $\beta_{i}$ is 1, the speaker probability-matches, i.e., chooses utterances proportional to their informativeness <equivalent to Luce’s choice rule,¿luce1959. Applied to the example in Table 2, if the speaker wants to refer to $o_{\textrm{small\_blue}}$ they have two semantically possible utterances, small and blue, where small is twice as informative as blue. They produce small with probability 1 when $\beta_{i}\rightarrow\infty$, probability 2/3 when $\beta_{i}=1$ and probability 1/4 when $\beta_{i}=0$.999Note that instead of a $\beta_{i}$ parameter weighting informativeness inside the utility function, other recent formulations have used an $\alpha$ parameter modulating the entire utility function, i.e. $P_{S_{1}}(u|o)\propto\exp{\alpha U(u,o)}$. These parameterizations are equivalent. In the present work, where informativeness and cost both play important roles, we chose the ‘flattened’ linear combination with independent weights for simplicity. Conversely, disregarding informativeness and focusing only on cost, any asymmetry in costs will be exaggerated with increasing $\beta_{c}$, such that the speaker will choose the least costly utterance with higher and higher probability as $\beta_{c}$ increases.

As has been pointed out by \citeAVanGompel2019, the basic Rational Speech Act model described so far M\BPBIC. Frank \BBA Goodman (\APACyear2012) does not generate overinformative referring expressions for two reasons. One of these is trivial: $U$ only contains one-word utterances. We can ameliorate this easily by allowing complex two-word utterances. We assume an intersective semantics for complex utterances $u_{\textrm{complex}}$ that consist of a two adjective sequence $u_{\textrm{size}}\in\{\textrm{\emph{big}},\textrm{\emph{small}}\}$ and $u_{\textrm{color}}\in\{\textrm{\emph{blue}},\textrm{\emph{red}}\}$, such that the meaning of a complex two-word utterance is defined as

[TABLE]

The resulting renormalized literal listener distributions for our example size-sufficient context in Figure 1(a) are shown in the left columns in Table 2,101010‘Normalization’ refers to the process of turning a set of numbers into a probability distribution by dividing each number by the sum of all the numbers in the set, such that they add up to 1. and the concomitant pragmatic speaker distributions are shown in the left columns in Table 3.111111An interactive toy version of this model is provided at http://forestdb.org/models/overinf.html.

Unfortunately, simply including complex utterances in the set of alternatives does not solve the problem. We turn again to the case where the speaker wants to communicate the small blue object. There are now two useful utterances, small and small blue, for referring to this object. Because they are equally informative (see bolded numbers in Table 2, column 3), the pragmatic speaker is equally likely to produce them (see bolded numbers in Table 3, column 3). The only way for the more complex utterance to be chosen with greater probability than the simple utterance is if it was the cheaper one. While this would achieve the desired mathematical effect, the cognitive plausibility of complex utterances being cheaper than simple utterances is highly dubious121212See also the discussion of cost functions in \citeAKrahmer2003, who explicitly introduce this monotonicity constraint as a constraint on the search space of possible referring expressions within a graph-based framework.. Thus we must look elsewhere to break the symmetry and account for overinformativeness. We propose that the place to look is the computation of informativeness itself.

2.2 RSA with continuous semantics

Here we introduce the crucial innovation: rather than assuming a deterministic Boolean semantics that returns true (1) or false (0) for any combination of expression and object, we relax to a continuous semantics that returns real values in the interval $[0,1]$. Formally, the only change is in the values that the lexicon can return:

[TABLE]

That is, rather than assuming that an object is unambiguously big (or not) or unambiguously blue (or not), this continuous semantics captures that objects count as big or blue to varying degrees <similar to approaches in fuzzy logic, prototype theory, and recent developments in NLP;¿zadeh1965fuzzy, Rosch1973, Bernardy2018.

Another approach to relaxing the deterministic Boolean semantics would be to relax the determinism. This can be done either by assuming a semantics which is fundamentally Boolean, but whose truth-values contain an element of randomness; or by assuming a fully deterministic Boolean semantics with intensional parameters that are themselves random variables. This is appealing because if would preserve the existing machinery of standard truth-functional compositional semantics. It can be shown that using continuous semantic values in the RSA model is equivalent to using Boolean values that are chosen non-deterministically. Conversely, marginalizing over the randomness in a Boolean semantics yields a probability of truth, which is a value between 0 and 1. For this reason we will sometimes refer to the relaxed semantics as a “noisy” semantics, and the deviation of the semantic value from 0 or 1 as the degree of noise. We will generally treat the relaxed semantics in its continuous value guise, as it simplifies exposition and development.

We now show via simulations that this model can qualitatively account both for speakers’ asymmetric propensity to overmodify with color rather than with size (in Section 2.2.1) and for speakers’ propensity to overmodify more with increasing scene variation (in Section 2.2.2). The intuition, using the example from Figure 1(a), is that blue and small do not apply equally well to all roughly blue, roughly small objects, and that a speaker might opt to include more modifiers when any one alone might not be a perfectly apt descriptor. Assuming that blue is more precise than small leads the speaker to overmodify more with color than with size – and further, the more variability is present in the scene, the more the precision of color helps weed out non-intended referents, i.e., the more color overmodification occurs.

2.2.1 Simulation 1: color-size asymmetry

To see the basic effect of switching to a continuous semantics, and to see how far we can get in capturing overinformativeness patterns with this change, let us explore a simple semantics in which all colors are treated the same, all sizes are as well, and the two compose via a product rule. That is, when an object $o$ is in the extension of a size adjective under a Boolean semantics – i.e., when the size can be truthfully predicated of $o$ – we take $\mathcal{L}(u,o)=x_{\text{size}}$, a constant; when it is not in the extension of the adjective – i.e., when the size cannot be truthfully predicated of $o$ – $\mathcal{L}(u,o)=1-x_{\text{size}}$. Similarly for color adjectives. This results in two free model parameters, $x_{\text{size}}$ and $x_{\text{color}}$, that can take on different values, capturing that size and color adjectives may apply more or less well/reliably to objects. Together with the product composition rule, Eq. 5, this fully specifies a relaxed semantic function for our reference domain.131313An interactive toy version of this model is provided at http://forestdb.org/models/overinf.html.

Now consider the RSA literal listener, Eq. 1, who uses these relaxed semantic values. Given an utterance, the listener simply normalizes over potential referents. As an example, the resulting renormalized literal listener distributions for the size-sufficient example context in Figure 1(a) are shown for values $x_{\text{size}}=.8$ and $x_{\text{color}}=.99$ on the right in Table 2.141414These values were chosen for the demonstration because they are the ones that result in the best approximation of the proportion of redundant referring expressions reported in \citeAVanGompel2019: 79% in size-sufficient contexts; 7% in color-sufficient contexts. Recall that in this context, the speaker intends for the listener to select the small blue pin. To see which would be the best utterance to produce for this purpose, we compare the literal listener probabilities in the $o_{\text{small\_blue}}$ column. The two best utterances under both the Boolean and the continuous semantics are bolded in the table: under the Boolean semantics, the two best utterances are small and small blue, with no difference in listener probability. In contrast, under the continuous semantics small has a smaller literal listener probability (.48) of retrieving the intended referent than the redundant small blue (.50). While this difference may appear small, it is enough to break the symmetry in utterance informativeness. Consequently, the pragmatic speaker will be more likely to produce small blue than small, though the precise probabilities depend on the cost and informativeness parameters $\beta_{c}$ and $\beta_{i}$. Table 3 shows the resulting pragmatic speaker probabilities under a low and a high $\beta_{i}$ with no utterance cost.

Crucially, the reverse is not the case when color is the distinguishing dimension. Consider the speaker in the same context wanting to communicate the big red pin. The two best utterances for this purpose are red (.57) and big red (.52). In contrast to the results for the small blue pin, the redundant utterance does not increase the literal listener probability of inferring the intended referent. The reason for this is that we defined color to be almost noiseless, with the result that the literal listener distributions in response to utterances containing color terms are more similar to those obtained via a Boolean semantics than the distributions obtained in response to utterances containing size terms. The reader is encouraged to verify this by comparing the row-wise distributions under the Boolean and continuous semantics in Table 2.

To better understand the consequences of continuous meanings in contexts like that depicted in Figure 1(a), we visualize the results of varying $x_{\text{size}}$ and $x_{\text{color}}$ in Figure 4. The deterministic Boolean semantics of utterances is approximated where the semantic values of both size and color utterances are close to 1 (.999, top right-most point in graph). In this case, the simple sufficient (small pin) and complex redundant utterance (small blue pin) are equally likely because they are both equally informative and utterances are assumed to have 0 cost. All other utterances are highly unlikely. The interesting question is under which circumstances, if any, the standard color-size asymmetry emerges. This asymmetry is found in the warmer region of the ‘small blue’ facet, characterized by values of $x_{\text{size}}$ that are lower than $x_{\text{color}}$, with high values for $x_{\text{color}}$. That is, redundant utterances are more likely than sufficient utterances when the redundant dimension (in this case color) is less noisy than the sufficient dimension (in this case size) and overall is close to noiseless. Thus, when size adjectives are noisier than color adjectives, the model produces overinformative referring expressions with color, but not with size – precisely the pattern observed in the literature Pechmann (\APACyear1989); Gatt \BOthers. (\APACyear2011). Note also that no difference in adjective cost is necessary for obtaining the overinformativeness asymmetry, though assuming a greater cost for size than for color does further increase the observed asymmetry (see Section 3.3 for further discussion).

2.2.2 Simulation 2: scene variation

In the previous section, we showed that extending RSA with continuous adjective semantics gives rise to color-size asymmetries when the semantics of color adjectives is closer to deterministic Boolean truth-functions than size adjectives. When modifiers are noisy, adding ‘stricter’ modifiers adds information. From this perspective, these additional modifiers are not overinformative; they are usefully redundant given the needs of the listener. Next, we show how the same mechanism accounts for why increased scene variation increases the probability that referring expressions are overmodified with color.

\citeA

Koolen2013 quantified scene variation as the number of feature dimensions along which pieces of furniture in a scene varied: type (e.g., chair, fan), size (big, small), and color (e.g., red, blue).151515They also included orientation (left-facing, right-facing) as a dimension along which objects could vary in certain cases. We ignore this dimension here for the sake of simplicity. Scene variation was manipulated across two experiments, which differed in the dimension necessary for unique reference (color was always redundant). In Exp. 1, only type was necessary (fan and couch in the low and high variation conditions in Figure 5(a), respectively). In Exp. 2, size and type were necessary (big chair and small chair in Figure 5(a), respectively). Across both experiments, lower rates of redundant color use were found in the low variation conditions (4% and 9%) than in the high variation conditions (24% and 18%). Here, we use simulations to explore the predictions that continuous semantics RSA – henceforth cs-RSA – makes for these situations.

Following \citeAKoolen2013, we considered any mention of color as a redundant mention. In Exp. 1, this includes the simple redundant utterances like blue couch as well as complex redundant utterances like small blue couch. In Exp. 2, where size was necessary for unique reference, only the complex redundant utterance small brown chair was truly redundant (brown chair was insufficient, but still included in counts of color mention). Because object type was a distinguishing dimension, we introduce an additional semantic value $x_{\text{type}}$, which encodes how noisy nouns are. The results of simulating these conditions with parameters $\beta_{i}=30$, $\beta_{c}=c(u_{\textrm{size}})=c(u_{\textrm{color}})=1$, $x_{\text{size}}=.8$, $x_{\text{color}}=.999$, and $x_{\text{type}}=.9$ are shown in Figure 5(b), under the assumption that the cost of a two-word utterance $c(u)$ is the sum of the costs of the one-word sub-utterances.161616These parameter values were chosen merely for convenience in illustrating the qualitative model predictions. We reused values from the previous example, where possible, but also included a cost per word. For both experiments, the model exhibits the empirically-observed qualitative effect of variation on the probability of redundant color mention: when variation is greater, redundant color mention is more likely. Indeed, this effect of scene variation is predicted by the model anytime the semantic values for size, type, and color are ordered as: $x_{\text{size}}\leq x_{\text{type}}<x_{\text{color}}$. If, on the other hand, $x_{\text{type}}$ is greater than $x_{\text{color}}$, the probability of redundantly mentioning color is close to zero and does not differ between variation conditions (in those cases, color mention reduces, rather than adds, information about the target).

To further explore the scene variation effect predicted by RSA, we turn again to Figure 1(a). Here, the target item is the small blue pin and there are two distractor items: a big blue pin and a big red pin. Thus, for the purpose of establishing unique reference, size is the sufficient dimension and color the insufficient dimension. We can measure scene variation as the proportion of distractor items that do not share the value of the insufficient feature with the target, that is, as the number of distractors $n_{\textrm{diff}}$ that differ in the value of the insufficient feature divided by the total number of distractors $n_{\textrm{total}}$:

[TABLE]

In Figure 1(a), there is one distractor that differs from the target in color (the big red pin) and there are two distractors in total. Thus, $\textrm{scene variation}=\frac{1}{2}=.5$. In general, this measure of scene variation is minimal when all distractors are of the same color as the target, in which case it is 0. Scene variation is maximal when all distractors except for one (in order for the dimension to remain insufficient for establishing reference) are of a different color than the target. That is, scene variation may take on values between 0 and $\frac{n_{\textrm{total}}-1}{n_{\textrm{total}}}$.171717Some readers might find this unintuitive: shouldn’t scene variation be maximal when there is an equal number of same and different colors? Or when the different colors are also all different from one another? As discussed in Section 1.2, there are many ways of quantifying (different aspects of) scene variation. We choose to explore this aspect of variation here as a reasonable first step; RSA makes predictions for other kinds of variation that would be equally straightforward to test.

Using the same parameter values as above, we generate model predictions for size-sufficient and color-sufficient contexts, manipulating scene variation by varying number of distractors (2, 3, or 4) and number of distractors that don’t share the insufficient feature value. The resulting model predictions are shown in Figure 6. The predicted probability of redundant adjective use is largely (though not completely) correlated with scene variation. Redundant adjective use increases with increasing scene variation when size is sufficient (and color redundant), but not when color is sufficient (and size redundant). The latter prediction depends, however, on the actual semantic value of color—with slightly lower semantic values for color, the model predicts small increases in redundant size use. In general: increased scene variation is predicted to lead to a greater increase in redundant adjective use for less noisy adjectives.

RSA with a continuous semantics thus captures the qualitative effects of color-size asymmetry and scene variation in production of redundant expressions, and it makes quantitative predictions for both. Testing these quantitative predictions, however, will require more data. In the remainder of the paper, we quantitatively evaluate cs-RSA on new datasets capturing the phenomena described in the Introduction (Table 1): modifier type and scene variation effects on modified referring expressions, typicality effects on color mention, and the choice of taxonomic level of reference in nominal choice.

3 Experiment 1: size and color modifiers under different scene variation conditions

Adequately assessing the explanatory value of RSA with continuous semantics requires evaluating how well it does at predicting the probability of various types of utterances occurring in large datasets of naturally produced referring expressions. While we showed in Section 2.2.2 that cs-RSA qualitatively predicts the pattern of overmodification under scene variation, we now test the model’s quantitative predictions more rigorously in an interactive web-based reference game paradigm. We then perform a Bayesian data analysis to both assess how likely the model is to generate the observed data – i.e., to obtain a measure of model quality – and to explore the posterior distribution of parameter values – i.e., to understand whether the asymmetries in adjectives’ semantic values and/or costs explored in the previous section are validated by the data.

3.1 Method

Participants

We recruited 58 pairs of participants (116 participants total) over Amazon’s Mechanical Turk who were each paid $1.75 for their participation.181818We aim to pay Mechanical Turk workers at a rate of$12 - $14. Data from another 7 pairs who prematurely dropped out of the experiment and who could therefore not be compensated for their work, were also included. Here and in all other experiments reported in this paper, participants’ IP address was limited to US addresses and only participants with a past work approval rate of at least 95% were accepted.

Procedure

Participants were paired up through a real-time multi-player interface Hawkins (\APACyear2015). One participant was assigned the speaker role and one the listener role. Before continuing to the experiment, participants were required to correctly answer a series of questions about the experimental procedure (see Appendix B). On each trial, both participants saw the same array of objects in independently randomized locations. One of these objects was privately designated as the target object to the speaker, and marked by a thick border (see Figure 7). The speaker’s task was to use an unrestricted chat box to send a message communicating the target to the listener, who subsequently clicked an object to make a response. Both participants then received feedback about whether the intended referent was selected and advanced to the next trial. They were explicitly told that using locative modifiers (like left or right) would be useless because the order of objects on their partner’s screen would be different than on their own screen. For natural interaction, we allowed both speakers and listeners to write freely in the chat window at any point, but listeners could only click on an object to advance to the next trial after the speaker sent an initial message. At the end of the experiments, participants completed a questionnaire in which they indicated whether their native language was English, whether they thought their partner was human, and how much they liked their partner.

Materials

Participants proceeded through 72 trials. Of these, half were critical trials of interest and half were filler trials. On critical trials, we varied which feature was sufficient for uniquely establishing reference, the total number of objects in the array, and the number of objects that shared the insufficient feature with the target.

Objects varied in color and size. On 18 trials, color was sufficient for establishing reference. On the other 18 trials, size was sufficient. Figure 7 shows an example of a size-sufficient trial. We further varied the amount of variation in the scene by varying the number of distractor objects in each array (2, 3, or 4) and the number of distractors that did share the redundant feature value with the target. That is, when size was sufficient, we varied the number of distractors that shared the same color as the target. This number had to be at least one, since otherwise the redundant property would have been sufficient for uniquely establishing reference, i.e. mentioning it would not have been redundant. Each total number of distractors was crossed with each possible number of distractors that shared the redundant property, leading to the following nine conditions: 2-1, 2-2, 3-1, 3-2, 3-3, 4-1, 4-2, 4-3, and 4-4, where the first number indicates the total number and the second number the shared number of distractors. Each condition occurred twice with each sufficient dimension. Objects never differed in type within one array (e.g., all objects are pins in Figure 7) but always differed in type across trials. Each object type could occur in two different sizes and two different colors. We used photo-realistic objects of intuitively fairly typical colors. The 36 different object types and the colors they could occur with are listed in Appendix C.

Fillers were target trials from Exp. 2, a replication of \citeAGrafEtAl2016. Each filler item contained a three-object grid. None of the filler objects occurred on target trials. Objects stood in various taxonomic relations to each other and required neither size nor color mention for unique reference. See Section 5 for a description of these materials.

Data pre-processing and exclusion

We collected data from 2177 critical trials. Because we did not restrict participants’ utterances in any way, they produced many different kinds of referring expressions. Testing the model’s predictions required, for each trial, classifying the produced utterance as an instance of a color-only mention (e.g., blue pin), a size-only mention (e.g., big pin), or a redundant color-and-size mention (e.g., big blue pin). To this end we applied a semi-automatic data pre-processing procedure in which a script first checked whether the speaker’s utterance contained a color or size term. In a second step, one of the authors (CG) manually checked and, if necessary, corrected the automatic classification. If no classification was possible, the trial was excluded. After exclusions, 2076 cases entered the analysis. See Appendix D for details on the pre-processing procedure.

3.2 Behavioral results

Proportions of redundant color-and-size utterances are shown in Figure 8 alongside model predictions (to be explained further in Section 3.3). There are three main questions of interest: first, do we replicate the color/size asymmetry in probability of redundant adjective use? Second, do we replicate the previously established effect of increased redundant color use with increasing scene variation? Third, is there an effect of scene variation on redundant size use and if so, is it smaller compared to that on color use, as is predicted under asymmetric semantic values for color and size adjectives?

We addressed all of these questions by conducting a single mixed effects logistic regression analysis predicting redundant over minimal adjective use from fixed effects of sufficient property (color vs. size), scene variation (proportion of distractors that do not share the insufficient property value with the target), and the interaction between the two.191919All mixed effects analyses reported in this paper were conducted with the lme4+ package Bates \BOthers. (\APACyear2015) in R R Core Team (\APACyear2017). All predictors were centered before entering the analysis. The model included the most sophisticated random effects structure that allowed the model to converge: by-speaker and by-item random intercepts.

We observed a main effect of sufficient property, such that speakers were more likely to redundantly use color than size adjectives ($\beta=3.54$, $SE=.22$, $p<.0001$), replicating the much-documented color-size asymmetry. We further observed a main effect of scene variation, such that redundant adjective use increased with increasing scene variation ($\beta=4.62$, $SE=.38$, $p<.0001$). Finally, we also observed a significant interaction between sufficient property and scene variation ($\beta=2.26$, $SE=.74$, $p<.003$). Simple effects analysis revealed that the interaction was driven by the scene variation effect being smaller in the color-sufficient condition ($\beta=3.49$, $SE=.65$, $p<.0001$) than in the size-sufficient condition ($\beta=5.75$, $SE=.38$, $p<.0001$), as predicted if size modifiers are noisier than color modifiers. That is, while the color-sufficient condition indeed showed a scene variation effect—and as far as we know, this is the first demonstration of an effect of scene variation on redundant size use—this effect was tiny compared to that of the size-sufficient condition.202020In order to address convergence issues with lmer+ when specifying the maximal random effects structure – i.e., by-speaker and by-item random intercepts and slopes for all fixed effects and their interactions – we ran a Bayesian binomial mixed effects model with weakly informative priors using the brms+ package Bürkner (\APACyear2017) that included the same fixed effects structure as the lmer model and the maximal random effects structure. The results were qualitatively identical, yielding evidence for main effects of redundant feature (posterior mean $\beta$ = 5.91, $95\%$ CI = $[$4.15,8.10$]$, $p(\beta>0)$ = .98), scene variation (posterior mean $\beta$ = 6.18, $95\%$ CI = $[$4.30,8.24$]$, $p(\beta>0)$ = 1), and their interaction (posterior mean $\beta$ = 3.31, $95\%$ CI = $[$-0.54,7.23$]$, $p(\beta>0)$ = .96).

3.3 Model evaluation

In order to evaluate RSA with continuous semantics we conducted a Bayesian data analysis. This allowed us to simultaneously generate model predictions and infer likely parameter values, by conditioning on the observed production data (coded into size, color, and size-and-color utterances as described above) and integrating over the five free parameters. To allow for differential costs for size and color, we introduce separate cost weights ($\beta_{c(\textrm{size})},\beta_{c(\textrm{color})}$) applying to size and color mentions, respectively, in addition to semantic values for color and size ($x_{\textrm{color}}$, $x_{\textrm{size}}$) and an informativeness parameter $\beta_{i}$. We assumed uniform priors for each parameter: $x_{\textrm{color}},x_{\textrm{size}}\sim\mathcal{U}(0,1)$, $\beta_{c(\textrm{size})},\beta_{c(\textrm{color})}\sim\mathcal{U}(0,40)$, $\beta_{i}\sim\mathcal{U}(0,40)$. Inference for the cognitive model was exact. We used Markov Chain Monte Carlo (MCMC) with a burn-in of 10000 and lag of 10 to draw 2000 samples from the joint posteriors on the five free parameters.

Point-wise maximum a posteriori (MAP) estimates of the model’s posterior predictives for just redundant utterance probabilities are shown alongside the empirical data in Figure 8. In addition, MAP estimates of the model’s posterior predictives for each combination of utterance, sufficient dimension, number of distractors, and number of different distractors (collapsing across different items) are plotted against all empirical utterance proportions in Figure 9. At this level, the model achieves a correlation of $r=.99$. Looking at results additionally on the by-item level yields a correlation of $r=.85$ (this correlation is expected to be lower both because each item contains less data, and because we did not provide the model any means to refer differently to, e.g., combs and pins). The model thus does a very good job of capturing the quantitative patterns in the data.

Posteriors over parameters are shown in Figure 10. Crucially, the semantic value of color is inferred to be higher than that of size – there is no overlap between the 95% highest density intervals (HDIs) for the two parameters. That is, size modifiers are inferred to be noisier than color modifiers. The high inferred $\beta_{i}$ (MAP $\beta_{i}$ = 31.4, HDI = [30.7,34.5]) suggests that this difference in semantic value contributes substantially to the observed color-size asymmetries in redundant adjective use and that speakers are maximizing quite strongly. As for cost, there is a lot of overlap in the inferred weights of size and color modifiers, which are both skewed very close to zero, suggesting that a cost difference (or indeed any cost at all) is neither necessary to obtain the color-size asymmetry and the scene variation effects, nor justified by the data. Recall further that we already showed in Section 2.2 that the color-size asymmetry in redundant adjective use requires an asymmetry in semantic value and cannot be reduced to cost differences. An asymmetry in cost only serves to further enhance the asymmetry brought about by the asymmetry in semantic value, but cannot carry the redundant use asymmetry on its own.

3.4 Discussion

In this section we reported a new dataset of freely produced referring expressions that replicated the well-documented color-size asymmetry in redundant adjective use, the effect of scene variation on redundant color use, and showed a novel effect of scene variation on redundant size use. We also showed that cs-RSA provides an excellent fit to these data. In particular, the crucial element in obtaining the color-size asymmetry in overmodification is that size adjectives be noisier than color adjectives, captured in RSA via a lower semantic value for size compared to color. The effect is that color adjectives are more informative than size adjectives when controlling for the number of distractors that each would rule out under a Boolean semantics. Asymmetries in the cost of the adjectives were not attested, and would only serve to further enhance the modification asymmetry resulting from the asymmetry in semantic value. In addition, we showed that asymmetric effects of scene variation on overmodification straightforwardly fall out of cs-RSA: scene variation leads to a greater increase in overmodification with less noisy modifiers because these modifiers (colors) on average provide more information about the target.

While we defer a broader discussion of the possible psychological and linguistic interpretations of continuous semantic values to the General Discussion in Section 6, it is worth reflecting on why size adjectives may be inherently noisier than color adjectives. Color adjectives are typically treated as absolute adjectives while size adjectives are inherently relative Pechmann (\APACyear1989); Kennedy \BBA McNally (\APACyear2005). That is, while both size and color adjectives are vague, size adjectives are arguably context-dependent in a way that color adjectives are not – whether an object is big depends inherently on its comparison class; whether an object is red does not.212121This is not entirely true, as has been repeatedly pointed out <e.g.,¿cohen1984models: red hair has a very different color than red wine, which in turn has a different color from a red bell pepper. If presented out of context, only the last red is likely to be judged as red. For discussion of the complex semantics of color terms, see \citeNPKennedy2010, Rothschild2009, Szabo2001. For our purposes, it suffices that one can give a color judgment but not a size judgment for an object presented in isolation. In addition, color as a property has been claimed to be inherently salient in a way that size is not Arts \BOthers. (\APACyear2011); van Gompel \BOthers. (\APACyear2019). Finally, color adjectives are rated as less subjective than size adjectives Scontras \BOthers. (\APACyear2017). All of this evidence suggests that the use of size adjectives may be more likely to vary across speakers and contexts than color.

Critically, our explanation of these phenomena departs from those offered by previous theories. \citeAPechmann1989 was the first to take the color-size asymmetry as evidence for speakers following an incremental strategy of object naming. That is, speakers initially start to articulate an adjective denoting a feature that listeners can quickly and easily recognize (i.e., color) before they have fully inspected the display and extracted the sufficient dimension. Another explanation appeals to saliency considerations: speakers may produce modifiers that denote features that are reasonably easy for the listener to perceive, so that, even when a feature is not fully distinguishing in context, it at least serves to restrict the number of objects that could plausibly be considered the target. Indeed, there has been some support for the idea that overmodification can be beneficial to listeners by facilitating target identification Arts \BOthers. (\APACyear2011); Rubio-Fernandez (\APACyear2016); Paraboni \BOthers. (\APACyear2007). The effect of scene variation on propensity to overmodify has typically been explained as the result of the demands imposed on visual search: in low-variation scenes, it is easier to discern the discriminating dimensions than in high-variation scenes, where it may be easier to simply start naming features of the target that are salient Koolen \BOthers. (\APACyear2013).

Finally, there have been various attempts to capture the color-size asymmetry in computational natural language generation models. The earliest contenders for models of definite referring expressions like the Full Brevity algorithm Dale (\APACyear1989) or the Greedy algorithm Dale (\APACyear1989) focused only on discriminatory value – that is, an utterance’s informativeness – in generating referring expressions. This is equivalent to the very simple interpretation of Grice’s Quantity maxim, and consequently these models demonstrated the same inability to capture the color-size asymmetry: they only produced the minimally specified expressions. Subsequently, the Incremental algorithm Dale \BBA Reiter (\APACyear1995) incorporated a preference order on features, with color ranked higher than size. The order is traversed and each encountered feature included in the expression if it serves to exclude at least one further distractor. This results in the production of overinformative color but not size adjectives. However, the resulting asymmetry is much greater than that evident in human speakers, and is deterministic rather than exhibiting the probabilistic production patterns that human speakers exhibit.

More recently, the PRO model van Gompel \BOthers. (\APACyear2019) has sought to integrate the observation that speakers seem to have a preference for including color terms with the observation that a preference does not imply the deterministic inclusion of said color term. In PRO, the uniquely distinguishing property (if there is one) is first selected deterministically. In additional steps, additional properties are added probabilistically, depending on both a salience parameter associated with the additional property and a parameter capturing speakers’ eagerness to overmodify. If both properties are uniquely distinguishing, a property is selected probabilistically depending on its associated salience parameter. The second step proceeds as before. This model successfully captures speakers’ overmodification patterns in contexts with one target and two distractors, in the choice of two properties (color, size) and three properties (color, size, border presence). While the PRO model – the most state-of-the-art computational model of human production of modified referring expressions – can capture the basic color-size asymmetry, it does not straightforwardly account for the more subtle systematicity with which the preference to overmodify with color changes based on scene variation or object typicality, which we turn to next.

4 Experiment 2: color typicality in modified referring expressions

Our modeling results in Experiment 1 raise interesting questions regarding the status of the inferred semantic values: do color modifiers have inherently higher semantic values than size modifiers? Is the difference constant? What if the color modifier is a less well known one like mauve? The way we have formulated the model thus far, there would indeed be no difference in semantic value between red and mauve. Moreover, the model is not equipped to handle potential object-level idiosyncracies such as the typicality effects discussed in Section 1.2: speakers are more likely to redundantly produce modifiers that denote atypical rather than typical object features, i.e., they are more likely to refer to a blue banana as a blue banana rather than as a banana, and they are more likely to refer to a yellow banana as a banana than as a yellow banana Sedivy (\APACyear2003); Westerbeek \BOthers. (\APACyear2015).

A natural first step toward explaining typicality effects is to introduce a more nuanced semantics for nouns in our model. In particular, we could imagine a continuous semantics in which banana fits better (i.e. has a semantic value closer to 1 for) the yellow banana than the brown, and fits the brown better than the blue; specific such hypothetical values are shown in the first row of Table 4. Let us further assume that modifying the noun with a color adjective leads to uniformly high semantic values close to 1 for those objects that a simple truth-conditional semantics would return ‘true’ for (see diagonal in Table 4) and a very low semantic value close to 0 for any utterance applied to any object that a simple truth-conditional semantics would return ‘false’ for.

The effect of running the speaker model forward with the standard literal listener treatment of the values in Table 4 for the three contexts in Figure 11, where banana is the strictly sufficient utterance for unique reference (i.e., color is redundant under the standard view) is as follows: with $\beta_{i}$ = 12 and $\beta_{c}$ = 5,222222The results hold qualitatively for any informativeness weight $>1$ and any cost weight $>0$. the resulting speaker probabilities for the minimal utterance banana are .95, .29, and .04, to refer to the yellow banana, the brown banana, and the blue banana, respectively. In contrast, the resulting speaker probabilities for the redundant yellow banana, brown banana, and blue banana are .05, .71, and .96, respectively. That is, redundant color mention increases with decreasing semantic value of the simple banana utterance.

This shows that cs-RSA can predict typicality effects if the semantic fit of the noun (and hence also of color-noun compounds) to an object is modulated by typicality. The reason the typicality effect arises is that, with the hypothetical values we assumed, the gain in informativeness between using the unmodified banana and the modified COLOR banana is greater in the blue than in the yellow banana case.

This example is somewhat oversimplified. In practice, speakers sometimes mention an object’s color without mentioning the noun. In the contexts presented in Figure 11 this does not make much sense because there is always a competitor of the same color present. In contrast, in the contexts in Figure 12(a) and Figure 12(c), color alone disambiguates the target. This suggests that we should consider among the set of utterance alternatives not just the simple type mentions (e.g., banana) and color-and-type mentions (e.g., yellow banana), but also simple color mentions (e.g., yellow). The dynamics of the model proceed as before.

An additional, more theoretically fraught, simplification concerns where typicality can enter into the semantics and how compositions proceeds. In the above, we have assumed that the semantic value of the modified expression is uniformly high, which is qualitatively what is necessary (and, as we will see below, empirically correct) in order for the typicality effects to emerge. However, there is no straightforward way to compositionally derive such uniformly high values from the semantic values of the nouns and the semantic values of the color modifiers, which we have not yet discussed. Indeed, compositional semantics of graded meanings is a well known problem for theories of modification Kamp \BBA Partee (\APACyear1995); Osherson \BBA Smith (\APACyear1981). Rather than try to solve it here, we note that RSA works at the level of whole utterances. Hence, if we can reasonably measure the semantic fit of each utterance to each possible referent, then cs-RSA will make predictions for production without the need to derive the semantic values compositionally. That is, if we can measure the typicality of the phrase blue banana for a banana, we don’t need to derive it from blue, banana, and a theory of composition. This separates pragmatic aspects of reference, which are the topic of this paper, from issues in compositional semantics, which are not; hence we will take this approach for experimentally testing the predictions of relaxed semantics RSA for typicality effects.

The stimuli for Exp. 1 were specifically designed to be realistic objects with low color-diagnosticity, so they did not include objects with low typicality values or large degrees of variation in typicality. This makes the dataset from Exp. 1 not well-suited for investigating typicality effects.232323We did elicit typicality norms for the items in Exp. 1 and replicated the previously documented typicality effects on the four items that did exhibit variation in typicality. See Appendix E for details. We therefore conducted a separate production experiment in the same paradigm but with two broad changes: first, objects’ color varied in typicality; and second, we did not manipulate object size, focusing only on color mention. This allows us to ask three questions: first, do we replicate the typicality effects reported in the literature – that is, are less color-typical objects more likely to lead to redundant color use than more color-typical objects? Second, does cs-RSA with empirically elicited typicality values as proxy for a continuous semantics capture speakers’ behavior? Third, does the semantic value depend only on typicality, or is there still a role for modifier type noise of the kind we investigated in the previous section? In addition, we can investigate the extent to which utterance cost, which we found not to play a role in the previous section, affects the choice of referring expression.

4.1 Method

Participants

We recruited 61 pairs of participants (122 participants total) over Amazon’s Mechanical Turk who were each paid $1.70 for their participation.

Procedure

The procedure of the reference game was identical to that of Exp. 1.

Materials

Each participant completed 42 trials. In this experiment, there were no filler trials, since pilot studies with and without fillers delivered very similar results. Each array presented to the participants consisted of three objects that could differ in type and color. One of the three objects functioned as a target and the other two as its distractors.

The stimuli were selected from seven color-diagnostic food items (apple, avocado, banana, carrot, pear, pepper, tomato), which all occurred in a typical, mid-typical and atypical color for that object. For example, the banana appeared in the colors yellow (typical), brown (midtypical), and blue (atypical). All items were presented as targets and as distractors. Pepper additionally occurred in a fourth color, which only functioned as a distractor due to the need for a green color competitor (as explained in the following paragraph).

We refer to the different context conditions as “informative”, “informative-cc”, “overinformative”, and “overinformative-cc” (see Figure 12). A context was “overinformative” (Figure 12(c)) when mentioning the type of the item, e.g., banana, was sufficient for unambiguously identifying the target. In this condition, the target never had a color competitor. This means that mentioning color alone (without a noun) was also unambiguously identifying. In contrast, in the overinformative condition with a color competitor (“overinformative-cc”, Figure 12(d)), color alone was not sufficient. In the informative conditions, color and type mention were necessary for unambiguous reference. Again, one context type did (Figure 12(a)) and one did not (Figure 12(d)) include a color competitor among its distractors.

Each participant saw 42 different contexts. Each of the 21 items (color-type combinations) was the target exactly twice, but the context in which they occurred was drawn randomly from the four possible conditions mentioned above. In total, there were 84 different possible configurations (seven target food items, each of them in three colors, where each could occur in four contexts). Trial order was randomized.

Data pre-processing and exclusion

We collected data from 1974 trials. The utterance produced on each trial was classified as belonging to one of the following categories: type-only (e.g., banana), color-and-type (e.g., yellow banana), and color-only (e.g., yellow). Referring expressions that could not be classified were excluded. See Appendix D for further details on exclusion criteria and the data pre-processing procedure. Overall, 1827 utterances entered the analysis.

4.2 Typicality norming

In order to test for typicality effects on the production data and to evaluate cs-RSA’s performance, we collected empirical typicality values for each utterance/object pair in three separate studies. The first study collected typicalities for color-and-type/object pairs (e.g., yellow banana as applied to a yellow banana, a blue banana, an orange pear, etc., see Figure 13(a)). The second study collected typicalities for type-only/object pairs (e.g., banana as applied to a yellow banana, a blue banana, an orange pear, etc., Figure 13(b)). The third study collected typicalities for color/color pairs (e.g., yellow as applied to a color patch of the average yellow from the yellow banana stimulus or to a color patch of the average orange from the orange pear stimulus, and so on, for all other colors, Figure 13(c)).

On each trial of the type or color-and-type studies, participants saw one of the stimuli used in the production experiment in isolation and were asked: “How typical is this object for a utterance”, where utterance was replaced by an utterance of interest. In the color typicality study, they were asked “How typical is this color for the color color?”, where color was replaced by one of the relevant color terms. They then adjusted a continuous sliding scale with endpoints labeled “very atypical” and “very typical” to indicate their response. A summary of the the three typicality norming studies is shown in Table 5.242424The typicality elicitation procedure we employed here is somewhat different from that employed by \citeAWesterbeek2015, who asked their participants “How typical is this color for this object?” We did this because the semantic values that enter into the RSA model are best conceptualized as the typicality of an object as an instance of an utterance, rather than a feature-category relation. See Appendix E for a comparison of our question and the Westerbeek question as applied to typicality norms for the items in Exp. 1. In general, the Type-object values are highly correlated with the Westerbeek question values.

Slider values were coded as falling between 0 (‘very atypical’) and 1 (‘very typical’). For each utterance-object combination, we computed mean typicality ratings. As an example, the means for the banana items and associated color patches are shown in Table 6. The values exhibit the same gradient as those hypothesized for the purpose of the example in Table 4. The means for all items are visualized in Figure 14. Mean typicality values for utterance-object pairs obtained in the norming studies are used in the analyses and visualizations in the following.

4.3 Behavioral results

Proportions of type-only (banana), color-and-type (yellow banana), color-only (yellow), and other (funky carrot) utterances are shown in Figure 15(a) as a function of the described item’s mean type-only (banana) typicality. Visually inspecting just the explicitly marked yellow banana, brown banana, and blue banana cases suggests a large typicality effect in the overinformative conditions as well as a smaller typicality effect in the informative conditions, such that color is less likely to be produced with increasing typicality of the object.

The following questions are of interest. First, do we replicate the previously documented typicality effect on redundant color mention (as suggested by the visual inspection of the banana item)? Second, does typicality affect color mention even when color is informative (i.e., technically necessary for establishing unique reference)? Third, are speakers sensitive to the presence of color competitors in their use of color or are typicality effects invariant to the distractor items?

To address these questions we conducted a mixed effects logistic regression predicting color use from fixed effects of typicality, informativeness, and color competitor presence. We used the typicality norms obtained in the type/object typicality elicitation study reported above (see Figure 13(b)) as the continuous typicality predictor. The informativeness condition was coded as a binary variable (color informative vs. color overinformative trial) as was color competitor presence (absent vs. present). All predictors were centered before entering the analysis. The model included by-speaker and by-item random intercepts, which was the most sophisticated random effects structure that allowed the model to converge.

We found a main effect of typicality, such that the more typical an object was for the type-only utterance, the lower the log odds of color mention ($\beta$ = -4.17, $SE$ = 0.45, $p<$ .0001), replicating previously documented typicality effects. Stepwise model comparison revealed that including interaction terms was not justified by the data, suggesting that speakers produce more typical colors less often even when the color is in principle necessary for establishing reference (i.e., in the informative conditions). This is notable: speakers sometimes call a yellow banana simply a banana even when other bananas are present, presumably because they can rely on listeners drawing the inference that they must have meant the most typical banana. In contrast, blue bananas’ color is always mentioned in the informative conditions.

There was also a main effect of informativeness, such that color mention was less likely when it was overinformative than when it was informative ($\beta$ = -5.56, $SE$ = 0.33, $p<$ .0001). Finally, there was a main effect of color competitor presence, such that color mention was more likely when a color competitor was absent ($\beta$ = 0.71, $SE$ = 0.16, $p<$ .0001). This suggests that speakers are indeed sensitive to the contextual utility of color – color typicality alone does not capture the full set of facts about color mention, as we already saw in Section 3.

4.4 Model evaluation

We evaluated the cs-RSA model on the obtained production data from Exp. 2. In particular, we were interested in using model comparison to address the following issues: First, can RSA using elicited typicality as the semantic values account for quantitative details of the production data? Second, are typicality values sufficient, or is there additional utility in including a noise offset determined by the type of modifier, as was used in the previous section? Third, does utterance cost explain any of the observed production behavior.

While the architecture of the model remained the same as that of the model presented in Section 2.2, we briefly review the minor necessary changes, some of which we already mentioned at the beginning of this section. These changes concerned the semantic values and the cost function.252525See Table 8 for an overview of the models reported in the paper.

4.4.1 Lexicon

Previously, we considered only three utterance alternatives: color, size, and color-size, collapsing over the precise values these took on. Here, we no longer collapse over these values, including in the lexicon each possible color adjective, type noun, and combination of the two. This substantially increased the size of the lexicon to 37 unique utterances. For each combination of utterance $u$ and object $o$ that occurred in the experiment, we included a separate semantic value $x_{u,o}$, elicited in the norming experiments described in Section 4.2 (rather than inferred as done for Exp. 1, to avoid overfitting). For any given context, we assumed the utterance alternatives that correspond to the individually present features and their combinations. For example, for the context in Figure 12(d), the set of utterance alternatives was yellow, green, pear, banana, avocado, yellow pear, yellow banana, and green avocado.

4.4.2 Semantics

We compared several choices of semantics for the model. In the full fixed plus empirical semantics version, we introduced a parameter $\beta_{\textrm{fixed}}$ interpolating between the empirically elicited typicality values ($\beta_{\textrm{fixed}}=0$) and the inferred type-level values as employed in Section 3 ($\beta_{\textrm{fixed}}=1$). The type-level values again consisted of one value for color terms and another for type terms, which are multiplied when the terms are composed in an utterance. In a lesioned empirical semantics version, we set $\beta_{\textrm{fixed}}=0$ and only used the empirical values. Conversly, in a lesioned fixed semantics version, we set $\beta_{\textrm{fixed}}=1$ and only used the inferred type-level values. This allowed us to perform a nested model comparison, since the latter models are special cases of the first.

4.4.3 Cost function

For the purpose of evaluating the model in Section 3 we inferred two constant costs (one for color and one for size), and found in the Bayesian Data Analysis that the role of cost in explaining the data was minimal at best. Here, we compared two different versions of utterance cost. In the fixed cost model we treated cost the same way as in the previous section and included only a color and type level cost, inferred from the data. We then compared this model to an empirical cost model, in which we included a more complex cost function. Specifically, we defined utterance cost $c(u)$ as follows:

[TABLE]

Here, $p(u)$ is negative log utterance frequency, as estimated from the Google Books corpus (years 1950 to 2008); $l(u)$ is the mean empirical length of the utterance in characters in the production data (e.g., sometimes yellow was abbreviated as yel, leading to an $l(u)$ smaller than 6); $\beta_{F}$ is a weight on frequency; and $\beta_{L}$ is a weight on length. Both $p(u)$ and $l(u)$ were normalized to fall into the interval $[0,1]$.262626Note that we changed the sign on frequency, which means that values closer to 1 in the normalized space reflect greater cost on both the length and the frequency dimension. The empirical cost function thus prefers short and frequent utterances (e.g., blue) over long and infrequent ones (turquoise-ish bananaesque thing). We compared both of these models to a simpler baseline in which utterances were assumed to have no cost.

4.4.4 Model comparison

To evaluate the effect of these choices of semantics and cost, we conducted a full Bayesian model comparison. Specifically, we computed the Bayes Factor for each comparison, a measure quantifying the support for one model over another in terms of the relative likelihood they each assign to the observed data. As opposed to classical likelihood ratios, which only use the maximum likelihood estimate, the likelihoods in the Bayes Factor integrate over all parameters, thus automatically correcting for the flexibility due to extra parameters (the “Bayesian Occam’s Razor”). Because it was intractable to analytically compute these integrals for our recursive model, we used Annealed Importance Sampling (AIS), a Monte Carlo algorithm commonly used to approximate these quantities. To ensure high-quality estimates, we took the mean over 100 independent samples for each model, with each chain running for 30,000 steps. The marginal log likelihoods for each model are shown in Table 7. The best performing model used fixed plus empirical semantics and did not include a cost term. Despite the greater number of parameters associated with adding the fixed semantics to the empirical semantics, the fixed plus empirical semantics models were preferred across the board compared to their empirical-only ($BF=3.7\times 10^{48}$ for fixed costs, $BF=2.1\times 10^{60}$ for empirical costs, and $BF=1.4\times 10^{71}$ for no cost) and fixed-only counterparts ($BF=6.5\times 10^{14}$ for fixed costs, $BF=1.0\times 10^{19}$ for empirical costs, and $BF=1.06\times 10^{15}$ for no cost). In comparison, additional cost-related parameters were not justified, with $BF=5.7\times 10^{21}$ for no cost compared to fixed cost and $BF=2.1\times 10^{27}$ for compared to empirical cost.

The correlation between empirical utterance proportions and the best model’s MAP predictions at the by-item level was $r=.94$. Predictions for the best-performing model are visualized alongside empirical proportions in Figure 15(b). The model successfully reproduces the empirically observed typicality effects in all four experimental conditions, with a reasonably good quantitative agreement. The interpolation weight between the fixed and empirical semantic values $\beta_{\textrm{fixed}}$ (Figure 16) is in the intermediate range: this provides evidence that a noisy truth-conditional semantics as employed in Exp. 1 is justified, but that taking into account graded category membership or typicality in an utterance’s final semantic value is also necessary.

There is one major, and interesting, divergence from the empirical data in conditions without color competitors. Here, color-and-type utterances are systematically somewhat underpredicted in the informative condition, and systematically somewhat overpredicted in the overinformative condition. The reverse is true for color-only utterances. It is worth looking at the posterior over parameters, shown in Figure 16, to understand the pattern. In particular, the utterance type level semantic value of type is inferred to be systematically higher than that of color, capturing that type utterances are less noisy than color utterances.272727Interestingly, the inferred semantic value for color is very similar in absolute terms to that in Exp. 1. An increase in color-only mentions in the overinformative condition could be achieved by reducing the semantic value for type. However, that would lead to a further and undesirable increase in color-only mentions in the informative condition as well. That is, the two conditions are in a tug-of-war with each other.

4.5 Discussion

In Experiment 2, we demonstrated that cs-RSA predicts color typicality effects in the production of referring expressions with only minimal extensions to support finer-grained prediction of item-by-item data (see Table 8 in the General Discussion for a more extensive comparison of the model details across sections). This suggests that the dynamics at work in the choice of color vs. size and in the choice of color as a function of the object’s color typicality are very similar: speakers choose utterances by considering the fine-grained differences in information about the intended referent communicated by the ultimately chosen utterance compared to its competitor utterances. For noisier utterances (e.g., banana as applied to a blue banana), including the ‘overinformative’ color modifier is useful because it provides information. For less noisy utterances (e.g., banana as applied to a yellow banana), including the color modifier is useless because the unmodified utterance is already highly informative with respect to the speaker’s intention. These dynamics can sometimes even result in the color modifier being left out altogether, even when there is another—very atypical—object of the same type present, simply because the literal listener is expected to prefer the typical referent strongly enough.

Model comparison demonstrated the need for assuming a semantics that interpolates between a noisy truth-conditional semantics as employed in Exp. 1 and empirically elicited typicality values. This may reflect semantic knowledge that goes beyond graded category membership, additional effects of compositionality, or perhaps simply differences between our empirical typicality measure and the “semantic fit” expected by RSA models. Perhaps surprisingly, we replicated the result from Exp. 1 that utterance cost does not add any predictive power, even when quantified via a more sophisticated cost function that takes into account an utterance’s length and frequency. In the next section, we move beyond the choice of modifier and ask whether cs-RSA provides a good account of referring expression production more generally.

5 Experiment 3: taxonomic level in unmodified referring expressions

In this section we investigate whether cs-RSA accounts for referring expression production beyond the choice of modifier. In particular, we focus on speakers’ choice of taxonomic level of reference in nominal referring expressions. A particular object can be referred to at its subordinate (dalmatian), basic (dog), or superordinate (animal) level, among other choices. This choice of reference level is interestingly different from that of adding modifiers in that there is no additional word-level cost associated with being more specific – the choice is between different one-word utterances, not between utterances differing in word count. Still, we hypothesized that similar factors may contribute: an expression’s contextual informativeness, its cognitive cost <short and frequent terms are preferred over long and infrequent ones,¿griffin1998,jescheniak1994, and its typicality <an utterance is more likely to be used if the object is a good instance of it,¿ Jolicoeur1984.

In order to evaluate cs-RSA for nominal choice, we proceeded as in Section 4: we collected production data within the same reference game setting, but varied the contextual informativeness of utterances by varying whether distractors shared the same basic or superordinate category with the target (see Figure 17). We also elicited typicality ratings for object-utterance combinations, which entered the model as the semantic values via the lexicon. We then conducted Bayesian data analysis, as in previous sections, for model comparison. Our key insight is that compared to a traditional Boolean semantics where class labels (e.g. dog) are strictly true or false of objects, a continuous semantics incorporating knowledge of typicality more successfully predicts preferences for taxonomic level of reference. That is, cs-RSA accurately predicts, from Gricean principles, that speakers will increase their preference for subordinate terms (e.g. penguin) when an object is a more atypical example of the basic-level (bird) and prefer basic-level terms (e.g. bird) when those objects are more atypical examples of the super-ordinate level (animal).

5.1 Method

Participants

We recruited 58 pairs of participants (116 participants total, the same participants as in Exp. 1) over Amazon’s Mechanical Turk who were each paid $1.75 for their participation.

Procedure and materials

The procedure was identical to that of Exp. 1.282828A separate earlier data set was reported at the annual meeting of the Cognitive Science Society Graf \BOthers. (\APACyear2016), and serves as a close replication of the reported study. Participants proceeded through 72 trials. Of these, half were critical trials of interest and half were filler trials (the critical trials from Exp. 1). On critical trials, we varied the level of reference that was sufficient to mention for uniquely establishing reference.

Stimuli were selected from nine distinct domains, each corresponding to distinct basic level categories such as dog. For each domain, we selected four subcategories to form our target set (e.g. dalmatian, pug, German Shepherd and husky). See Table 11 in Appendix F for a full list of domains and their associated target items. Each domain also contained an additional item which belonged to the same basic level category as the target (e.g., greyhound) and items which belonged to the same supercategory but not the same basic level (e.g., elephant or squirrel). The latter items were used as distractors.

Each trial consisted of a display of three images, one of which was designated as the target object. Each pair of participants saw each target exactly once, for a total of 36 trials. These target items were randomly assigned distractor items which were selected from three different context conditions, corresponding to different communicative pressures (see Figure 17). The subordinate necessary contexts contained one distractor of the same basic category and one distractor of the same superordinate category (e.g., target: dalmatian, distractors: greyhound (also a dog) and squirrel (also an animal)). The basic sufficient contexts contained either two distractors of the same superordinate category but different basic category as the target (e.g., target: husky, distractors: hamster and elephant) or one distractor of the same superordinate category and one unrelated item (e.g., target: pug, distractors: cow and table). The superordinate sufficient contexts contained two unrelated items (e.g., target: German Shepherd, distractors: shirt and cookie).

This context manipulation served as a manipulation of utterance informativeness: any target could be referred to at the subordinate (dalmatian), basic (dog) or superordinate (animal) level. However, the level of reference necessary for uniquely referring differed across contexts.

Data pre-processing and exclusion

We collected data from 2193 critical trials. Each referring expression was classified as containing the target’s correct sub(ordinate, e.g., dalmatian), basic (e.g., dog), or super(ordinate, e.g., animal) level term, or excluded if classification was not possible. See Appendix D for details on exclusion criteria and the pre-processing procedure. After exclusions and pre-processing, 1872 cases entered the analysis.

Typicality norming

In order to test for typicality effects on the production data and to evaluate cs-RSA’s performance, we again collected empirical typicality values for each utterance/object pair. See Appendix G for details.

5.2 Behavioral results and discussion

Proportions of sub, basic, and super level utterances are shown in Figure 18. Overall, super level mentions are highly dispreferred ($<2\%$), so we focus in this section only on predictors of sub over basic level mentions. The clearest pattern of note is that sub level mentions are only preferred in the most constrained context that necessitates the sub level mention for unique reference (e.g., target: dalmatian, distractor: greyhound; see Figure 17(a)). Nevertheless, even in these contexts there is a non-negligible proportion of basic level mentions (28%). This includes cases of using just the basic level term (6%, e.g., dog for the German Shepherd when one of the distractors was a greyhound, an atypical dog, akin to the unmodified cases in the informative conditions discussed in Section 4.3) as well as basic level terms with additional modifying material (22%). In the remaining contexts, where the sub and basic level are equally informative, there is a clear preference for the basic level. Finally, mitigating this context effect, sub level mentions increased with increasing typicality of the object as an instance of the sub level utterance.

What explains these preferences? In order to test for effects of informativeness, length, frequency, and typicality on nominal choice we conducted a mixed effects logistic regression predicting sub over basic level mention from centered predictors for the factors of interest. Because the maximal model with by-speaker and by-item slopes for all fixed effects did not converge, we simplified the random effects structure, including only by-speaker and by-item random intercepts.

Frequency was coded as the difference between the sub and the basic level’s log frequency, as extracted from the Google Books Ngram English corpus ranging from 1960 to 2008. Speakers prefer more frequent words over less frequent ones Oldfield \BBA Wingfield (\APACyear1965).

Length was coded as the ratio of the subordinate to the basic level term’s length in characters. Even among one-word utterances, speakers prefer shorter ones over longer ones Degen \BOthers. (\APACyear2013); Rohde \BOthers. (\APACyear2012). We used the mean number of characters in the utterances participants produced. For example, the minivan, when referred to at the subcategory level, was sometimes called “minivan” and sometimes “van” leading to a mean empirical length of 5.71. This is the value that was used, rather than 7, the length of “minivan”.

Typicality was coded as the ratio of the target’s sub to basic level label typicality.292929Typicalities were elicited in a separate norming study that was identical in procedure to that of Exp. 1a. See Appendix G for details about the study. That is, the higher the ratio, the more typical the object was for the sub level label compared to the basic level; or in other words, a higher ratio indicates that the object was relatively atypical for the basic label compared to the sub label. For instance, the panda was relatively atypical for its basic level “bear” (mean rating 0.75) compared to the sub level term “panda bear” (mean rating 0.98), which resulted in a relatively high typicality ratio. We predicted that subordinate terms may be preferred when an object is a particularly good instance of that term or a particularly bad instance of the basic level term, compared to the other objects in the context.

Informativeness condition was coded as a three-level factor: sub necessary, basic sufficient, and super sufficient, where basic sufficient (two superordinate distractors) and basic sufficient (one superordinate distractor) were collapsed into basic sufficient. Condition was Helmert-coded: two contrasts over the three condition levels were included in the model, comparing each level against the mean of the remaining levels (in order: sub necessary, basic sufficient, super sufficient). This allowed us to determine whether the probabilities of type mention for neighboring conditions were significantly different from each other, as suggested by Figure 18.

The log odds of mentioning the sub level term were greater in the sub necessary condition than in either of the other two conditions ($\beta=2.11$, $SE=.17$, $p<.0001$), and greater in the basic sufficient condition than in the super sufficient condition ($\beta=.60$, $SE=.15$, $p<.0001$), suggesting that the contextual informativeness of the sub level mention has a gradient effect on utterance choice.303030Importantly, model comparison between the reported model and one that subsumes basic and super under the same factor level revealed that the three-level condition variable is justified ($\chi^{2}(1)=12.82$, $p<.0004$), suggesting that participants do not simply revert to the basic level when no basic-level distractor is in context. There was also a main effect of typicality, such that the sub level term was preferred for objects that were more typical for the sub level compared to the basic level description ($\beta=4.82$, $SE=1.35$, $p<.001$). In addition, there was a main effect of length, such that as the length of the sub level term increased compared to the basic level term (“chihuahua”/“dog” vs. “pug”/“dog”), the sub level term was dispreferred (“chihuahua” is dispreferred compared to “pug”, $\beta=-.95$, $SE=.27$, $p<.001$). The main effect of frequency did not reach significance ($\beta=.08$, $SE=.11$, $p<.45$).

Unsurprisingly, there was also significant by-participant and by-domain variation in sub level term mention. For instance, mentioning the sub over the basic level term was preferred more in some domains (e.g. in the “candy” domain) than in others. Likewise, some domains had a greater preference for basic level terms (e.g. the “shirt” domain). Using the super term also ranged from hardly being observable (e.g., plant in the “flower” domain) to being used more frequently (e.g., furniture in the “table” domain and vehicle in the “car” domain). We thus replicated the well-documented preference to refer to objects at the basic level, which is partly modulated by contextual informativeness and partly a result of the basic level term’s cognitive cost and typicality compared to its sub level competitor, mirroring the results from Exp. 2.

Perhaps surprisingly, we did not observe an effect of frequency on sub level term mention. This is likely due to the modality of the experiment: the current study was a written production study, while most studies that have identified frequency as a factor governing production choices are spoken production studies. It may be that the cognitive cost of typing longer words may be disproportionately higher than that of producing longer words in speech, thus obscuring a potential effect of frequency. Support for this hypothesis comes from studies comparing written and spoken language, which has found that spoken descriptions are likely to be longer than written descriptions and, in English, seem to have a lower propositional information density than written descriptions van Miltenburg \BOthers. (\APACyear2018).313131In order to address convergence issues with lmer+ when specifying the full random effects structure – i.e., by-speaker and by-item random intercepts and slopes for all fixed effects – we also ran a Bayesian binomial mixed effects model with weakly informative priors using the brms+ package Bürkner (\APACyear2017) that included the same fixed effects structure as the lmer model and the full random effects structure. The results were qualitatively identical, yielding evidence for main effects of context (sub vs basic sufficient: posterior mean $\beta$ = 2.44, $95\%$ CI = $[$1.87,3.06$]$, $p(\beta>0)$ = 1; basic vs super sufficient: posterior mean $\beta$ = 0.70, $95\%$ CI = $[$0.32,1.09$]$, $p(\beta>0)$ = 1), typicality (posterior mean $\beta$ = 9.96, $95\%$ CI = $[$3.55,17.51$]$, $p(\beta>0)$ = 1), and length (posterior mean $\beta$ = -1.12, $95\%$ CI = $[$-2.00,-0.31$]$, $p(\beta<0)$ = 1).

5.3 Model evaluation

We evaluated cs-RSA on the production data from Exp. 3. The architecture of the model is identical to that of the model presented in Section 4.4. The only difference is the set of alternative utterances.323232See Table 8 for an overview of the models reported in the paper. Whereas the models from the previous sections treated all individual features and feature combinations present in the display as utterance alternatives, for computational efficiency we now consider only the three different levels of reference to the target as alternatives, i.e. subordinate (dalmatian), basic (dog), and superordinate (animal). So, even when a German Shepherd is present as a distractor, German Shepherd is not considered an alternative utterance for the dalmatian target. This has minimal effects on model predictions as long as German Shepherd has low semantic fit to the dalmatian target.

In Experiment 2, we tested which of three different semantics was most justified – the empirically elicited typicality semantics, a fixed semantics with type-level semantic values, and one that combined both. Here, the relevant comparison is between a fixed Boolean noun semantics (i.e. 1 if the object belongs to the given class label, and 0 otherwise) and the empirically elicited typicality semantics. Again, we introduced a parameter that interpolated between these semantic values. Additionally, we evaluated which cost function was best supported by the data: the one defined in (7) (a linear weighted combination of an utterance’s length and its frequency) or a simpler baseline in which utterances were assumed to have no cost.

We employed the same procedure as in the previous section to compute the Bayes Factor for the comparison between the two cost models, and to compute the posteriors over parameters. Priors were again $\beta_{i}\sim\mathcal{U}(0,20)$, $\beta_{F}\sim\mathcal{U}(0,5)$, $\beta_{L}\sim\mathcal{U}(0,5)$, $\beta_{t}\sim\mathcal{U}(0,5)$. Despite the greater number of parameters associated with adding the cost function, the model that includes non-zero costs was preferred compared to its no-cost counterpart ($BF=2.8\times 10^{77}$). Posteriors over parameters are shown in Figure 19. First, we observe that the semantic interpolation value was highly skewed toward 0, strongly indicating that empirical typicality values strongly improve model performance over a Boolean baseline. Second, the weight on frequency is close to zero. That is, in line with the results from the mixed effects regression, it is an utterance’s length, but not its frequency, that affects the probability with which it is produced in this paradigm.333333As discussed in previous sections, the lack of importance of a word’s frequency may well be attributable to the written modality within which participants generated referring expressions.

MAP model predictions are shown alongside empirical utterance proportions in Figure 18. The correlation between empirical utterance proportions and the model’s MAP predictions at the level of targets, utterances, and conditions was $r=.86$. Further collapsing across targets yields a correlation of $r=.95$. The model captures the qualitative patterns well, though it somewhat overpredicts subordinate level and underpredicts basic level choices. It also accounts for the strong preference against super level mentions. The reason for this is that the semantics for each utterance (eg., dalmatian, dog, animal) is taken from the empirically elicited typicality values for each utterance-object pair. As can be seen in the left panel of Figure 24, the target images used in this experiment were generally rated as less typical instances of the superordinate level term than of the basic or subordinate level term. This difference is enough to lead to a general bias against using the superordinate level term, especially when coupled with the fact that superordinate terms tend to be costlier than basic level terms.

6 General Discussion

In this paper we have provided a unified account of referring expression choice that solves a long-recognized puzzle for rational theories of language use: why do speakers’ referring expressions often and systematically exhibit seeming overinformativeness? We have shown here that by allowing contextual utterance informativeness to be computed with respect to a continuous (or noisy) rather than a Boolean semantics, utterances that seem overinformative can in fact be sufficiently informative. This happens when what seems like the prima facie sufficiently informative utterance is in fact noisy and may lead a literal listener astray; adding redundancy ensures successful communication. This simple modification to the Rational Speech Act approach allowed us to capture: the basic well-documented asymmetry for speakers to be more likely to redundantly use color adjectives than size adjectives; the interaction between sufficient dimension and scene variation in the probability of redundancy; and typicality effects in both color modifier choice and noun choice.

We have thus shown that with one key innovation – a continuous semantics – one can retain the assumption that speakers rationally trade off informativeness and cost of utterances in language production. Rather than being wastefully overinformative, adding redundant modifiers or referring at a lower taxonomic level than strictly necessary is in fact appropriately informative. This innovation thus not only provides a unified explanation for a number of key patterns within the overinformative referring expression literature that have thus far eluded a unified explanation; it also extends to the domain of nominal choice. And in contrast to previously proposed computational models, it is straightforwardly extendable to any instance of definite referring expressions of the sort we have examined here.

6.1 Comparison of model components across experiments

In order to address the possible concern that the different models employed are too different from one another to be comparable, we begin by providing an overview of the parts of the model that remained the same or differed across experiments. While the core architecture with relaxed semantics remained constant throughout the paper, some peripheral components were adjusted to accommodate the aims of the different experiments. These different choices are fully consistent with one another, and many of them were justified against alternatives via model comparison. We have provided an overview of the best-fitting RSA models for each of the three reported production datasets in Table 8.

Most prominently, Exps. 2 and 3 aimed to predict patterns of reference via typicality at the object-level; in those cases the model thus required semantic values for each utterance-object pair in the lexicon. While these values could have in principle been inferred from the data, as we inferred the two type-level values in Exp. 1, it would have introduced a large number of additional parameters (see size of lexicon). Instead, we addressed this problem by empirically eliciting these values in an independent task and introducing a single free concentration parameter $\beta_{t}$ that modulated their strength. In the case of Exp. 2, we found that the best-fitting model smoothly integrated these empirical values with type-level values used in Exp. 1.

The need to make object-level predictions also drove decisions about what to use as the cost function and the set of alternative utterances. For instance, in Exp. 3 we could have inferred the cost of each noun but this again would have introduced a large number of free parameters and risked overfitting. Instead we used the empirically estimated length and frequency of each word. For Exp. 2, we tested models both using fixed costs for each modifier as in Exp. 1 and empirical length and frequency costs as in Exp. 3, but our model comparison showed that neither sufficiently improved the model’s predictions.

Finally, the set of alternative utterances differed slightly across the three experiments for computational reasons. Because Exp. 1 collapsed over the particular levels of size and color, it was practical to consider all utterances in the lexicon for every target. In Exp. 2 and Exp. 3, however, the space of possible utterances was large enough that this exhaustive approach became impractical. We noticed that the probability of using some utterances (e.g. ‘table’ to refer to a Dalmatian) was low enough that we could prune the utterance space to only those that could plausibly apply to the objects in context without substantially altering the model’s behavior. Future work must address how predictions may change as more complex referring expressions outside the scope of this paper enter the set of alternatives (e.g. the option of combining adjectives with nominal expressions, as in the cute, spotted dog). In the following we discuss a number of intriguing questions that this work raises and avenues for future research it suggests.

6.2 Comparison with PRO

While a detailed comparison of cs-RSA with PRO van Gompel \BOthers. (\APACyear2019), the hitherto most state-of-the-art computational model of human production of modified referring expressions, is outside the scope of this paper, we include some comparative remarks here. PRO has the advantage of being computationally more efficient than cs-RSA, partly because it aims to be an algorithmic-level model, which may be of importance for applications. PRO may further have the advantage of having fewer parameters, though this is a bit harder to evaluate in general: while PRO as applied to the choice of color and size in principle involves 2 parameters, $s$ (size preference) and $e$ (overspecification eagerness), in the 2019 paper the maximum likelihood parameter values are estimated on each of the experimental conditions separately, effectively resulting in 6 parameters. In the extension to 3 properties, this results in 14 parameters, and this number increases further as more properties and conditions are added. If the parameter values had been estimated on all conditions jointly, which is what we did in the evaluation of cs-RSA instead of separately, then indeed PRO would have fewer effective parameters than cs-RSA, though it is unclear how this would affect the data fit.

One advantage of the cs-RSA approach is greater generality. For instance, it is not immediately clear how PRO should be extended to contexts that vary in the number and nature of distractors, where empirical overmodification proportions change, but the PRO predictions would not. We see this as one of the great strengths of cs-RSA: scene variation effects fall out of the model directly. Finally, we have proposed here a way to account for typicality effects. PRO may be able to accommodate typicality effects if its preference parameters can be made to be sensitive to typicality. In general, a systematic comparison and possible combination of these models is an important next step.

6.3 ‘Overinformativeness’

This work challenges the traditional notion of overinformativeness as it is commonly employed in the linguistic and psychological literature. The reason that redundant referring expressions are interesting for psycholinguists to study is that they seem to constitute a clear violation of rational theories of language production. For example, Grice’s Quantity-2 maxim, which asks of speakers to “not make [their] contribution more informative than is required” Grice (\APACyear1975), appears violated by any redundant referring expression – if one feature uniquely distinguishes the target object from the rest and a second one does not, mentioning the second does not contribute any information that is not already communicated by the first. Hence, the second is considered ‘overinformative’, a referring expression that contains it ‘overspecified.’

This conception of (over-)informativeness assumes that all modifiers are born equal – i.e., that there are no a priori differences in the utility of mentioning different properties of an object. Under this conception of modifiers, there are hard lines between modifiers that are and aren’t informative in a context. However, what we have shown here is that under a continuous semantics, a modifier that would be regarded as overinformative under the traditional conception may in fact communicate information about the referent. The more visual variation there is in the scene, and the less noisy the redundant modifier is compared to the modifier that selects the dimension that uniquely singles out the target, the more information the redundant modifier adds about the referent, and the more likely it therefore is to be mentioned. This work thus challenges the traditional notion of utterance overinformativeness by providing an alternative that captures the quantitative variation observed in speakers’ production in a principled way while still assuming that speakers are aiming to be informative, and is compatible with other efficiency-based accounts of ‘overinformative’ referring expressions <e.g.,¿sedivy2003a,rubiofernandez2016.

But this raises a question: what counts as a truly overinformative utterance under RSA with a continuous semantics? Cs-RSA shifts the standard for overinformativeness and turns it into a graded notion: the less expected the use of a redundant modifier is contextually, the more the use of that modifier should be considered overinformative. For example, consider again Figure 8: the less scene variation there is, the more truly overinformative the use of the redundant modifier is. Referring to the big purple stapler when there are only purple staplers in the scene should be considered overinformative. If there is one red stapler, the utterance should be judged less overinformative, and the more non-purple staplers there are, the less overinformative the utterance should be judged. We leave a systematic test of this prediction for our stimuli for future research, though we point to some qualitative examples where it has been borne out previously in the next subsection.

6.4 Comprehension

While the account proposed in this paper is an account of the production of referring expressions, it can be extended straightforwardly to comprehension. RSA models typically assume that listeners interpret utterances by reasoning about their model of the speaker. In this paper we have provided precisely such a model of the speaker. In what way should the predicted speaker probabilities enter into comprehension? There are two interpretations of this question: first, what is the ultimate interpretation that listeners who reason about speakers characterized by the model provided in this paper arrive at, i.e. what are the predictions for referent choice? And second, how do the production probabilities enter into online processing of prima facie overinformative utterances? The first question has a clear answer. For the second question we offer a more speculative answer.

6.4.1 Choice of referent

Most RSA reference models, unlike the one reported in this paper, have focused on comprehension M\BPBIC. Frank \BBA Goodman (\APACyear2012); Degen \BOthers. (\APACyear2013); Qing \BBA Franke (\APACyear2015); Franke \BBA Degen (\APACyear2016). The formula that characterizes pragmatic listeners’ referent choices is:

[TABLE]

That is, the pragmatic listener interprets utterance $u$ (e.g., the big purple stapler) via Bayesian inference, taking into account both the speaker probability of producing the big purple stapler and its alternatives, given a particular object $o$ the speaker had in mind, as well as the listener’s prior beliefs about which object the speaker is likely to intend to refer to in the context. For the situations considered in this paper, in which the utterance is semantically compatible with only one of the referents in the context, this always predicts that the listener should choose the target. And indeed, in Exps. 1-3 the error rate on the listeners’ end was always below 1%. From a referent choice point of view, then, these contexts are not very interesting. They are much more interesting from an online processing point of view, which we discuss next.

6.4.2 Online processing

The question that has typically been asked about the online processing of redundant utterances is this: do redundant utterances, compared to their minimally specified alternatives, help or hinder comprehenders in choosing the intended referent? ‘Help’ and ‘hinder’ are typically translated into ‘speed up’ and ‘slow down’, respectively. What does the RSA model presented here have to say about this?

In sentence processing, the current wisdom is that the processing effort spent on linguistic material is related to how surprising it is Hale (\APACyear2001); Levy (\APACyear2008). In particular, an utterance’s log reading time is linear in its surprisal Smith \BBA Levy (\APACyear2013), where surprisal is defined as $-\log p(u)$. In these studies, surprisal is usually estimated from linguistic corpora. Consequently, an utterance of the big purple stapler receives a particular probability estimate independent of the non-linguistic context it occurred in. Here we provide a speaker model from which we can derive estimates of pragmatic surprisal directly for a particular context. We can thus speculate on a linking hypothesis: the more expected a redundant utterance is under the pragmatic continuous semantics speaker model, the faster it should be to process compared to its minimally specified alternative, all else being equal. We have shown that redundant expressions are more likely than minimal expressions when the sufficient dimension is relatively noisy and scene variation is relatively high. Under our speculative linking hypothesis, the redundant expression should be easier to process in these sorts of contexts than in contexts where the redundant expression is relatively less likely.

Is there evidence that listeners do behave in accordance with this prediction? Indeed, the literature reports evidence that in situations where the redundant modifier does provide some information about the referent, listeners are faster to respond and select the intended referent when they observe a redundant referring expression than when they observe a minimal one Arts \BOthers. (\APACyear2011); Paraboni \BOthers. (\APACyear2007). However, there is also evidence that redundancy sometimes incurs a processing cost: both \citeAEngelhardt2011 and \citeADavies2013 (Exp. 2) found that listeners were slower to identify the target referent in response to redundant compared to minimal utterances. It is useful to examine the stimuli they used. In the Engelhardt et al study, there was only one distractor that varied in type, i.e., type was sufficient for establishing reference. This distractor varied either in size or in color. Thus, scene variation was very low and redundant expressions therefore likely surprising. Interestingly, the incurred cost was greater for redundant size than for redundant color modifiers, in line with the RSA predictions that color should be generally more likely to be used redundantly than size. In the Davies et al. study, the ‘overinformative’ conditions contained displays of four objects which differed in type. Stimuli were selected via a production pre-test: only those objects that in isolation were not referred to with a modifier were selected for the study. That is, stimuli were selected precisely on the basis that redundant modifier use would be unlikely.

While the online processing of redundant referring expressions is yet to be systematically explored under the cs-RSA account, this cursory overview of the patterns reported in the existing literature suggests that pragmatic surprisal may be a plausible linking function from model predictions to processing times. Excitingly, it has the potential for unifying the equivocal processing time evidence by providing a model of utterance probabilities that can be computed from the features of the objects in the context.

6.5 Continuous semantics

The crucial component of the model that allows for capturing ‘overinformativeness’ effects is the continuous semantics. In this section, we consider the nature of these continuous semantic values. Readers already convinced of the utility of a continuous semantics are invited to skip to the next section.

For the purpose of Exp. 1 (modifier choice), a semantic value was assigned to modifier type. The semantics of modifiers was underlyingly truth-conditional and the semantic value captured the probability that a modifier’s truth conditions would accidentally be inverted. This model included only two semantic values, one for size and one for color, which we inferred from the data. For the datasets from Exps. 2 and 3, we then extended the continuous semantics to apply at the level of utterance-object combinations (e.g., banana vs. blue banana as applied to the blue banana item, dalmatian vs. dog as applied to the dalmatian item) to account for typicality effects in modifier and nominal choice. In this instantiation of the model, the semantic value differed for every utterance-object combination and captured how good of an instance of an utterance an object was. These values were elicited experimentally to avoid over-fitting, and for the dataset from Exp. 2 we found further that a combination of a relaxed (noisy) truth-conditional semantics and the empirically elicited continuous semantics best accounted for the obtained production data.

What we have said nothing about thus far is what determines these semantic values; in particular, which aspects of language users’ experience – perceptual, conceptual, communicative, linguistic – they represent. We will offer some speculative remarks and directions for future research here.

First, semantic values may represent the difficulty associated with verifying whether the property denoted by the utterance holds of the object. This difficulty may be perceptual – for example, it may be relatively easier to visually determine of an object whether it is red than whether it it is big (at least in our stimuli). Similarly, at the object-utterance level, it may be easier to determine of a yellow banana than of a blue banana whether it exhibits banana-hood, consequently yielding a lower semantic value for a blue banana than for a yellow banana as an instance of banana. Further, the value may be context-invariant or context-dependent. If it is context-invariant, the semantic value inferred for color vs. size, for instance, should not vary by making size differences more salient and color differences less salient. If, instead, it is context-dependent, increasing the salience of size differences and decreasing the salience of color differences should result, e.g., in color modifiers being more noisy, with concomitant effects on production, i.e., redundant color modifiers should become less likely. This is indeed what \citeAViethen2017 found. Similarly, \citeAVanGompel2014 found that the asymmetry in redundant use of color vs. with size disappeared when participants were shown displays with very noticeable size contrasts and barely noticeable color contrasts.

Another possibility is that semantic values represent aspects of agents’ prior beliefs (world knowledge) about the correlations between features of objects. For example, conditioning on an object being a banana, experience dictates that the probability of it being yellow is much greater than of it being blue. This predicts the relative ordering of the typicality values we elicited empirically, i.e., the blue banana received a lower semantic value than the yellow banana as an instance of banana.

Another possibility is that the semantic values capture the past probability of communicative success in using a particular expression. For example, the semantic value of banana as applied to a yellow banana may be high because in the past, referring to yellow bananas simply as banana was on average successful. Conversely, the semantic value of banana as applied to a blue banana may be low because in the past, referring to blue bananas simply as banana was on average unsuccessful (or the speaker may have uncertainty about its communicative success because they have never encountered blue bananas before). Similarly, the noise difference between color and size modifiers may be due to the inherent relativity of size modifiers compared to color modifiers – while color modifiers vary somewhat in meaning across domains (consider, e.g., the difference in redness between red hair and red wine), the interpretation of size modifiers is highly dependent on a comparison class (consider, e.g., the difference between a big phone and a big building). In negotiating what counts as red, then, speakers are likely to agree more often than in negotiating what counts as big. That is, size adjectives are more subjective than color adjectives. If semantic values encode adjective subjectivity, speakers should be even more likely to redundantly use adjectives that are more objective than color. In a study showing that adjective subjectivity is almost perfectly correlated with an adjective’s average distance from the noun, \citeAscontras2017 collected subjectivity ratings for many different adjectives and found that material adjectives like wooden and plastic are rated to be even more objective than color adjectives. Thus, under the hypothesis that semantic values represent adjective subjectivity, material adjectives should be even more likely to be used redundantly than color adjectives. This is not the case. For instance, \citeAsedivy2003a reports that material adjectives are used redundantly about as often as size adjectives. Hence, while the hypothesis that semantic values capture the past probability of communicative success in using a particular expression has yet to be systematically investigated, subjectivity alone seems not to be the determining factor.

Finally, it is also possible that semantic values are simply an irreducible part of the lexical entry of each utterance-object pair. This seems unlikely because it would require a separate semantic value for each utterance and object token, and most potentially encounterable object tokens in the world have not been encountered, making it impossible to store utterance-token-level values. However, it is possible that, reminiscent of prototype theory, semantic values are stored at the level of utterances and object types. This view of semantic values suggests that they should not be updated in response to further exposure of objects. For example, if semantic values were a fixed component of the lexical entry banana, then even being exposed to a large number of blue bananas should not change the value. This seems unlikely but merits further investigation.

The various possibilities for the interpretation of the continuous semantic values included in the model are neither independent nor incompatible with each other. Disentangling these possibilities presents an exciting avenue for future research.

What is highlighted by the above discussion is that we have been using the term ‘semantics’ at a fairly high level, to refer to conventional aspects of meaning that are relatively stable across contexts – in RSA these are the representations on which the literal listener performs computations. These real valued representations could be primitives arising in lexical representations and threading through composition, which would constitute a fundamentally different basic semantics than has often been assumed. Alternatively, the necessary real values could arise by adding the right kind of use or world-knowledge related noise to a standard Boolean truth-conditional semantics. Most minimally, intensional parameters of a standard semantics could be set stochastically. Whatever their source, these continuous values provide the right basis for capturing the production choices explored in this paper. When to assume a relaxed semantics, and what the implications of such a relaxation are for other semantic and pragmatic phenomena, are questions for future research.

6.6 Audience design

One question which has plagued the literature on language production is that of whether, and to what extent, speakers actually tailor their utterances to their audience Clark \BBA Murphy (\APACyear1982); Horton \BBA Keysar (\APACyear1996); Brown-Schmidt \BBA Heller (\APACyear2014). This is also known as the issue of audience design. With regards to redundant referring expressions, the question is whether speakers produce redundant expressions because it is helpful to them (i.e., due to internal production pressures) or because it is helpful to their interlocutor. For instance, \citeAWalker1993 shows that redundancy is more likely when processing resources are limited. On the other hand, there is evidence that redundant utterances are frequently used in response to signs of listener non-comprehension, when responding to listener questions, or when speaking to strangers Baker \BOthers. (\APACyear2008). Audience design has played an especially large role in explanations of typicality effects. For example, \citeAHuettig2011 found that listeners, after hearing a noun with a diagnostic color (e.g., frog), are more likely to fixate objects of that diagnostic color (green), indicating that typical object features like color are rapidly activated and aid visual search. Similarly, \citeAArts2011 showed that overspecified expressions result in faster referent identification. Nevertheless, such benefits might simply be a happy coincidence and speakers might not, in fact, be deliberately designing their utterances for their addressees.

RSA seems to make a claim about this issue: speakers are trying to be informative with respect to a literal listener. That is, it would seem that speakers produce referring expressions that are tailored to their listeners. However, this is misleading. The ontological status of the literal listener is as a “dummy component” that allows the pragmatic recursion to get off the ground. Actual listeners are, in line with previous work and briefly discussed above, more likely fall into the class of pragmatic $L_{1}$ listeners; listeners who reason about the speaker’s intended meaning via Bayesian inference M\BPBIC. Frank \BBA Goodman (\APACyear2012); Goodman \BBA Stuhlmüller (\APACyear2013).343434But see \citeAFrankeDegen2016 for an evaluation of the distribution of listener and speaker types in Quantity inferences. Because RSA is a computational-level theory Marr (\APACyear1982) of language use, it does not claim that the mechanism of language production requires that speakers actively consult an  internal model of a listener every time they choose an utterance, just that the distribution of utterances they produce reflect informativity with respect to such a model. It is possible that this distribution is cached or computed using some other algorithm that doesn’t explicitly involve a listener component. Thus, the RSA model as formulated here remains agnostic about whether speakers’ (over-) informativeness should be considered geared towards listeners’ needs or simply a production-internal process. Instead, the claim is that redundancy emerges as a property of the communicative situation as a whole.

6.7 Other factors that affect redundancy

RSA with a continuous semantics as presented in this paper straightforwardly accounts for effects of typicality, cost, and scene variation on redundancy in referring expressions. However, other factors have been identified as contributing to redundancy. For example, \citeArubiofernandez2016 showed that colors are mentioned more often redundantly for clothes than for geometrical shapes. Her explanation is that knowing an object’s color is generally more useful for clothing than it is for shapes. It is plausible that agents’ knowledge of goals may be relevant here. For example, knowing the color of clothing is relevant to the goal of deciding what to wear or buy. In contrast, knowing the color of geometrical shapes is rarely relevant to any everyday goal agents might have. While the RSA model as implemented here does not accommodate an agent’s goals, it can be extended to do so via projection functions, as has been done for capturing figurative language use <e.g.,¿kao2014 or question-answer behavior Hawkins \BOthers. (\APACyear2015). This should be explored further in future research.

One factor that has been repeatedly discussed in the literature and that we have not taken up here is the incrementality of language production, both at the conceptual level of content or property selection and at the level of linguistic realization. For instance, according to \citeAPechmann1989, incrementality is to blame for redundancy: speakers retrieve and subsequently produce words as soon as they can. Because color modifiers are easier to retrieve than size modifiers, speakers produce them regardless of whether or not they are redundant. The problem with this account is that it predicts that color adjectives should occur before size adjectives, thereby inverting the well-documented adjective ordering preferences for English Bloomfield (\APACyear1933); Sproat \BBA Shih (\APACyear1991); Scontras \BOthers. (\APACyear2017). Pechmann does observe some, but not many, instances of this. An interesting test case for the incrementality hypothesis are cases where adjective ordering preferences are weak. \citeAfukumura2018 reports one such case in which speakers prefer to order more discriminative and more available properties before less discriminative and less available ones, highlighting incrementality as an important factor affecting the choice of referring expression. On the other hand, it is unclear how incrementality – whether in linguistic realization or in content selection – could account for the systematic increase in color redundancy with increasing scene variation and decreasing color typicality, unless one makes the auxiliary assumption that the more contextually discriminative or salient color is, the more available (i.e., easily retrievable) the modifier is. Indeed, \citeAClark2004 emphasize the importance of salience against the common ground in speakers’ decisions about which of an object’s properties to include in a referring expression.

There are other ways incrementality could play a role in modifier choice. For example, mentioning the color adjective may buy the speaker time when the noun is hard to retrieve. This predicts that in languages with post-nominal adjectives, where this delay strategy cannot be used for noun planning, redundant color mention should be less frequent; indeed, this is what \citeArubiofernandez2016 shows for Spanish.353535Rubio-Fernández herself argues for an efficiency-based account of the Spanish/English asymmetry in color overmodification: “color adjectives are a more efficient cue in pre-nominal position than in post-nominal position because in the latter case the hearer’s visual search is initially guided by the noun” (Rubio-Fernandez, \APACyear2016, p. 9). Cs-RSA as presented here does not predict positional asymmetries in modifier use. However, see \citeAcohn2018 for an incremental version of RSA that captures the asymmetry reported by \citeArubiofernandez2016 as the result of incremental rather than global reasoning about utterance informativeness. In sum, the incrementality of language production clearly affects the choice of referring expression; the ways in which considerations of incrementality should be incorporated in RSA are to be explored further <see¿[for a step in this direction]cohn2018.

6.8 Extensions to other language production phenomena

In this paper we focused on providing a computationally explicit account of definite modified and nominal referring expressions in reference games, focusing on the use of prenominal size and color adjectives as well as on the taxonomic level of noun reference. The cs-RSA model can be straightforwardly extended to different nominal domains and different properties. For instance, the literature has also explored ‘overinformative’ referring expressions that include material (wooden, plastic), other dimensional (long, short), and other physical (spotted, striped) adjectives.

However, beyond the relatively limited linguistic forms we have explored here, future research should also investigate the very intriguing potential for this approach to be extended to any language production phenomenon that involves a choice in which aspects of an event or entity to mention (content selection) and how to realize that content linguistically, including in the domain of reference (pronouns, names, definite descriptions with post-nominal modification) and event descriptions. For example, in investigations of optional instrument mentions, \citeAbrown1987 showed that atypical instruments are more likely to be mentioned than typical ones – if a stabbing occurred with an icepick, speakers prefer The man was stabbed with an ice pick rather than The man was stabbed. If instead a stabbing occurred with a knife, The man was stabbed is preferred over The man was stabbed with a knife). This is very much parallel to the case of atypical color mention.

Similarly, the approach outlined here might be extended to the case of non-restrictive modifiers. In these cases, the modifier is not used to distinguish a target referent from possible competitors. Instead, the speaker may intend to communicate an aspect of an already contextually established referent, as in Sit by the newly painted table, where the speaker is warning the listener not to put their elbows on the table Dale \BBA Reiter (\APACyear1995); or Forrest looks at the massive crowd, where the speaker is commenting on the extraordinary size of the crowd Hahn \BOthers. (\APACyear2018). In these cases, the table and the crowd are contextually given referents; the modifiers newly painted and massive are used to highlight informative aspects of these referents. The approach proposed in this paper could be extended to these cases by allowing the speaker to be informative with respect to goals other than getting the listener to infer the intended referent. For instance, the speaker may want to be informative with respect to the goal of highlighting task-relevant properties of contextually given referents, as in the newly painted table case.

More generally, the approach should extend to any phenomenon that affords a choice between a more or less specific utterance. Whenever the more specific utterance adds relevant information compared to the less specific one, it should be produced. This is related to surprisal based theories of production like Uniform Information Density <UID,¿jaeger2006, levy2007, frank2008, jaeger2010, where speakers have been found to be more likely to omit linguistic signal if the underlying meaning or syntactic structure is highly predictable to the listener. Importantly, UID diverges from our account in that it is an account of the choice between meaning-equivalent alternative utterances and includes no pragmatic reasoning component.

6.9 Conclusion

In conclusion, we have provided an account of redundant referring expressions that challenges the traditional notion of ‘overinformativeness’, unifies multiple language production literatures, and has the potential for many further extensions. We take this work to provide evidence that, rather than being wastefully overinformative, speakers are usefully redundant.

Appendix A Effects of semantic value on utterance probabilities

Here we visualize the effect of different adjective types’ semantic value on the probability of producing the insufficient color-only utterance (blue pin), the sufficient size-only utterance (small pin), or the redundant color-and-size utterance (small blue pin) to refer to the target in context Figure 1(a) under varying $\beta_{i}$ values, in Figure 20. This constitutes a generalization of Figure 4, which is duplicated in row 6 ($\beta_{i}=30$).

Appendix B Pre-experiment quiz

Before continuing to the main experiment, each participant was required to correctly respond “True” or “False” to the following statements. Correct answers are given in parentheses after the statement.

• •

The speaker can click on an object. (False)

• •

The listener wants to click on the object that the speaker is telling them about. (True)

• •

The target is the object which has the red circle around it. (False)

• •

Only the speaker can send messages. (False)

• •

There are a total of 72 rounds. (True)

• •

The locations of the three objects are the same for the speaker and the listener. (False)

Appendix C Exp. 1 items

Table 9 lists the 36 object types from Exp. 1 and the colors they appeared in:

Appendix D Data pre-processing and exclusions

D.1 Exp. 1

Median completion time was 10 minutes. One participant was excluded because their native language was not English (but they participated in the listener role, so their exclusion was of no consequence to the analysis). 83% of participants thought their partner was human. Participants were not excluded if they didn’t believe their partner was human.

We collected data from 2177 critical trials. Because we did not restrict participants’ utterances in any way, they produced many different kinds of referring expressions. Testing the model’s predictions required, for each trial, classifying the produced utterance as an instance of a color-only mention, a size-only mention, or a color-and-size mention (or excluding the trial if no classification was possible). To this end we conducted the following semi-automatic data pre-processing.

An R script first automatically checked whether the speaker’s utterance contained a precoded color (i.e. black, blue, brown, gold, green, orange, pink, purple, red, silver, violet, white, yellow) or size (i.e. big, bigger, biggest, huge, large, larger, largest, little, small, smaller, smallest, tiny) term. In this way, 95.7 % of cases were classified as mentioning size and/or color. However, this did not capture that sometimes, participants produced meaning-equivalent modifications of color/size terms for instance by adding suffixes (e.g., bluish), using abbreviations (e.g., lg for large or purp for purple), or using non-precoded color labels (e.g., lime or lavender). Expressions containing a typo (e.g., pruple instead of purple) could also not be classified automatically. In the next step, one of the authors (CG) therefore manually checked the automatic coding to include these kinds of modifications in the analysis. This covered another 1.9% of trials. Most of the time, participants converged on a convention of producing only the target’s size and/or color, e.g., purple or big blue, but not a determiner (e.g., the) or the noun corresponding to the object’s type (e.g., comb). Determiners were omitted in 88.6 % of cases and nouns were omitted in 71.6 % of cases. We did not analyze this any further.

There were 50 cases (2.3%) in which the speaker made reference to the distinguishing dimension in an abstract way, e.g. different color, unique one, ripest, very girly, or guitar closest to viewer. While interesting as utterance choices,363636Certain participants seemed to have deliberately used this as a strategy even though simply mentioning the distinguishing property would have been shorter in most cases. In all, only 12 participants produced these kinds of utterances: one 18 times, one 8 times, one 6 times, two 3 times, one 2 times, and the remaining six only once each. these cases were excluded from the analysis. There were 3 cases that were nonsensical, e.g. bigger off a shade, which were also excluded. In 6 cases only the insufficient dimension was mentioned – these were excluded from the analysis reported in the next section, where we are only interested in minimal or redundant utterances, not underinformative ones, but were included in the Bayesian data analysis reported in Section 3.3. Finally, we excluded six trials where the speaker did not produce any utterances, and 33 trials on which the listener selected the wrong referent, leading to the elimination of 1.5% of trials. After the exclusion, 2076 cases classified as one of color, size, or color-and-size entered the analysis.

D.2 Exp. 2

Median completion time was 7 minutes. All participants self-reported English as their native language. 93% of participants thought their partner was human. Participants were not excluded if they believed their partner was human.

Two participant-pairs were excluded because they did not finish the experiment and therefore could not receive payment. Trials on which the speaker did not produce any utterances were also excluded, resulting in the exclusion of two additional participant-pairs. Finally, there were 10 speakers who consistently used roundabout descriptions instead of direct referring expressions (e.g., monkeys love… to refer to banana).373737It is unclear whether these participants misunderstood task instructions, or were simply being playful. This is interestingly different from Exp. 1, where participants did not produce such descriptions, presumably because the object type was identical and therefore there were no differences between objects except for color and size. We speculate that on average the functional differences between objects that differ in type are greater than between those that differ in color or size, with the former consequently being more inspiring for the generation of creative referring expressions. These pairs were also excluded, since such indirect expressions do not inform our questions about modifier production.

We analyzed data from 1974 trials. Just as in Exp. 1, participants communicated freely, which led to a vast amount of different referring expressions. To test the model’s predictions, the utterance produced for each trial was to be classified as belonging to one of the following categories: type-only (“banana”), color-and-type (“yellow banana”), and color-only (“yellow”) utterances. Referring expressions that included superordinate categories (“yellow fruit”), descriptions (“has green stem”), color-circumscriptions (“funky carrot”), and negations (“yellow but not banana”) were regarded as other and excluded. To this end we conducted the following semi-automatic data pre-processing.

The referring expressions were analyzed similarly to Exp. 1. First, 32 trials (1.6%) were excluded because the listener selected the wrong referent. 109 trials (5.6%) were excluded because the referring expressions included one of the exceptional cases described above (e.g., using negations). An R script then automatically checked the remaining 1833 utterances for whether they contained a precoded color term (i.e. green, purple, white, black, brown, yellow, orange, blue, pink, red, grey) or type (i.e. apple, banana, carrot, tomato, pear, pepper, avocado). This way, 96.5% of the remaining cases were classified as mentioning type and/or color.

However, this did not capture that sometimes, participants produced meaning-equivalent modifications of color/type terms for instance by adding suffixes (e.g., pinkish), using abbreviations (e.g., yel for yellow), or using non-precoded color and type labels (e.g., lavender or jalapeno). In addition, expressions that contained a typo (e.g., blakc instead of black) could also not be classified automatically. One of the authors (EK) therefore manually hand-coded these cases. There were 6 cases (0.3%) that could not be categorized and were excluded. Overall, 1827 utterances were classified as one of color, type, or color-and-type entered the analysis.

D.3 Exp. 3

We collected 2193 referring expressions. To determine the level of reference for each trial, we followed the following procedure. First, speakers’ and listeners’ messages were parsed automatically; the referring expression used by the speaker was extracted for each trial and checked for whether it contained the current target’s correct sub(ordinate), basic, or super(ordinate) level term using a simple grep search. In this way, 71.4% of trials were labelled as mentioning a pre-coded level of reference. In the next step, remaining utterances were checked manually by one of the authors (CG) to determine whether they contained a correct level of reference term which was not detected by the grep search due to typos or grammatical modification of the expression. In this way, meaning-equivalent alternatives such as doggie for dog, or reduced forms such as gummi, gummies and bears for gummy bears were counted as containing the corresponding level of reference term. This covered another 15.0% of trials. 41 trials on which the listener selected the wrong referent were excluded, leading to the elimination of 2.1% of trials. Six trials were excluded because the speaker did not produce any utterances. Additionally, a total of 12.5% of correct trials were excluded because the utterance consisted only of an attribute of the superclass (the living thing for animal), of the basic level (can fly for bird), of the subcategory (barks for dog) or of the particular instance (the thing facing left) rather than a category noun. These kinds of attributes were also mentioned in addition to the noun on trials which were included in the analysis for 8.9% of sub level terms, 18.9% of basic level terms, and 60.9% of super level terms. On 1.2% of trials two different levels of reference were mentioned; in this case the more specific level of reference was counted as being mentioned in this trial. After all exclusion and pre-processing, 1872 cases classified as one of sub, basic, or super entered into the analysis.

Appendix E Typicality effects in Exp. 1

To assess whether we replicate the color typicality effects previously reported in the literature Sedivy (\APACyear2003); Westerbeek \BOthers. (\APACyear2015); Rubio-Fernandez (\APACyear2016), we elicited color typicality norms for each of the items in Exp. 1 and then included typicality as an additional predictor of redundant adjective use in the regression analysis reported in Section 3.2.

E.1 Methods

E.1.1 Participants

We recruited 60 participants over Amazon’s Mechanical Turk who were each paid $0.25 for their participation.

E.1.2 Procedure and materials

On each trial, participants saw one of the big versions of the items used in Exp. 1 and were asked to answer the question “How typical is this for an X?” on a continuous slider with endpoints labeled “very atypical” to “very typical.” X was a referring expression consisting of either only the correct noun (e.g., stapler) or the noun modified by the correct color (e.g., red stapler). Figure 21 shows an example of a modified trial.

Each participant saw each of the 36 objects once. An object was randomly displayed in one of the two colors it occurred with in Exp. 1 and was randomly displayed with either the correct modified utterance or the correct unmodified utterance, in order to obtain roughly equal numbers of object-utterance combinations.

Importantly, we only elicited typicality norms for unmodified utterances and utterances with color modifiers, but not utterances with size modifiers. This was because it is impossible to obtain size typicality norms for objects presented in isolation, due to the inherently relational nature of size adjectives. Consequently, we only test for the effect of typicality on size-sufficient trials, i.e. when color is redundant.

E.2 Results and discussion

We coded the slider endpoints as 0 (“very atypical”) and 1 (“very typical”), essentially treating each response as a typicality value between 0 and 1. For each combination of object, color, and utterance (modified/unmodified), we computed that item’s mean. Mean typicalities were generally lower for unmodified than for modified utterances: mean typicality for unmodified utterances was .67 (sd=.17, mode=.76) and for modified utterances .75 (sd=.12, mode=.81). This can also be seen on the left in Figure 22. Note that, as expected given how the stimuli were constructed, typicality was generally skewed towards the high end, even for unmodified utterances. This means that there was not much variation in the difference in typicality between modified and unmodified utterances. We will refer to this difference as typicality gain, reflecting the overall gain in typicality via color modification over the unmodified baseline. As can be seen on the right in Figure 22, in most cases typicality gain was close to zero.

This makes the typicality analysis difficult: if typicality gain is close to zero for most cases (and, taking into account confidence intervals, effectively zero), it is hard to evaluate the effect of typicality on redundant adjective use. In order to maximize power, we therefore conducted the analysis only on those items for which for at least one color the confidence intervals for the modified and unmodified utterances did not overlap. There were only four such cases: (pink) golfball, (pink) wedding cake, (green) chair, and (red) stapler, for a total of 231 data points.

Predictions differ for size-sufficient and color-sufficient trials. Given the typicality effects reported in the literature and the predictions of cs-RSA, we expect greater redundant color use on size-sufficient trials with increasing typicality gain. The predictions for redundant size use on color-sufficient trials are unclear from the previous literature. Cs-RSA, however, predicts greater redundant size use with decreasing typicality gain: small color typicality gains reflect the relatively low out-of-context utility of color. In these cases, it may be useful to redundantly use a size modifier even if that modifier is noisy. If borne out, these predictions should surface in an interaction between sufficient property and typicality gain. Visual inspection of the empirical proportions of redundant adjective use in Figure 23 suggests that this pattern is indeed borne out.

In order to investigate the effect of typicality gain on redundant adjective use, we conducted a mixed effects logistic regression analysis predicting redundant over minimal adjective use from fixed effects of scene variation, sufficient dimension, the interaction of scene variation and sufficient property, and the interaction of typicality gain and sufficient property. This is the same model as reported in Section 3.2, with the only difference that the interaction between sufficient property and typicality gain was added. All predictors were centered before entering the analysis. The model contained the most sophisticated random effects structure that allowed it to converge: by-participant and by-item (where item was a color-object combination) random intercepts.

The model summary is shown in Table 10. We replicate the effects of sufficient property and scene variation observed earlier on this smaller dataset. Crucially, we observe a significant interaction between sufficient property and typicality gain.383838Conducting the same analysis on the entire dataset (i.e., using all of the noisy typicality estimates, replicated the scene variation and sufficient property effects. The interaction of typicality gain and sufficient property went in the same direction numerically, but failed to reach significance ($\beta=1.52$, $SE=1.45$, $p<.29$). Simple effects analysis reveals that this interaction is due to a positive effect of typicality gain on redundant adjective use in the size-sufficient condition ($\beta=4.47$, $SE=1.65$, $p<.007$) but a negative effect of typicality gain on redundant adjective use in the color-sufficient condition ($\beta=-5.77$, $SE=2.49$, $p<.03$).

An important point is of note: the typicality elicitation procedure we employed here is somewhat different from that employed by \citeAWesterbeek2015, who asked their participants “How typical is this color for this object?” We did this for conceptual reasons: the values that go into the semantics of the RSA model are most easily conceptualized as the typicality of an object as an instance of an utterance. While the typicality of a feature for an object type no doubt plays into how good of an instance of the utterance the object is, deriving our typicalities from the statistical properties of the subjective distributions of features over objects is beyond the scope of this paper. However, in a separate experiment we did ask participants the Westerbeek question. The correlation between mean typicality ratings from the Westerbeek version and the unmodified “How typical is this for X” version was .75. The correlation between the Westerbeek version and the modified version was .64. The correlation between the Westerbeek version and typicality gain was -.52.

For comparison, including typicality means obtained via the Westerbeek question as a predictor instead of typicality gain on the four high-powered items replicated the significant interaction between typicality and sufficient property ($\beta=-6.77$, $SE=1.88$, $p<.0003$). Simple effects analysis revealed that the interaction is again due to a difference in slope in the two sufficient property conditions: in the size-sufficient condition, color is less likely to be mentioned with increasing color typicality ($\beta=-3.66$, $SE=1.18$, $p<.002$), whereas in the color-sufficient condition, size is more likely to be mentioned with increasing color typicality ($\beta=3.09$, $SE=1.45$, $p<.04$).393939Again, conducting this analysis on the entire dataset yielded only a marginal interaction of sufficient property and color typicality in the right direction ($\beta=-1.10$, $SE=.64$, $p<.09$).

We thus overall find moderate evidence for typicality effects in our dataset. Typicality effects are strong for those items that clearly display typicality differences between the modified and unmodified utterance, but much weaker for the remaining items. That the evidence for typicality effects is relatively scarce is no surprise: the stimuli were specifically designed to minimize effects of typicality. However, the fact that both ways of quantifying typicality predicted redundant adjective use in the expected direction suggests that with more power or with stimuli that exhibit greater typicality variation, these effects may show up more clearly.

Appendix F Experiment 3 items

Table 11 lists all items used in Exp. 3 and the mean empirical utterance lengths that participants produced to refer to them:

Appendix G Typicality norms for Experiment 3

Analogous to the color typicality norms elicited for utterances in Exps. 1-2, we elicited typicality norms for utterances in Exp. 3. The elicited typicalities were used in the mixed effects analyses and Bayesian Data Analysis reported in Section 5.

G.0.1 Methods

Participants

We recruited 240 participants over Amazon’s Mechanical Turk who were each paid $0.50 for their participation.

Procedure and materials

On each trial, participants saw one of the images used in Exp. 3 and were asked to answer the question “How typical is this for an X?” on a continuous slider with endpoints labeled “very atypical” to “very typical.” X was a nominal referring expression. We did not test all utterance-object combinations, which would have led to an explosion of conditions. Instead, we tested each target object with its three utterances (e.g., the dalamtian was paired with dalmatian, dog, and animal; the pug was paired with pug, dog, and animal, etc.). That yielded a total of 108 combinations – four targets in nine domains with three utterances each. We further tested each distractor item that shared the target’s superordinate category (dist-samesuper, e.g., elephants share the superordinate category animal with dogs) on both the basic level and the superordinate level term (e.g., dog for elephant and animal for elephant), for a total of 469 combinations. Finally, we also tested each distractor of a different superordinate category than the target on the target’s superordinate level term (dist-diffsuper, e.g., animal for rose). This yielded another 168 combinations. Overall, we obtained typicality norms for 745 object-utterance combinations. All other object-utterance combinations were assumed to have typicality 0. Each participant rated 45 items: 7 targets, 10 dist-diffsuper, and 28 dist-samesuper cases. These were randomly sampled from the overall pool of items in each category.

G.0.2 Results and discussion

Each combination was rated at least 5 times and at most 27 times. We coded the slider endpoints as 0 (“very atypical”) and 1 (“very typical”). In order to evaluate the model, we used each object-utterance combination’s typicality mean as input.

Typicality ratings by item type (target, dist-samesuper, dist-diffsuper) and utterance type (sub, basic, super) are visualized in Figure 24. As expected, typicality was close to 0 for distractor items with a different superordinate category as the target, and for subordinate/basic level terms used with distractors of the same superordinate category. However, even for these cases, there was some variation.

For targets, typicality of the object for the utterance decreased with increasing reference level, mirroring the typicality ratings obtained for Exp. 1 – a particular object is a better instance of the more specific term than of the more general term for that object.