Word Familiarity and Frequency

TL;DR

This study investigates how word familiarity ratings relate to word frequency across different corpora and languages, revealing that familiarity influences frequency more in spoken language and as corpus size increases.

Contribution

It provides a comprehensive analysis of the correlation between psycholinguistic familiarity ratings and corpus-based frequency measures for English and Japanese.

Findings

Familiar words tend to be more frequent, but high frequency does not imply familiarity.

Correlation between familiarity and frequency increases with larger corpora.

Spoken language corpora show a stronger correlation with familiarity than written corpora.

Abstract

Word frequency is assumed to correlate with word familiarity, but the strength of this correlation has not been thoroughly investigated. In this paper, we report on our analysis of the correlation between a word familiarity rating list obtained through a psycholinguistic experiment and the log-frequency obtained from various corpora of different kinds and sizes (up to the terabyte scale) for English and Japanese. Major findings are threefold: First, for a given corpus, familiarity is necessary for a word to achieve high frequency, but familiar words are not necessarily frequent. Second, correlation increases with the corpus data size. Third, a corpus of spoken language correlates better than one of written language. These findings suggest that cognitive familiarity ratings are correlated to frequency, but more highly to that of spoken rather than written language.