Bears, all bears, and some bears. Language Constraints on Language Models' Inductive Inferences

TL;DR

This study investigates whether vision-language models exhibit human-like inductive reasoning based on language constraints, finding they behave similarly to children in differentiating generic, universal, and indefinite statements.

Contribution

It demonstrates that general-purpose vision-language models replicate human-like inductive inferences influenced by language constraints, revealing organized differences based on inductive reasoning.

Findings

Models align with human behavior in differentiating statement types

Differences are organized by inductive constraints, not surface form

Models show sensitivity to all and some quantifiers in inference tasks

Abstract

Language places subtle constraints on how we make inductive inferences. Developmental evidence by Gelman et al. (2002) has shown children (4 years and older) to differentiate among generic statements ("Bears are daxable"), universally quantified NPs ("all bears are daxable") and indefinite plural NPs ("some bears are daxable") in extending novel properties to a specific member (all > generics > some), suggesting that they represent these types of propositions differently. We test if these subtle differences arise in general purpose statistical learners like Vision Language Models, by replicating the original experiment. On tasking them through a series of precondition tests (robust identification of categories in images and sensitivities to all and some), followed by the original experiment, we find behavioral alignment between models and humans. Post-hoc analyses on their representations revealed that these differences are organized based on inductive constraints and not surface-form differences.