Characterizing the visual representation of objects from the child's view

TL;DR

This study analyzes young children's visual experiences through first-person videos, revealing skewed object category exposure, high variability in exemplars, and strong superordinate category groupings despite diverse views.

Contribution

It provides a detailed characterization of children's visual input, highlighting the importance of superordinate structures for robust object category learning.

Findings

Children's visual experience is highly skewed towards a few categories.

Objects are viewed from unusual angles and in cluttered, occluded scenes.

Detected categories show stronger superordinate groupings than canonical images.

Abstract

Children acquire object category representations from their everyday experiences in the first few years of life. What do the inputs to this learning process look like? We analyzed first-person videos of young children's visual experience at home from the BabyView dataset ($N$ = 31 participants, 868 hours, ages 5--36 months), using a supervised object detection model to extract common object categories from more than 3 million frames. We found that children's object category exposure was highly skewed: a few categories (e.g., cups, chairs) dominated children's visual experiences while most categories appeared rarely, replicating previous findings from a more restricted set of contexts. Category exemplars were highly variable: children encountered objects from unusual angles, in highly cluttered scenes, and partially occluded views; many categories (especially animals) were most frequently viewed as depictions. Surprisingly, despite this variability, detected categories (e.g., giraffes, apples) showed stronger groupings within superordinate categories (e.g., animals, food) relative to groupings derived from canonical photographs of these categories. We found this same pattern when using high-dimensional embeddings from both self-supervised visual and multimodal models; this effect was also recapitulated in densely sampled data from individual children. Understanding the robustness and efficiency of visual category learning will require the development of models that can exploit strong superordinate structure and learn from non-canonical, sparse, and variable exemplars.