Culture Affordance Atlas: Reconciling Object Diversity Through Functional Mapping
Joan Nwatu, Longju Bai, Oana Ignat, Rada Mihalcea

TL;DR
This paper introduces the Culture Affordance Atlas, a culturally grounded object categorization framework that reduces socioeconomic biases in vision-language models, enhancing their fairness and effectiveness across diverse cultural contexts.
Contribution
We propose a function-centric framework and create the Culture Affordance Atlas, significantly reducing socioeconomic performance gaps in VL models and identifying culturally essential objects often overlooked.
Findings
Function-centric labels reduce socioeconomic gaps by median 6 percentage points.
The Atlas includes 46 functions and 288 objects, improving cultural inclusivity.
Empirical analysis with CLIP shows enhanced model fairness.
Abstract
Culture shapes the objects people use and for what purposes, yet mainstream Vision-Language (VL) datasets frequently exhibit cultural biases, disproportionately favoring higher-income, Western contexts. This imbalance reduces model generalizability and perpetuates performance disparities, especially impacting lower-income and non-Western communities. To address these disparities, we propose a novel function-centric framework that categorizes objects by the functions they fulfill, across diverse cultural and economic contexts. We implement this framework by creating the Culture Affordance Atlas, a re-annotated and culturally grounded restructuring of the Dollar Street dataset spanning 46 functions and 288 objects publicly available at https://lit.eecs.umich.edu/CultureAffordance-Atlas/index.html. Through extensive empirical analyses using the CLIP model, we demonstrate that…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsEthics and Social Impacts of AI · Multimodal Machine Learning Applications · Domain Adaptation and Few-Shot Learning
