Visual Knowledge Tracing
Neehar Kondapaneni, Pietro Perona, Oisin Mac Aodha

TL;DR
This paper introduces a new approach to track and predict how humans learn to classify complex visual tasks by modeling their evolving visual features and decision functions, validated on real datasets.
Contribution
It proposes a novel task of visual knowledge tracing and develops recurrent models to predict human classification behavior in challenging visual learning scenarios.
Findings
Recurrent models effectively predict human classification behavior.
Models perform well on medical image and species identification tasks.
New datasets collected for evaluating visual knowledge tracing methods.
Abstract
Each year, thousands of people learn new visual categorization tasks -- radiologists learn to recognize tumors, birdwatchers learn to distinguish similar species, and crowd workers learn how to annotate valuable data for applications like autonomous driving. As humans learn, their brain updates the visual features it extracts and attend to, which ultimately informs their final classification decisions. In this work, we propose a novel task of tracing the evolving classification behavior of human learners as they engage in challenging visual classification tasks. We propose models that jointly extract the visual features used by learners as well as predicting the classification functions they utilize. We collect three challenging new datasets from real human learners in order to evaluate the performance of different visual knowledge tracing methods. Our results show that our recurrent…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDigital Imaging for Blood Diseases · Domain Adaptation and Few-Shot Learning · COVID-19 diagnosis using AI
