MetaCOG: A Hierarchical Probabilistic Model for Learning Meta-Cognitive Visual Representations

TL;DR

MetaCOG is a hierarchical probabilistic model that monitors neural object detectors, learns their performance, and improves accuracy by detecting and correcting errors without ground-truth labels.

Contribution

It introduces a novel meta-cognitive framework that jointly infers scene structure and detector reliability using Bayesian inference.

Findings

MetaCOG accurately recovers detector performance parameters.

It improves overall system accuracy when combined with neural object detectors.

MetaCOG is robust to varying levels of detection errors.

Abstract

Humans have the capacity to question what we see and to recognize when our vision is unreliable (e.g., when we realize that we are experiencing a visual illusion). Inspired by this capacity, we present MetaCOG: a hierarchical probabilistic model that can be attached to a neural object detector to monitor its outputs and determine their reliability. MetaCOG achieves this by learning a probabilistic model of the object detector's performance via Bayesian inference -- i.e., a meta-cognitive representation of the network's propensity to hallucinate or miss different object categories. Given a set of video frames processed by an object detector, MetaCOG performs joint inference over the underlying 3D scene and the detector's performance, grounding inference on a basic assumption of object permanence. Paired with three neural object detectors, we show that MetaCOG accurately recovers each detector's performance parameters and improves the overall system's accuracy. We additionally show that MetaCOG is robust to varying levels of error in object detector outputs, showing proof-of-concept for a novel approach to the problem of detecting and correcting errors in vision systems when ground-truth is not available.