Perception Learning: A Formal Separation of Sensory Representation Learning from Decision Learning

TL;DR

Perception Learning (PeL) is a new framework that separately optimizes sensory representations independently of decision-making, focusing on label-free perceptual qualities and providing metrics for evaluation.

Contribution

We formalize the separation of perception and decision, introduce PeL for task-agnostic sensory learning, and develop metrics to evaluate perceptual quality independently of tasks.

Findings

PeL optimizes sensory interfaces using task-agnostic signals.

PeL updates preserve perceptual invariants orthogonal to task risk.

A suite of metrics effectively certifies perceptual quality.

Abstract

We introduce Perception Learning (PeL), a paradigm that optimizes an agent's sensory interface $f_\phi:\mathcal{X}\to\mathcal{Z}$ using task-agnostic signals, decoupled from downstream decision learning $g_\theta:\mathcal{Z}\to\mathcal{Y}$. PeL directly targets label-free perceptual properties, such as stability to nuisances, informativeness without collapse, and controlled geometry, assessed via objective representation-invariant metrics. We formalize the separation of perception and decision, define perceptual properties independent of objectives or reparameterizations, and prove that PeL updates preserving sufficient invariants are orthogonal to Bayes task-risk gradients. Additionally, we provide a suite of task-agnostic evaluation metrics to certify perceptual quality.