Asynchronous Perception Machine For Efficient Test-Time-Training

TL;DR

The paper introduces Asynchronous Perception Machine (APM), an efficient architecture for test-time-training that processes image patches asynchronously, recognizing out-of-distribution images without dataset-specific pre-training, and demonstrating potential for scalable semantic clustering.

Contribution

The paper presents APM, a novel asynchronous architecture for TTT that encodes semantic awareness, learns from a single representation, and scales to large datasets, providing empirical validation of GLOM's insights.

Findings

APM can recognize out-of-distribution images without pre-training.

APM achieves competitive performance with existing TTT methods.

APM can produce semantic clusterings in a single forward pass.

Abstract

In this work, we propose Asynchronous Perception Machine (APM), a computationally-efficient architecture for test-time-training (TTT). APM can process patches of an image one at a time in any order asymmetrically and still encode semantic-awareness in the net. We demonstrate APM's ability to recognize out-of-distribution images without dataset-specific pre-training, augmentation or any-pretext task. APM offers competitive performance over existing TTT approaches. To perform TTT, APM just distills test sample's representation once. APM possesses a unique property: it can learn using just this single representation and starts predicting semantically-aware features. APM demostrates potential applications beyond test-time-training: APM can scale up to a dataset of 2D images and yield semantic-clusterings in a single forward pass. APM also provides first empirical evidence towards validating GLOM's insight, i.e. input percept is a field. Therefore, APM helps us converge towards an implementation which can do both interpolation and perception on a shared-connectionist hardware. Our code is publicly available at this link: https://rajatmodi62.github.io/apm_project_page/.