PixRO: Pixel-Distributed Rotational Odometry with Gaussian Belief Propagation

TL;DR

This paper introduces a novel pixel-level rotational odometry algorithm using Gaussian Belief Propagation, aiming to perform complex visual processing directly in-pixel to improve efficiency in camera motion estimation.

Contribution

It proposes a decentralized, probabilistic approach for pixel-level rotation estimation leveraging GBP, enabling high-level motion cues to be synthesized directly at the sensor level.

Findings

Effective rotation estimation demonstrated on real-world datasets

Analysis of GBP's practicality for distributed pixel-level inference

Potential for reducing computational and transmission overhead

Abstract

Images are the standard input for most computer vision algorithms. However, their processing often reduces to parallelizable operations applied locally and independently to individual pixels. Yet, many of these low-level raw pixel readings only provide redundant or noisy information for specific high-level tasks, leading to inefficiencies in both energy consumption during their transmission off-sensor and computational resources in their subsequent processing. As novel sensors featuring advanced in-pixel processing capabilities emerge, we envision a paradigm shift toward performing increasingly complex visual processing directly in-pixel, reducing computational overhead downstream. We advocate for synthesizing high-level cues at the pixel level, enabling their off-sensor transmission to directly support downstream tasks more effectively than raw pixel readings. This paper conceptualizes a novel photometric rotation estimation algorithm to be distributed at pixel level, where each pixel estimates the global motion of the camera by exchanging information with other pixels to achieve global consensus. We employ a probabilistic formulation and leverage Gaussian Belief Propagation (GBP) for decentralized inference using messaging-passing. The proposed proposed technique is evaluated on real-world public datasets and we offer a in-depth analysis of the practicality of applying GBP to distributed rotation estimation at pixel level.