Efficient Online Learning with Predictive Coding Networks: Exploiting Temporal Correlations

TL;DR

This paper introduces PCN-TA, a biologically plausible online learning algorithm that leverages temporal correlations to reduce computational costs in robotic perception tasks, facilitating real-time adaptation on resource-limited hardware.

Contribution

The paper proposes PCN-TA, a novel predictive coding network that exploits temporal correlations to significantly lower inference steps and weight updates, enhancing efficiency for edge robotic applications.

Findings

Achieves 10% fewer weight updates than backpropagation.

Requires 50% fewer inference steps than baseline predictive coding networks.

Demonstrates effective online learning on the COIL-20 robotic perception dataset.

Abstract

Robotic systems operating at the edge require efficient online learning algorithms that can continuously adapt to changing environments while processing streaming sensory data. Traditional backpropagation, while effective, conflicts with biological plausibility principles and may be suboptimal for continuous adaptation scenarios. The Predictive Coding (PC) framework offers a biologically plausible alternative with local, Hebbian-like update rules, making it suitable for neuromorphic hardware implementation. However, PC's main limitation is its computational overhead due to multiple inference iterations during training. We present Predictive Coding Network with Temporal Amortization (PCN-TA), which preserves latent states across temporal frames. By leveraging temporal correlations, PCN-TA significantly reduces computational demands while maintaining learning performance. Our experiments on the COIL-20 robotic perception dataset demonstrate that PCN-TA achieves 10% fewer weight updates compared to backpropagation and requires 50% fewer inference steps than baseline PC networks. These efficiency gains directly translate to reduced computational overhead for moving another step toward edge deployment and real-time adaptation support in resource-constrained robotic systems. The biologically-inspired nature of our approach also makes it a promising candidate for future neuromorphic hardware implementations, enabling efficient online learning at the edge.