See far with TPNET: a Tile Processor and a CNN Symbiosis

TL;DR

TPNET is a novel neural network architecture inspired by human visual perception, combining a Tile Processor with CNNs to enhance 3D perception accuracy and efficiency in high-resolution stereo imaging.

Contribution

Introducing TPNET, a hybrid system that offloads image correction to a Tile Processor, enabling more accurate disparity prediction with reduced network complexity.

Findings

Disparity prediction from TPNET is twice as accurate as hand-crafted algorithms.

Tile Processor reduces input feature dimensions and provides invariant data for real-time high-res stereo perception.

TPNET enables efficient 3D perception with less dependence on end-to-end training.

Abstract

Throughout the evolution of the neural networks more specialized cells were added to the set of basic building blocks. These cells aim to improve training convergence, increase the overall performance, and reduce the number of required labels, all while preserving the expressive power of the universal network. Inspired by the partitioning of the human visual perception system between the eyes and the cerebral cortex, we present TPNET, which offloads universal and application-specific CNN from the bulk processing of the high resolution pixel data and performs the translation-variant image correction while delegating all non-linear decision making to the network. In this work, we explore application of TPNET to 3D perception with a narrow-baseline (0.0001-0.0025) quad stereo camera and prove that a trained network provides a disparity prediction from the 2D phase correlation output by the Tile Processor (TP) that is twice as accurate as the prediction from a carefully hand-crafted algorithm. The TP in turn reduces the dimensions of the input features of the network and provides instrument-invariant and translation-invariant data, making real-time high resolution stereo 3D perception feasible and easing the requirement to have a complete end-to-end network.