Deep Learning Stereo Vision at the edge

TL;DR

This paper introduces a hybrid deep learning and classical stereo vision system optimized for low-power embedded devices, achieving high accuracy and robustness without structured light, suitable for real-time high-definition video applications.

Contribution

The paper presents a novel System on Chip stereo vision solution combining deep learning and classical methods, optimized for embedded, power-constrained environments.

Findings

Ranks as the best System on Chip solution in Middlebury challenge

Achieves robustness to noise surpassing previous solutions

Supports real-time high-definition video processing

Abstract

We present an overview of the methodology used to build a new stereo vision solution that is suitable for System on Chip. This new solution was developed to bring computer vision capability to embedded devices that live in a power constrained environment. The solution is constructured as a hybrid between classical Stereo Vision techniques and deep learning approaches. The stereoscopic module is composed of two separate modules: one that accelerates the neural network we trained and one that accelerates the front-end part. The system is completely passive and does not require any structured light to obtain very compelling accuracy. With respect to the previous Stereo Vision solutions offered by the industries we offer a major improvement is robustness to noise. This is mainly possible due to the deep learning part of the chosen architecture. We submitted our result to Middlebury dataset challenge. It currently ranks as the best System on Chip solution. The system has been developed for low latency applications which require better than real time performance on high definition videos.