Machine learning method for light field refocusing

TL;DR

This paper presents RefNet, a machine learning approach that enables real-time light field refocusing, significantly faster and with better color prediction than traditional methods, without pre-processing.

Contribution

Introduces RefNet, a novel neural network for real-time light field refocusing that outperforms traditional methods in speed and color accuracy.

Findings

RefNet is at least 134 times faster than existing approaches.

RefNet produces superior color prediction compared to Fourier slice and shift-and-sum methods.

RefNet accurately refocuses 16 images with various parameters in real-time.

Abstract

Light field imaging introduced the capability to refocus an image after capturing. Currently there are two popular methods for refocusing, shift-and-sum and Fourier slice methods. Neither of these two methods can refocus the light field in real-time without any pre-processing. In this paper we introduce a machine learning based refocusing technique that is capable of extracting 16 refocused images with refocusing parameters of \alpha=0.125,0.250,0.375,...,2.0 in real-time. We have trained our network, which is called RefNet, in two experiments. Once using the Fourier slice method as the training -- i.e., "ground truth" -- data and another using the shift-and-sum method as the training data. We showed that in both cases, not only is the RefNet method at least 134x faster than previous approaches, but also the color prediction of RefNet is superior to both Fourier slice and shift-and-sum methods while having similar depth of field and focus distance performance.