# LiFF: Light Field Features in Scale and Depth

## TL;DR

LiFF is a novel 4D light field feature detector and descriptor that is scale-invariant, robust to challenging visual effects, and improves 3D reconstruction accuracy while being computationally efficient.

## Contribution

Introduces LiFF, a new 4D light field feature detector and descriptor that enhances robustness and efficiency for structure from motion tasks.

## Key findings

- LiFF improves 3D reconstructions in SfM.
- LiFF runs an order of magnitude faster than previous 4D methods.
- LiFF inherently estimates depth for each feature.

## Abstract

Feature detectors and descriptors are key low-level vision tools that many higher-level tasks build on. Unfortunately these fail in the presence of challenging light transport effects including partial occlusion, low contrast, and reflective or refractive surfaces. Building on spatio-angular imaging modalities offered by emerging light field cameras, we introduce a new and computationally efficient 4D light field feature detector and descriptor: LiFF. LiFF is scale invariant and utilizes the full 4D light field to detect features that are robust to changes in perspective. This is particularly useful for structure from motion (SfM) and other tasks that match features across viewpoints of a scene. We demonstrate significantly improved 3D reconstructions via SfM when using LiFF instead of the leading 2D or 4D features, and show that LiFF runs an order of magnitude faster than the leading 4D approach. Finally, LiFF inherently estimates depth for each feature, opening a path for future research in light field-based SfM.