i-FlatCam: A 253 FPS, 91.49 $\mu$J/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR

TL;DR

i-FlatCam is an innovative ultra-compact lensless camera system designed for real-time eye tracking in VR/AR, achieving high frame rates and efficiency with novel computational and compression techniques.

Contribution

It introduces the first lensless eye tracking pipeline with a compact design, combining a unified compression scheme and intra-channel reuse for enhanced performance.

Findings

Achieves 253 FPS for eye tracking

Attains 3.16° accuracy in eye tracking

Consumes 91.49 μJ per frame

Abstract

We present a first-of-its-kind ultra-compact intelligent camera system, dubbed i-FlatCam, including a lensless camera with a computational (Comp.) chip. It highlights (1) a predict-then-focus eye tracking pipeline for boosted efficiency without compromising the accuracy, (2) a unified compression scheme for single-chip processing and improved frame rate per second (FPS), and (3) dedicated intra-channel reuse design for depth-wise convolutional layers (DW-CONV) to increase utilization. i-FlatCam demonstrates the first eye tracking pipeline with a lensless camera and achieves 3.16 degrees of accuracy, 253 FPS, 91.49 $\mu$J/Frame, and 6.7mm x 8.9mm x 1.2mm camera form factor, paving the way for next-generation Augmented Reality (AR) and Virtual Reality (VR) devices.