Array Camera Image Fusion using Physics-Aware Transformers

TL;DR

This paper introduces a physics-aware transformer model for fusing data from array cameras with diverse specifications, along with a scalable synthetic data generation method, enabling effective image synthesis across varied spectral and temporal parameters.

Contribution

The paper presents a novel physics-aware transformer architecture and a scalable synthetic data generation approach for multi-camera data fusion and image synthesis.

Findings

Effective fusion of diverse camera data demonstrated

Synthetic data generation enables training for varied camera configurations

Successful image synthesis across spectral and temporal variations

Abstract

We demonstrate a physics-aware transformer for feature-based data fusion from cameras with diverse resolution, color spaces, focal planes, focal lengths, and exposure. We also demonstrate a scalable solution for synthetic training data generation for the transformer using open-source computer graphics software. We demonstrate image synthesis on arrays with diverse spectral responses, instantaneous field of view and frame rate.