Imitation Learning based Auto-Correction of Extrinsic Parameters for A Mixed-Reality Setup

TL;DR

This paper presents an imitation learning approach using CNNs to automatically correct extrinsic calibration errors in a mixed reality system, reducing the need for time-consuming manual calibration.

Contribution

The authors introduce a CNN-based auto-correction method for extrinsic parameters in mixed reality setups, streamlining calibration without extensive manual tuning.

Findings

CNN effectively reduces calibration errors

Auto-correction process is faster than traditional methods

System maintains accurate 3D projection alignment

Abstract

In this paper, we discuss an imitation learning based method for reducing the calibration error for a mixed reality system consisting of a vision sensor and a projector. Unlike a head mounted display, in this setup, augmented information is available to a human subject via the projection of a scene into the real world. Inherently, the camera and projector need to be calibrated as a stereo setup to project accurate information in 3D space. Previous calibration processes require multiple recording and parameter tuning steps to achieve the desired calibration, which is usually time consuming process. In order to avoid such tedious calibration, we train a CNN model to iteratively correct the extrinsic offset given a QR code and a projected pattern. We discuss the overall system setup, data collection for training, and results of the auto-correction model.