Dynamic Object Comprehension: A Framework For Evaluating Artificial Visual Perception

TL;DR

This paper discusses the importance of developing and evaluating visual perception systems for AR and MR, highlighting the limitations of current metrics and proposing new evaluation criteria to advance the field.

Contribution

It introduces a framework for evaluating visual perception in AR and MR, addressing the inadequacy of existing metrics and proposing new evaluation criteria.

Findings

Current evaluation metrics are insufficient for AR/MR visual perception

Proposed new criteria better assess system understanding of physical environments

Framework aims to guide future development of visual perception systems

Abstract

Augmented and Mixed Reality are emerging as likely successors to the mobile internet. However, many technical challenges remain. One of the key requirements of these systems is the ability to create a continuity between physical and virtual worlds, with the user's visual perception as the primary interface medium. Building this continuity requires the system to develop a visual understanding of the physical world. While there has been significant recent progress in computer vision and AI techniques such as image classification and object detection, success in these areas has not yet led to the visual perception required for these critical MR and AR applications. A significant issue is that current evaluation criteria are insufficient for these applications. To motivate and evaluate progress in this emerging area, there is a need for new metrics. In this paper we outline limitations of current evaluation criteria and propose new criteria.