A transition towards virtual representations of visual scenes

TL;DR

This paper proposes a unified architecture for visual scene understanding that enhances flexibility and coherence, facilitating 3D virtual synthesis and supporting diverse applications in virtual and augmented reality.

Contribution

It introduces a novel, adaptable framework that unifies various visual scene understanding tasks for improved 3D virtual scene synthesis.

Findings

Demonstrates the architecture's effectiveness through a practical proof of concept

Enhances flexibility and coherence in visual scene understanding systems

Supports multiple application areas in virtual and augmented reality

Abstract

Visual scene understanding is a fundamental task in computer vision that aims to extract meaningful information from visual data. It traditionally involves disjoint and specialized algorithms for different tasks that are tailored for specific application scenarios. This can be cumbersome when designing complex systems that include processing of visual and semantic data extracted from visual scenes, which is even more noticeable nowadays with the influx of applications for virtual or augmented reality. When designing a system that employs automatic visual scene understanding to enable a precise and semantically coherent description of the underlying scene, which can be used to fuel a visualization component with 3D virtual synthesis, the lack of flexibility and unified frameworks become more prominent. To alleviate this issue and its inherent problems, we propose an architecture that addresses the challenges of visual scene understanding and description towards a 3D virtual synthesis that enables an adaptable, unified and coherent solution. Furthermore, we expose how our proposition can be of use into multiple application areas. Additionally, we also present a proof of concept system that employs our architecture to further prove its usability in practice.