Szloca: towards a framework for full 3D tracking through a single camera in context of interactive arts

TL;DR

This paper introduces Szloca, a framework that enables full 3D tracking of objects and humans using only a single RGB camera, advancing interactive arts and large-area applications.

Contribution

It presents an original method to derive 3D positional data from monocular images without complex training, incorporating depth estimation into 2D video inputs.

Findings

Achieves 3D tracking with a single camera in real-time

Provides a novel approach to depth estimation from 2D images

Enhances interactive arts applications with full 3D data

Abstract

Realtime virtual data of objects and human presence in a large area holds a valuable key in enabling many experiences and applications in various industries and with exponential rise in the technological development of artificial intelligence, computer vision has expanded the possibilities of tracking and classifying things through just video inputs, which is also surpassing the limitations of most popular and common hardware setups known traditionally to detect human pose and position, such as low field of view and limited tracking capacity. The benefits of using computer vision in application development is large as it augments traditional input sources (like video streams) and can be integrated in many environments and platforms. In the context of new media interactive arts, based on physical movements and expanding over large areas or gallaries, this research presents a novel way and a framework towards obtaining data and virtual representation of objects/people - such as three-dimensional positions, skeltons/pose and masks from a single rgb camera. Looking at the state of art through some recent developments and building on prior research in the field of computer vision, the paper also proposes an original method to obtain three dimensional position data from monocular images, the model does not rely on complex training of computer vision systems but combines prior computer vision research and adds a capacity to represent z depth, ieto represent a world position in 3 axis from a 2d input source.