A topological solution to object segmentation and tracking

TL;DR

This paper introduces a non-learning, topological method for object segmentation and tracking based on light ray structures, effectively handling appearance changes in cluttered environments.

Contribution

It presents a novel topological approach that uses light ray reflections to segment and track objects without any learning, addressing a key challenge in computer vision.

Findings

Successfully segments objects in synthetic videos with severe appearance changes

Tracks object identities invariantly over time in cluttered scenes

Operates without any learning or training data

Abstract

The world is composed of objects, the ground, and the sky. Visual perception of objects requires solving two fundamental challenges: segmenting visual input into discrete units, and tracking identities of these units despite appearance changes due to object deformation, changing perspective, and dynamic occlusion. Current computer vision approaches to segmentation and tracking that approach human performance all require learning, raising the question: can objects be segmented and tracked without learning? Here, we show that the mathematical structure of light rays reflected from environment surfaces yields a natural representation of persistent surfaces, and this surface representation provides a solution to both the segmentation and tracking problems. We describe how to generate this surface representation from continuous visual input, and demonstrate that our approach can segment and invariantly track objects in cluttered synthetic video despite severe appearance changes, without requiring learning.