Using Motion and Internal Supervision in Object Recognition

TL;DR

This thesis investigates how motion information and internal supervision can enhance unsupervised object recognition in dynamic scenes, focusing on adaptive detection, hand recognition, and gaze direction extraction.

Contribution

It introduces novel methods leveraging motion for adaptive object-part detection and unsupervised learning of hands and gaze direction, inspired by infant learning processes.

Findings

Motion aids adaptive object-part detection in dynamic scenes.

Unsupervised methods successfully recognize hands by appearance and context.

Gaze direction can be extracted using motion cues in unsupervised learning.

Abstract

In this thesis we address two related aspects of visual object recognition: the use of motion information, and the use of internal supervision, to help unsupervised learning. These two aspects are inter-related in the current study, since image motion is used for internal supervision, via the detection of spatiotemporal events of active-motion and the use of tracking. Most current work in object recognition deals with static images during both learning and recognition. In contrast, we are interested in a dynamic scene where visual processes, such as detecting motion events and tracking, contribute spatiotemporal information, which is useful for object attention, motion segmentation, 3-D understanding and object interactions. We explore the use of these sources of information in both learning and recognition processes. In the first part of the work, we demonstrate how motion can be used for adaptive detection of object-parts in dynamic environments, while automatically learning new object appearances and poses. In the second and main part of the study we develop methods for using specific types of visual motion to solve two difficult problems in unsupervised visual learning: learning to recognize hands by their appearance and by context, and learning to extract direction of gaze. We use our conclusions in this part to propose a model for several aspects of learning by human infants from their visual environment.