Hierarchy of GANs for learning embodied self-awareness model

TL;DR

This paper introduces a hierarchical self-awareness model for embodied agents using cross-modal GANs to process high-dimensional visual data, enabling the detection of normal and abnormal behaviors in semi-autonomous vehicles.

Contribution

It proposes a novel hierarchical GAN-based self-awareness model that handles high-dimensional visual data for embodied agents, advancing beyond low-dimensional sensor approaches.

Findings

Hierarchical GANs effectively model complex behaviors.

The model detects abnormalities in semi-autonomous vehicles.

Self-supervised detection of GAN levels improves adaptability.

Abstract

In recent years several architectures have been proposed to learn embodied agents complex self-awareness models. In this paper, dynamic incremental self-awareness (SA) models are proposed that allow experiences done by an agent to be modeled in a hierarchical fashion, starting from more simple situations to more structured ones. Each situation is learned from subsets of private agent perception data as a model capable to predict normal behaviors and detect abnormalities. Hierarchical SA models have been already proposed using low dimensional sensorial inputs. In this work, a hierarchical model is introduced by means of a cross-modal Generative Adversarial Networks (GANs) processing high dimensional visual data. Different levels of the GANs are detected in a self-supervised manner using GANs discriminators decision boundaries. Real experiments on semi-autonomous ground vehicles are presented.