Intrinsic Robotic Introspection: Learning Internal States From Neuron Activations

TL;DR

This paper introduces an introspective framework that learns internal states from neural network activations to enhance robotic control, leading to faster learning and improved performance in actor-critic models.

Contribution

It proposes a novel method to extract internal states from neural activations using a Variational Autoencoder, improving robotic learning efficiency.

Findings

Internal states reduce training episodes by about 1300

Faster convergence to high success rates in robotic tasks

Enhanced actor-critic performance through learned internal states

Abstract

We present an introspective framework inspired by the process of how humans perform introspection. Our working assumption is that neural network activations encode information, and building internal states from these activations can improve the performance of an actor-critic model. We perform experiments where we first train a Variational Autoencoder model to reconstruct the activations of a feature extraction network and use the latent space to improve the performance of an actor-critic when deciding which low-level robotic behaviour to execute. We show that internal states reduce the number of episodes needed by about 1300 episodes while training an actor-critic, denoting faster convergence to get a high success value while completing a robotic task.