Semiotics Networks Representing Perceptual Inference

TL;DR

This paper introduces a semiotic network model for perceptual inference that enhances object perception and communication simulation, demonstrated through an improved image classifier on MNIST data.

Contribution

The paper presents a novel semiotic network framework integrating encoding and decoding components to model perceptual inference and communication, with an application to visual classification.

Findings

Perceptualized network outperforms baseline on MNIST.

Efficient image representations improve classification.

Model applicable beyond human perception systems.

Abstract

Every day, humans perceive objects and communicate these perceptions through various channels. In this paper, we present a computational model designed to track and simulate the perception of objects, as well as their representations as conveyed in communication. We delineate two fundamental components of our internal representation, termed "observed" and "seen", which we correlate with established concepts in computer vision, namely encoding and decoding. These components are integrated into semiotic networks, which simulate perceptual inference of object perception and human communication. Our model of object perception by a person allows us to define object perception by {\em a network}. We demonstrate this with an example of an image baseline classifier by constructing a new network that includes the baseline classifier and an additional layer. This layer produces the images "perceived" by the entire network, transforming it into a perceptualized image classifier. This facilitates visualization of the acquired network. Within our network, the image representations become more efficient for classification tasks when they are assembled and randomized. In our experiments, the perceptualized network outperformed the baseline classifier on MNIST training databases consisting of a restricted number of images. Our model is not limited to persons and can be applied to any system featuring a loop involving the processing from "internal" to "external" representations.