Classification and Feature Transformation with Fuzzy Cognitive Maps

TL;DR

This paper introduces a novel Fuzzy Cognitive Map (FCM) based classifier that executes multiple iterations before output collection, demonstrating promising performance and data transformation capabilities comparable to classical methods.

Contribution

The work proposes a fully connected FCM classifier with multiple iteration steps, trained with gradient algorithms, and investigates its data transformation and classification performance.

Findings

FCM classifier achieves performance comparable to classical methods.

Multiple FCM iterations improve class separability in feature space.

Fuzzy Cognitive Maps can effectively transform data for classification tasks.

Abstract

Fuzzy Cognitive Maps (FCMs) are considered a soft computing technique combining elements of fuzzy logic and recurrent neural networks. They found multiple application in such domains as modeling of system behavior, prediction of time series, decision making and process control. Less attention, however, has been turned towards using them in pattern classification. In this work we propose an FCM based classifier with a fully connected map structure. In contrast to methods that expect reaching a steady system state during reasoning, we chose to execute a few FCM iterations (steps) before collecting output labels. Weights were learned with a gradient algorithm and logloss or cross-entropy were used as the cost function. Our primary goal was to verify, whether such design would result in a descent general purpose classifier, with performance comparable to off the shelf classical methods. As the preliminary results were promising, we investigated the hypothesis that the performance of $d$-step classifier can be attributed to a fact that in previous $d-1$ steps it transforms the feature space by grouping observations belonging to a given class, so that they became more compact and separable. To verify this hypothesis we calculated three clustering scores for the transformed feature space. We also evaluated performance of pipelines built from FCM-based data transformer followed by a classification algorithm. The standard statistical analyzes confirmed both the performance of FCM based classifier and its capability to improve data. The supporting prototype software was implemented in Python using TensorFlow library.