Koopcon: A new approach towards smarter and less complex learning

TL;DR

This paper presents a novel dataset condensation method using Autoencoders and Koopman operator theory, enabling efficient data compression while preserving essential information for machine learning tasks.

Contribution

It introduces a new dataset condensation approach that combines autoencoders, Koopman theory, and optimal transport to create compact, informative data representations.

Findings

Classifiers trained on condensed data perform comparably to those trained on original data.

The method reduces computational resources needed for large datasets.

Effective data condensation is achieved through a two-stage process involving synthesis and evaluation.

Abstract

In the era of big data, the sheer volume and complexity of datasets pose significant challenges in machine learning, particularly in image processing tasks. This paper introduces an innovative Autoencoder-based Dataset Condensation Model backed by Koopman operator theory that effectively packs large datasets into compact, information-rich representations. Inspired by the predictive coding mechanisms of the human brain, our model leverages a novel approach to encode and reconstruct data, maintaining essential features and label distributions. The condensation process utilizes an autoencoder neural network architecture, coupled with Optimal Transport theory and Wasserstein distance, to minimize the distributional discrepancies between the original and synthesized datasets. We present a two-stage implementation strategy: first, condensing the large dataset into a smaller synthesized subset; second, evaluating the synthesized data by training a classifier and comparing its performance with a classifier trained on an equivalent subset of the original data. Our experimental results demonstrate that the classifiers trained on condensed data exhibit comparable performance to those trained on the original datasets, thus affirming the efficacy of our condensation model. This work not only contributes to the reduction of computational resources but also paves the way for efficient data handling in constrained environments, marking a significant step forward in data-efficient machine learning.