Strategies for Conceptual Change in Convolutional Neural Networks

TL;DR

This paper compares various strategies for adapting learned representations in convolutional neural networks to improve task switching efficiency and data utilization, demonstrating their superiority over training from scratch across multiple domains.

Contribution

It introduces and empirically evaluates multiple adaptation strategies for CNNs, highlighting their effectiveness over baseline methods in different data scenarios.

Findings

Adaptive strategies outperform training from scratch.

Strategies are effective across images, music, and digits.

Fewer training data needed with adaptive methods.

Abstract

A remarkable feature of human beings is their capacity for creative behaviour, referring to their ability to react to problems in ways that are novel, surprising, and useful. Transformational creativity is a form of creativity where the creative behaviour is induced by a transformation of the actor's conceptual space, that is, the representational system with which the actor interprets its environment. In this report, we focus on ways of adapting systems of learned representations as they switch from performing one task to performing another. We describe an experimental comparison of multiple strategies for adaptation of learned features, and evaluate how effectively each of these strategies realizes the adaptation, in terms of the amount of training, and in terms of their ability to cope with restricted availability of training data. We show, among other things, that across handwritten digits, natural images, and classical music, adaptive strategies are systematically more effective than a baseline method that starts learning from scratch.