Enhancing Visual Perception in Novel Environments via Incremental Data Augmentation Based on Style Transfer

TL;DR

This paper introduces a method to improve autonomous agents' visual perception in new environments by using style transfer for incremental data augmentation, enhancing robustness against novel scenarios.

Contribution

It proposes a novel approach combining Variational Prototyping Encoder with neural style transfer for incremental data augmentation to handle unseen environments.

Findings

Models trained with augmented data outperform those trained only on original data.

Data augmentation significantly improves model robustness in novel environments.

Generative models can be effectively used for domain-specific data enhancement.

Abstract

The deployment of autonomous agents in real-world scenarios is challenged by "unknown unknowns", i.e. novel unexpected environments not encountered during training, such as degraded signs. While existing research focuses on anomaly detection and class imbalance, it often fails to address truly novel scenarios. Our approach enhances visual perception by leveraging the Variational Prototyping Encoder (VPE) to adeptly identify and handle novel inputs, then incrementally augmenting data using neural style transfer to enrich underrepresented data. By comparing models trained solely on original datasets with those trained on a combination of original and augmented datasets, we observed a notable improvement in the performance of the latter. This underscores the critical role of data augmentation in enhancing model robustness. Our findings suggest the potential benefits of incorporating generative models for domain-specific augmentation strategies.