# Towards Generalizing Sensorimotor Control Across Weather Conditions

## TL;DR

This paper presents a framework that enables vision-based vehicle control models to generalize across various weather conditions using limited labeled data, unlabeled images, and domain translation techniques.

## Contribution

It introduces a teacher-student learning approach combined with image-to-image translation to transfer knowledge across weather domains with minimal labeled data.

## Key findings

- Effective generalization across weather conditions with limited labels
- Utilization of unlabeled images improves model robustness
- Auxiliary networks reduce model size without accuracy loss

## Abstract

The ability of deep learning models to generalize well across different scenarios depends primarily on the quality and quantity of annotated data. Labeling large amounts of data for all possible scenarios that a model may encounter would not be feasible; if even possible. We propose a framework to deal with limited labeled training data and demonstrate it on the application of vision-based vehicle control. We show how limited steering angle data available for only one condition can be transferred to multiple different weather scenarios. This is done by leveraging unlabeled images in a teacher-student learning paradigm complemented with an image-to-image translation network. The translation network transfers the images to a new domain, whereas the teacher provides soft supervised targets to train the student on this domain. Furthermore, we demonstrate how utilization of auxiliary networks can reduce the size of a model at inference time, without affecting the accuracy. The experiments show that our approach generalizes well across multiple different weather conditions using only ground truth labels from one domain.