# Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout

## TL;DR

This paper introduces a novel training framework for navigation agents that improves generalization to unseen environments by using environmental dropout and semi-supervised back-translation, leading to state-of-the-art results.

## Contribution

It proposes environmental dropout to simulate unseen environments and semi-supervised back-translation for data augmentation, enhancing generalization in navigation tasks.

## Key findings

- Outperforms state-of-the-art on unseen test set
- Achieves top rank on Room-to-Room leaderboard
- Significantly improves generalization to unseen environments

## Abstract

A grand goal in AI is to build a robot that can accurately navigate based on natural language instructions, which requires the agent to perceive the scene, understand and ground language, and act in the real-world environment. One key challenge here is to learn to navigate in new environments that are unseen during training. Most of the existing approaches perform dramatically worse in unseen environments as compared to seen ones. In this paper, we present a generalizable navigational agent. Our agent is trained in two stages. The first stage is training via mixed imitation and reinforcement learning, combining the benefits from both off-policy and on-policy optimization. The second stage is fine-tuning via newly-introduced 'unseen' triplets (environment, path, instruction). To generate these unseen triplets, we propose a simple but effective 'environmental dropout' method to mimic unseen environments, which overcomes the problem of limited seen environment variability. Next, we apply semi-supervised learning (via back-translation) on these dropped-out environments to generate new paths and instructions. Empirically, we show that our agent is substantially better at generalizability when fine-tuned with these triplets, outperforming the state-of-art approaches by a large margin on the private unseen test set of the Room-to-Room task, and achieving the top rank on the leaderboard.