Adaptive Diffusion Terrain Generator for Autonomous Uneven Terrain Navigation

TL;DR

This paper introduces ADTG, a diffusion-based terrain generator that adaptively creates diverse and complex terrains for training autonomous navigation policies, significantly improving performance over existing methods.

Contribution

The novel ADTG method uses Denoising Diffusion Probabilistic Models to dynamically generate terrains tailored to current policy performance, enhancing training diversity and realism.

Findings

ADTG outperforms procedural and natural environments in training policies.

Policies trained with ADTG achieve higher navigation success rates.

ADTG seamlessly transitions between similar and novel terrains by adjusting noise levels.

Abstract

Model-free reinforcement learning has emerged as a powerful method for developing robust robot control policies capable of navigating through complex and unstructured terrains. The effectiveness of these methods hinges on two essential elements: (1) the use of massively parallel physics simulations to expedite policy training, and (2) an environment generator tasked with crafting sufficiently challenging yet attainable terrains to facilitate continuous policy improvement. Existing methods of environment generation often rely on heuristics constrained by a set of parameters, limiting the diversity and realism. In this work, we introduce the Adaptive Diffusion Terrain Generator (ADTG), a novel method that leverages Denoising Diffusion Probabilistic Models to dynamically expand existing training environments by adding more diverse and complex terrains adaptive to the current policy. ADTG guides the diffusion model's generation process through initial noise optimization, blending noise-corrupted terrains from existing training environments weighted by the policy's performance in each corresponding environment. By manipulating the noise corruption level, ADTG seamlessly transitions between generating similar terrains for policy fine-tuning and novel ones to expand training diversity. Our experiments show that the policy trained by ADTG outperforms both procedural generated and natural environments, along with popular navigation methods.