ForestSim: A Synthetic Benchmark for Intelligent Vehicle Perception in Unstructured Forest Environments

TL;DR

ForestSim is a high-fidelity synthetic dataset designed to improve semantic segmentation for autonomous vehicles operating in unstructured forest environments, addressing the scarcity of real-world data in such settings.

Contribution

We introduce ForestSim, a comprehensive synthetic dataset with photorealistic images and pixel-accurate labels for training perception models in forested off-road environments.

Findings

State-of-the-art models perform well on ForestSim despite scene complexity.

The dataset covers diverse environments, seasons, and foliage densities.

Benchmark results demonstrate ForestSim's utility for perception system development.

Abstract

Robust scene understanding is essential for intelligent vehicles operating in natural, unstructured environments. While semantic segmentation datasets for structured urban driving are abundant, the datasets for extremely unstructured wild environments remain scarce due to the difficulty and cost of generating pixel-accurate annotations. These limitations hinder the development of perception systems needed for intelligent ground vehicles tasked with forestry automation, agricultural robotics, disaster response, and all-terrain mobility. To address this gap, we present ForestSim, a high-fidelity synthetic dataset designed for training and evaluating semantic segmentation models for intelligent vehicles in forested off-road and no-road environments. ForestSim contains 2094 photorealistic images across 25 diverse environments, covering multiple seasons, terrain types, and foliage densities. Using Unreal Engine environments integrated with Microsoft AirSim, we generate consistent, pixel-accurate labels across 20 classes relevant to autonomous navigation. We benchmark ForestSim using state-of-the-art architectures and report strong performance despite the inherent challenges of unstructured scenes. ForestSim provides a scalable and accessible foundation for perception research supporting the next generation of intelligent off-road vehicles. The dataset and code are publicly available: Dataset: https://vailforestsim.github.io Code: https://github.com/pragatwagle/ForestSim