CHALET: Cornell House Agent Learning Environment

TL;DR

CHALET is a versatile 3D house simulation environment designed for training and evaluating autonomous agents in household tasks involving navigation, object manipulation, and multi-modal reasoning.

Contribution

It introduces a flexible, realistic house simulator supporting diverse activities and configurations, facilitating research in embodied AI and multi-modal learning.

Findings

Supports complex household tasks including object manipulation and navigation

Enables creation of diverse house layouts for varied experiments

Facilitates research in language, vision, and planning integration

Abstract

We present CHALET, a 3D house simulator with support for navigation and manipulation. CHALET includes 58 rooms and 10 house configuration, and allows to easily create new house and room layouts. CHALET supports a range of common household activities, including moving objects, toggling appliances, and placing objects inside closeable containers. The environment and actions available are designed to create a challenging domain to train and evaluate autonomous agents, including for tasks that combine language, vision, and planning in a dynamic environment.