HoME: a Household Multimodal Environment

TL;DR

HoME is a comprehensive, multimodal simulation environment based on diverse 3D house layouts, designed to advance artificial agents' learning across vision, audio, physics, and interaction tasks.

Contribution

We present HoME, an open-source, scalable platform integrating multimodal sensory inputs within realistic household environments for AI research.

Findings

Supports diverse AI tasks including reinforcement learning and language grounding

Enables learning and transfer in realistic, multimodal household settings

Open-source platform compatible with OpenAI Gym

Abstract

We introduce HoME: a Household Multimodal Environment for artificial agents to learn from vision, audio, semantics, physics, and interaction with objects and other agents, all within a realistic context. HoME integrates over 45,000 diverse 3D house layouts based on the SUNCG dataset, a scale which may facilitate learning, generalization, and transfer. HoME is an open-source, OpenAI Gym-compatible platform extensible to tasks in reinforcement learning, language grounding, sound-based navigation, robotics, multi-agent learning, and more. We hope HoME better enables artificial agents to learn as humans do: in an interactive, multimodal, and richly contextualized setting.