RoboLight: A Dataset with Linearly Composable Illumination for Robotic Manipulation

TL;DR

RoboLight is a comprehensive dataset capturing robotic manipulation under varied lighting conditions, combining real-world HDR data with synthetic interpolation to facilitate research in perception and manipulation robustness.

Contribution

The paper introduces RoboLight, a novel dataset with synchronized real and synthetic data under systematically varied lighting, enabling improved perception in robotic manipulation tasks.

Findings

High-quality HDR data captures lighting variations effectively.

Synthetic data interpolation extends dataset size and diversity.

Demonstrated use cases show improved perception robustness.

Abstract

In this paper, we introduce RoboLight, the first real-world robotic manipulation dataset capturing synchronized episodes under systematically varied lighting conditions. RoboLight consists of two components. (a) RoboLight-Real contains 2,800 real-world episodes collected in our custom Light Cube setup, a calibrated system equipped with eight programmable RGB LED lights. It includes structured illumination variation along three independently controlled dimensions: color, direction, and intensity. Each dimension is paired with a dedicated task featuring objects of diverse geometries and materials to induce perceptual challenges. All image data are recorded in high-dynamic-range (HDR) format to preserve radiometric accuracy. Leveraging the linearity of light transport, we introduce (b) RoboLight-Synthetic, comprising 196,000 episodes synthesized through interpolation in the HDR image space of RoboLight-Real. In principle, RoboLight-Synthetic can be arbitrarily expanded by refining the interpolation granularity. We further verify the dataset quality through qualitative analysis and real-world policy roll-outs, analyzing task difficulty, distributional diversity, and the effectiveness of synthesized data. We additionally demonstrate three representative use cases of the proposed dataset. The full dataset, along with the system software and hardware design, will be released as open-source to support continued research.