Kleinkram: Open Robotic Data Management

TL;DR

Kleinkram is an open-source, modular system that manages large-scale unstructured robotic datasets with scalable storage, integrated workflows, and user-friendly interfaces, facilitating research and data sharing.

Contribution

It introduces a comprehensive, open-source platform for scalable robotic data management with integrated workflows and compatibility with standard formats.

Findings

Managed over 30 TB of robotic data successfully.

Enabled scalable storage and sharing of diverse datasets.

Streamlined data validation and benchmarking workflows.

Abstract

We introduce Kleinkram, a free and open-source system designed to solve the challenge of managing massive, unstructured robotic datasets. Designed as a modular, on-premises cloud solution, Kleinkram enables scalable storage, indexing, and sharing of datasets, ranging from individual experiments to large-scale research collections. Kleinkram natively integrates with standard formats such as ROS bags and MCAP and utilises S3-compatible storage for flexibility. Beyond storage, Kleinkram features an integrated "Action Runner" that executes customizable Docker-based workflows for data validation, curation, and benchmarking. Kleinkram has successfully managed over 30 TB of data from diverse robotic systems, streamlining the research lifecycle through a modern web interface and a robust Command Line Interface (CLI).