Rodrigues Network for Learning Robot Actions

TL;DR

The paper introduces the Rodrigues Network, a neural architecture that incorporates kinematic structure via the Neural Rodrigues Operator, enhancing robot action prediction and learning.

Contribution

It proposes a novel kinematics-aware neural operator and architecture that improve expressivity and performance in robot action learning tasks.

Findings

Significant improvements in synthetic kinematic and motion prediction tasks.

Effective in imitation learning on robotic benchmarks.

Enhances single-image 3D hand reconstruction.

Abstract

Understanding and predicting articulated actions is important in robot learning. However, common architectures such as MLPs and Transformers lack inductive biases that reflect the underlying kinematic structure of articulated systems. To this end, we propose the Neural Rodrigues Operator, a learnable generalization of the classical forward kinematics operation, designed to inject kinematics-aware inductive bias into neural computation. Building on this operator, we design the Rodrigues Network (RodriNet), a novel neural architecture specialized for processing actions. We evaluate the expressivity of our network on two synthetic tasks on kinematic and motion prediction, showing significant improvements compared to standard backbones. We further demonstrate its effectiveness in two realistic applications: (i) imitation learning on robotic benchmarks with the Diffusion Policy, and (ii) single-image 3D hand reconstruction. Our results suggest that integrating structured kinematic priors into the network architecture improves action learning in various domains.