Learning Dynamics of a Ball with Differentiable Factor Graph and Roto-Translational Invariant Representations

TL;DR

This paper introduces an end-to-end learning framework combining a dynamics model and factor graph estimator, utilizing roto-translational invariant representations and a novel network architecture to accurately predict ball trajectories in dynamic environments.

Contribution

It presents a novel approach that integrates a Gram-Schmidt process and self-multiplicative bypasses to improve trajectory prediction accuracy in complex sports scenarios.

Findings

Achieved RMSE of 37.2 mm at first bounce

Achieved RMSE of 71.5 mm after second bounce

Reduced validation error compared to data augmentation methods

Abstract

Robots in dynamic environments need fast, accurate models of how objects move in their environments to support agile planning. In sports such as ping pong, analytical models often struggle to accurately predict ball trajectories with spins due to complex aerodynamics, elastic behaviors, and the challenges of modeling sliding and rolling friction. On the other hand, despite the promise of data-driven methods, machine learning struggles to make accurate, consistent predictions without precise input. In this paper, we propose an end-to-end learning framework that can jointly train a dynamics model and a factor graph estimator. Our approach leverages a Gram-Schmidt (GS) process to extract roto-translational invariant representations to improve the model performance, which can further reduce the validation error compared to data augmentation method. Additionally, we propose a network architecture that enhances nonlinearity by using self-multiplicative bypasses in the layer connections. By leveraging these novel methods, our proposed approach predicts the ball's position with an RMSE of 37.2 mm of the paddle radius at the apex after the first bounce, and 71.5 mm after the second bounce.