Deep Functional Predictive Control for Strawberry Cluster Manipulation using Tactile Prediction

TL;DR

This paper presents a deep functional predictive control method utilizing tactile prediction models to improve the manipulation of strawberries and similar objects during robot pushing tasks, addressing complex physical interactions.

Contribution

It introduces a novel deep predictive control system that uses tactile predictions to manage complex robot-object interactions during pushing tasks.

Findings

Successfully controlled strawberry stem displacement in real robot tests.

Demonstrated effectiveness beyond strawberry manipulation.

Offers a promising approach for complex physical robot interactions.

Abstract

This paper introduces a novel approach to address the problem of Physical Robot Interaction (PRI) during robot pushing tasks. The approach uses a data-driven forward model based on tactile predictions to inform the controller about potential future movements of the object being pushed, such as a strawberry stem, using a robot tactile finger. The model is integrated into a Deep Functional Predictive Control (d-FPC) system to control the displacement of the stem on the tactile finger during pushes. Pushing an object with a robot finger along a desired trajectory in 3D is a highly nonlinear and complex physical robot interaction, especially when the object is not stably grasped. The proposed approach controls the stem movements on the tactile finger in a prediction horizon. The effectiveness of the proposed FPC is demonstrated in a series of tests involving a real robot pushing a strawberry in a cluster. The results indicate that the d-FPC controller can successfully control PRI in robotic manipulation tasks beyond the handling of strawberries. The proposed approach offers a promising direction for addressing the challenging PRI problem in robotic manipulation tasks. Future work will explore the generalisation of the approach to other objects and tasks.