Feature-Based Transfer Learning for Robotic Push Manipulation

TL;DR

This paper introduces a data-efficient transfer learning method for robotic push manipulation that uses local surface features and point cloud models to generalize across different object shapes, reducing sample complexity and improving transferability.

Contribution

It extends previous contact-based predictors by incorporating local surface features and point cloud models, enabling better generalization without requiring CAD models.

Findings

Outperforms baseline predictors in simulation tests.

Achieves transfer learning across various object shapes.

Demonstrates real robot proof of concept.

Abstract

This paper presents a data-efficient approach to learning transferable forward models for robotic push manipulation. Our approach extends our previous work on contact-based predictors by leveraging information on the pushed object's local surface features. We test the hypothesis that, by conditioning predictions on local surface features, we can achieve generalisation across objects of different shapes. In doing so, we do not require a CAD model of the object but rather rely on a point cloud object model (PCOM). Our approach involves learning motion models that are specific to contact models. Contact models encode the contacts seen during training time and allow generating similar contacts at prediction time. Predicting on familiar ground reduces the motion models' sample complexity while using local contact information for prediction increases their transferability. In extensive experiments in simulation, our approach is capable of transfer learning for various test objects, outperforming a baseline predictor. We support those results with a proof of concept on a real robot.

Figures

Tables

Table 1. TABLE I: Average normalised error, models trained on 500 500 500 training pushes

Object	Baseline	RO	RO+3OE	RO+5OE
Simulation
Cuboid	$0.185\pm 0.126$	$0.035\pm 0.041$	$0.036\pm 0.035$	$0.036\pm 0.036$
Cube	$0.176\pm 0.127$	$0.059\pm 0.081$	$0.052\pm 0.067$	$0.052\pm 0.067$
Rounded	$0.196\pm 0.141$	$0.057\pm 0.075$	$0.059\pm 0.068$	$0.061\pm 0.071$
Triangular	$0.254\pm 0.130$	$0.153\pm 0.086$	$0.124\pm 0.095$	$0.129\pm 0.097$
Cylinder	$0.219\pm 0.111$	$0.129\pm 0.068$	$0.149\pm 0.111$	$0.160\pm 0.119$
Total	$0.206\pm 0.131$	$0.087\pm 0.085$	$0.084\pm 0.091$	$0.088\pm 0.096$
Real robot
Large box	$0.213\pm 0.098$	$0.127\pm 0.027$	$0.161\pm 0.053$	$0.163\pm 0.054$
Small box	$0.276\pm 0.077$	$0.064\pm 0.054$	$0.113\pm 0.071$	$0.106\pm 0.076$
Total	$0.244\pm 0.093$	$0.096\pm 0.052$	$0.137\pm 0.066$	$0.135\pm 0.072$

Equations

$$S=\{x_j:x_j\in {\mathbb{R}}^3\times SO(3)\times {\mathbb{R}}^2{\}}_{j\in [1,N_S]}.$$

$$\mathcal{K}(x\mid \mu ,\sigma )={\mathcal{N}}_3(p\mid {\mu }_p,{\sigma }_p)\Theta (q\mid {\mu }_q,{\sigma }_q){\mathcal{N}}_2(r\mid {\mu }_r,{\sigma }_r),$$

$$P(x)\simeq \sum _{j=i}^K{w}_j\mathcal{K}(x\mid x_j,\sigma ),$$

$$\mathcal{M}(m|\mu ,\sigma )={\mathcal{N}}_3(p\mid {\mu }_p,{\sigma }_p)\Theta (q\mid {\mu }_q,{\sigma }_q),$$

$$u_j=v_j^{-1}\circ s,$$

$$C^L$$

$$\frac{1}{Z}\sum _{j=1}^{N_O}{w}_j{\mathcal{N}}_3(p\mid p_j,{\sigma }_p)\Theta (q\mid q_j,{\sigma }_q){\mathcal{N}}_2(r\mid r_j,{\sigma }_r),$$

$$w_j=\{\begin{array}{ll}1& \text{if }\Vert p_j-z_j\Vert <{\delta }_E\\ 0& \text{otherwise,}\end{array}$$

$$Q=P(s,u,v,r),$$

$$M(m_b\mid c,a)=\frac{{\sum }_{j=i}^{N_a}{w}_j\mathcal{M}(m_b\mid m_j,{\sigma }_m)\mathcal{K}(c\mid c_j,{\sigma }_c)}{{\sum }_{j=i}^{N_a}{w}_j\mathcal{K}(c\mid c_j,{\sigma }_c)},$$

$$d_{ang}=1-(q_{test}\cdot q_{pred}{)}^2,$$

$$d_{norm}=\frac{1}{2}{d}_{ang}+\frac{1}{2L}{d}_{lin},$$

Full text

I INTRODUCTION

Pushing is an essential skill for both humans and robots. While prehensile manipulation is versatile and elegant, neither is it always possible nor suitable. Objects may be too large or heavy to grasp or the robot may not be equipped with grippers. Mobile robots in particular are posed to encounter an unpredictable variety of real-world manipulation tasks without the ability to grasp, i.e. adding a gripper to NASA’s K-Rex rover means extra payload [7]. Rocks on the moon may nevertheless obstruct its path. In a more mundane context, precise pushing allows industrial and personal robots to operate efficiently in cluttered environments by removing obstacles and reducing pre-grasp uncertainty [3]. In both described contexts, robots may encounter objects of various sizes, shapes, and materials. Hence, there is a strong case for learning transferable forward models of robotic push manipulation.

However, generalisation across objects is challenging and still an open problem. Analytical approaches to pushing rely on knowledge of intrinsic physical parameters which may not be available, or expensive to obtain. Data-driven pushing models tend to either generalise poorly, require impractically large amounts of data, or both.

We present a contact-based approach to push prediction that enables generalisation across objects of different shapes. Our aim is not to improve the quality of predictions for a specific object, but to efficiently learn a generative model that will enable the robot to make reasonable predictions on novel objects. We have also based our approach on features that can be extracted from a point cloud obtained on the fly by the robot, so to remove the need of modelling various types of objects. We do so by approximating the global motion of the object by learning a set of local experts. Each expert is a specialised predictor learned from demonstration.

In particular, we learn two types of experts: 1) static contact models and 2) motion models. The former models encode the contact between the pusher and the object, and between the object and the environment. Meanwhile, the latter models make predictions on how the object will behave under a specific push. In other words, instead of learning motions for a specific object, we break down the problem to learn how few local points of interest (i.e. robot-object and object-environment contacts) will change under a push. We then use those predictions to estimate the global motion of the object.

Figure 1 shows our test scenario in which a Pioneer 3DX pushes an object not previously encountered, composed of two piled dice. All our models have been trained in a simulated environment, also the one tested in the real scenario. To cope with the discrepancy between the simulated and the real world, we learned the motion models by rollouts with sampled friction coefficients (see Sec. IV-C).

When encountering a novel object, we use the contact model to generate robot-object and object-environment contacts similar to those seen during training, and apply the contact-specific motion models to predict how the object will behave under a push. As the generated contacts are similar to those seen during training time, the motion models can predict on familiar ground.

Our main contributions are thus twofold. First, we propose a novel approach to transfer learning of robotic push manipulation. Second, we test that approach in extensive experiments in a simulated environment, and in a proof of concept on a real robot.

II RELATED WORK

One approach to predicting the result of a push involves building models informed by classical mechanics [12, 13, 11]. In order to be precise, such models require explicit representation of intrinsic physical parameters such as friction. Most of those approaches focus on planar sliding of simple two-dimensional (2D) objects. Recently, there has been a promising stream of work, however, that aims to augment analytical models with data-driven approaches, e.g. for modelling the stochastic nature of friction [16].

In view of avoiding the tuning of physical parameters, a second approach is to learn forward models from data, thereby implicitly encoding physical parameters in the model. A wide range of features and machine learning techniques have been used for pushing prediction, e.g. recently [2]. However, most approaches tend to work well only on the objects that they have been trained on, although notable exceptions exist, e.g. [10]. Recently, artificial neural networks have been applied to robotic pushing. Such deep learning approaches typically use RGB images as input, and may be used to construct end-to-end systems which optimise perception, prediction, and control jointly for task performance. While such work holds great promise, existing approaches require hundreds of hours of video material corresponding to thousands of actions for training to be applicable in real world scenarios [1, 4]. Generalisation across objects further increases sample complexity.

Most related to our approach is our own previous work [10, 17, 9]. In [10, 17], we dealt with the problem of learning forward models to predict how objects behave under push operations, and in [9], we proposed to use local features to transfer learned grasps to novel objects. In this paper, we investigate both approaches further to learn contact-specific motion models for push operations.

III BACKGROUND

III-A Surface features

A central aspect of our approach is the probabilistic modelling of surface features, extracted from three-dimensional (3D) object point clouds. Features are composed of a 3D position, a 3D orientation, and a 2D local surface descriptor that encodes local curvatures. Let us denote by $SO(3)$ the group of rotations in three dimensions. A feature belongs to the space $SE(3)\times\mathbb{R}^{2}$ , where $SE(3)=\mathbb{R}^{3}\times SO(3)$ is the group of 3D poses, and surface descriptors are composed of two real numbers. We thus represent an object as a set $S$ of $N_{S}$ surface features $x_{j}$

[TABLE]

Let us denote the separation of a feature $x$ into $p\in\mathbb{R}^{3}$ for position, a quaternion $q\in SO(3)$ for orientation, and $r\in\mathbb{R}^{2}$ for the surface descriptor. For compactness, we also denote the pose of a feature as $v$. As a result, we have $x=(v,r)$, and $v=(p,q)$.

We now describe how we compute $r$, the local curvature descriptors. The surface normal at $p$ is computed from the nearest neighbours of $p$ using a PCA-based method, e.g. [6]. Surface descriptors correspond to the local principal curvatures [15]. The curvature at point $p$ is encoded along two directions that both lie in the plane tangential to the object’s surface, i.e. perpendicular to the surface normal at $p$. The first direction, $k_{1}\in\mathbb{R}^{3}$ , is a direction of the highest curvature. The second direction, $k_{2}\in\mathbb{R}^{3}$, is perpendicular to $k_{1}$. The curvatures along $k_{1}$ and $k_{2}$ are denoted by $r_{1}\in\mathbb{R}$ and $r_{2}\in\mathbb{R}$ respectively, forming a 2-dimensional feature vector $r=(r_{1},r_{2})\in\mathbb{R}^{2}$. The surface normals and principal directions allow us to define the 3D orientation $q$ that is associated to a point $p$. Next, we introduce the notation and concepts required to represent motion models.

III-B Rigid body motions

Let us now consider the problem of a robotic agent with link $L$ pushing an object $B$, observed at discrete time steps $t$, $t+1$; see Fig. 2 for a visual representation of the notation. We assume that both $L$ and $B$ are 3D rigid bodies and that the interaction between them is quasi-static. In this context quasi-static means that the object only moves when it is being pushed by the robot with a constant low-speed velocity. Let $l_{t}\in SE(3)$ denote the pose of link $L$ and $b_{t}\in SE(3)$ the pose of $B$, measured at time $t$ in an inertial reference frame $W$ which we will refer to as the world frame. In this paper, we aim to determine the motion of an object resulting from a pushing action $a$ from an action set $\mathcal{A}$. Let $m_{b}\in SE(3)$ denote the rigid body transformation from $b_{t}$ to $b_{t+1}$. Using the same notation introduced for 3D poses, we represent the separation of $m_{b}$ into $p\in\mathbb{R}^{3}$ for the translation, and a quaternion $q\in SO(3)$ for the rotation. Our approach is probabilistic in that we we aim to learn not a deterministic mapping $a\mapsto m_{b}$ but rather a conditional probability density function (PDF) over rigid body motions $P(m_{b}\mid a)$.

In order to determine $P(m_{b}\mid a)$, we use a PoE. The key idea of this approach is that candidate motions are evaluated by a set of models (the experts), each giving a likelihood (opinion). For a candidate to be considered likely, all experts must agree. This is implemented by taking the product of opinions. In contrast to mixture models, each expert thus has a veto in the sense that a single zero probability results in an overall zero likelihood.

In our context of motion prediction, we consider multiple local predictors of the global object motion $m_{b}$ (see Fig. 2). Each of those models predicts the object’s local motion at point where the object is in contact with the robot or the environment. The PoE is a natural fit for this context as each contact encodes a local kinematic constraint on the object’s motion. To be considered probable, a motion $m_{b}$ must satisfy all of those constraints simultaneously. To formalise this, let us denote a local surface feature in contact with robot link $L$ as $x^{L}=(v^{L},r^{L})$. We represent the contact by the pose $u^{L}\in SE(3)$ of $L$ relative to the local feature frame $v^{L}$. In addition to the robot-object contact, we consider at set of $N_{E}$ contacts $u^{E}_{k}$ with corresponding features $x^{E}_{k}=(v^{E}_{k},r^{E}_{k})$. Let us now again consider the robot applying a pushing action $a$ to the object. In addition to $m_{b}$, we can now additionally observe the motion of each local feature frame, namely $m_{v^{L}}$ for the robot-object contact, and $m_{v^{E}_{k}}$ for the object-environment contacts where $k=1,\dotsc,N_{E}$ .

We note that each local contact feature frame $v^{L}$ and $v^{E}_{k}$ resides at a relative pose $h$ to $b$ which, for a particular $v$ is given by $h=v^{-1}\circ b$, where $\circ$ denotes the pose composition operator, and $v^{-1}$ is the inverse of $v$, with $v^{-1}=(-q^{-1}p,q^{-1})$. As we assume that we are dealing with rigid bodies, $h_{t}=h_{t+1}\ \forall\ t$. For any given $v$, $m_{v}$ and $h$, we can thus compute the corresponding $m_{b}$. In other words, given the initial pose of a local contact frame, its relative pose to the object pose, and its local motion, we can compute the corresponding object motion. Coming back to our previous set-up, we learn a model for the motion of the robot-object contact frame $v^{L}$ given by $P(m_{v^{L}}\mid a)$ and a motion model for object-environment contact frames, given by $P(m_{v^{E}_{k}}\mid a)$ where $k=1,\dotsc,N_{E}$ . Given the relative poses $h^{L}$ and $h^{E}_{k}$, those motion models also define probability densities over $m_{b}$. Crucially, this formulation is transferable in that the learned motion is given in the local contact frame. When predicting the motion for a novel object, the corresponding relative poses $h^{L}$ and $h^{E}_{k}$ are used to generate $P(m_{b}\mid a)$. In the next section, we discuss how we estimate probability densities from data.

III-C Kernel density estimation

In this paper, we represent PDFs non-parametrically with a set of features, or particles. The underlying PDF is created through kernel density estimation [14], by assigning a kernel function to each particle supporting the density. For contact models, we consider PDFs defined on surface features $x$, i.e. on $SE(3)\times\mathbb{R}^{2}$. For that purpose, let us denote by $\mu$ a surface feature given by $\mu=(\mu_{p},\mu_{q},\mu_{r})$, and by $\sigma$ a triplet of real numbers $\sigma=(\sigma_{p},\sigma_{q},\sigma_{r})$. We thus define our kernel as

[TABLE]

where $\mu$ is the kernel mean point, $\sigma$ is the kernel bandwidth, $\mathcal{N}_{n}$ is an $n$-variate isotropic Gaussian kernel, and $\Theta$ corresponds to a pair of antipodal von Mises-Fisher distributions which form a Gaussian-like distribution on $SO(3)$ (for details see [5]). Given a set of $K$ surface features, the probability density in a region of space is then determined by the local density of the particles in that region, as

[TABLE]

where $\sigma\in\mathbb{R}^{3}$ is the kernel bandwidth and $w_{j}\in\mathbb{R}^{+}$ is a weight associated to $x_{j}$ such that $\sum_{j}w_{j}=1$. For motion models, we consider a kernel function defined over surface features and 3D motions, hence over $SE(3)\times SE(3)\times\mathbb{R}^{2}$. That kernels is defined as the product of $\mathcal{K}$ and the kernel function

[TABLE]

where $m=(p,q)$ is the motion to be evaluated, $\mu=(\mu_{p},\mu_{q})$, and $\sigma=(\sigma_{p},\sigma_{q})$.

IV PROPOSED APPROACH

At training time, we learn a contact model comprising both a robot-object contact model and an object-environment contact model. For each action in our action set, we then learn local motion models specific to that contact model. At prediction time, we can query any novel point cloud to find the closest set of contacts and interpolate a prediction for the object movement.

IV-A Contact model

A contact model encodes the joint probability distribution of surface features and of the 3D pose of the robot’s link in contact. At prediction time, we obtain a point cloud $O$ of the novel object from a single shot taken from a depth camera. Given a set of $N_{O}$ surface features $\{x_{j}\}_{j=1}^{N_{O}}$, with $x_{j}=(v_{j},r_{j})$ and $v_{j}=(p_{j},q_{j})$, a robot-object contact model $C^{L}$ is constructed from features from the object’s surface. Surface features close to the link surface are more important than those lying far from the surface. Features are thus weighted, to make their influence on $C^{L}$ decrease with their squared distance to the link. Let us denote by $u_{j}=(p_{j},q_{j})$ the pose of $L$ relative to the pose $v_{j}$ of the $j$th surface feature. In other words, $u_{j}$ is defined as

[TABLE]

where $s$ denotes the pose of $L$ in the world frame, $\circ$ denotes the pose composition operator, and $v_{j}^{-1}$ is the inverse of $v_{j}$. The contact model is then estimated as

[TABLE]

where $Z$ is a normalising constant, and $u=(p,q)$.

Additionally, we learn an object-environment contact model $C^{E}$ from $N_{C^{E}}$ samples obtained from $O$. For each sampled feature $x_{j}$, we attach a frame $z_{j}$ to the closest point in the environment and represent the environment contact by the pose $u_{j}$ of $z_{j}$ relative to $v_{j}$. For the object-environment contact model, we opted for a simple binary weighting function

[TABLE]

where $\delta_{E}$ is a cut-off distance. We then estimate the object-environment contact model using the same formulation given for the robot-object contact model.

IV-B Query density

A query density results from the combination of a learnt contact model with a novel object point cloud $O$. The purpose of a query density is both to generate and evaluate poses of the corresponding robot’s link (or the environment) on the new object. Imagine you have only learned how to push a cube from one of its side faces. When you need to push a triangular prism you will not place your finger on one of the corners, expecting it to move like the cube. However, if you place your finger on a side face you may be able to use your experience on the cube to make a prediction. We refer the reader to [9] for further details on how to compute the query density, providing only a brief overview here. In summary, a query density $Q$ is a PDF defined as

[TABLE]

where $v$ denotes a point on the object’s surface, expressed in the world frame, $r$ is the surface curvature of such a point, $u$ denotes the pose of a body relative to a local frame on the object, and $s$ is the absolute pose in the world frame of the body. In our context, the body is either the robot’s link or a local environment frame.

At prediction time, we use a robot-object query density $Q^{L}$ to generate poses of the robot’s link on the new object, and to attach a robot-object contact frame as an expert for prediction. We achieve the former by marginalising with respect to $u$, $v$, and $r$, obtaining the distribution $Q^{L}(s)$ which models the pose $s$ in the world frame of the link $L$. We approximate the robot-object query density by $K_{Q^{L}}$ kernels centred on the set of weighted robot link poses obtained from the pose sampling algorithm proposed by [9]. To generate a robot-object feature frame $v^{L}$, we choose it such that it maximises $Q^{L}(v)$. For optimisation we use simulated annealing [8].

In addition to the robot-object query density, we create an object-environment query density by combining the object-environment contact model with $O$. While we are free to position the robot in the world, we consider the state of the environment as fixed for a given time $t$. We hence use the object-environment query density only to select the set of $N_{E}$ query object-environment contact frames $v^{E}_{k}$. This is equivalent to marginalising $Q^{E}$ with respect to $s$, $u$, and $r$ to obtain the distribution $Q^{E}(v)$ over poses in the world frame of local feature frames on the object’s surface. We then sample from $Q^{E}(v)$ for a total of $N_{E}$ times.

IV-C Motion model

Having learned a contact model, we then proceed to learning both a robot-object motion model, and an object-environment motion model for each action $a$ in the action set. We learn those models from the data by observing the local motion of robot-object and object-environment frames at training time. For each action $a$, we simulate a set of $N_{a}$ rollouts which are the kernels of the corresponding motion model. Each rollout has a different friction coefficient for the contacts uniformly sampled from a distribution in the range $[0.15,0.35]$, similar to the approach taken by [16].

In doing so, we learn to predict motion conditional on both the contact pose $u$ and the local curvature $r$ in the form of PDFs $P(m_{v^{L}}\mid u^{L},r^{L},a)$ and $P(m_{v^{E}_{k}}\mid u^{E}_{k},r^{E}_{k},a)$. At prediction time, we use the relative poses $h^{L}$ and $h^{E}_{k}$ of the contact frames with respect to the object pose $b$ to define PDFs over object motion P($m_{b}\mid u^{L},r^{L},a)$ and $P(m_{b}\mid u^{E}_{k},r^{E}_{k},a)$ from the local motion PDFs, as discussed before. Each kernel in the PDF of each motion model is defined over $SE(3)\times SE(3)\times\mathbb{R}^{2}$. In order to generate predictions, we obtain an opinion on candidate motions from each local expert defined as the conditional PDF

[TABLE]

where $m_{b}=(p_{m_{b}},q_{m_{b}})$ is the candidate object motion to evaluate, $c=(u,r)$ is the relative pose and surface curvature associated with the conditioning contact frame, $m_{j}$ is the $j$th motion kernel, $\sigma_{m}$ is the motion kernel bandwidth parameter, and $\sigma_{c}=(\sigma_{p},\sigma_{q},\sigma_{r})$ is the contact feature bandwidth parameter. Finally, we combine all local motion models in a PoE, as introduced in the background section, and select the best prediction by obtaining the argument with the maximum likelihood. For that purpose, we again use simulated annealing as a convenient optimisation procedure.

V EXPERIMENTS

We evaluated our approach empirically in experiments on a simulated and a real Pioneer 3DX mobile robot equipped with a bumper for pushing objects. Our experiments focus on object transfer, the main aspiration of this paper. We learned contact and motion models in simulation for one object (i.e. the cube), and then evaluated the predictive performance of the learned models on a set of five test objects in simulation, and two test objects in a real world scenario. We also compared our approach to a baseline predictor which encodes the planar sliding constraints (Sec. V-D).

For our experiments, we varied the trained models along two dimensions, first the contact information used for prediction, and second the number of training samples. We considered three values in each dimension and all combinations between them, leading to a total of nine evaluated models. During both training and testing, all objects were placed in an obstacle-free planar environment. Due to the geometry of the objects considered, and the kinematics of our robot, our experiments are a study of planar sliding behaviour. Investigating performance in the context of rolling and toppling, as well as in cluttered environments, is future work. For all simulation experiments, we used the Gazebo simulator in version 7.8 with the Open Dynamics Engine (ODE).

V-A Training set

As our training object, we chose a cube with side length $0.2$ m and mass $0.5$ kg. Using a Kinect depth camera, we took a single shot of the object, and learned contact models from the thus obtained point cloud. Specifically, we learned two robot-object contact models. Due to the symmetry of the bumper, we only distinguish between front and side link without differentiating between left and right.

In the case of the first robot-object contact model, the bumper’s front link is in full contact with one face of the cube. In the case of the second robot-object contact model, the bumper’s side link is in full contact with a face of the cube. In both cases, we used a cut-off distance of $1$ cm for the weighting function. Furthermore, we learned an object-environment contact model from $1,000$ surface features sampled from the point cloud. Here, we used a cut-off distance of $5$ cm for the weighting function. We note that in our set-up devoid of obstacles and clutter, the object’s environment contacts are exclusively made with the ground.

In the next step, we learned a motion model for each combination of action and robot-object contact model. To that end, we generated training pushes as follows. For each robot-object contact model, we constructed a query density from which we generated $100$ feasible training contacts, discarding infeasible instances that placed the robot in mid-air or in collision with either the object or the environment. At each feasible contact, we then repeatedly applied a set of pushes. Specifically, we considered an action set of three pushes, one straight linear push and two angular pushes. In all cases, a velocity command was fed to the robot’s controller for a fixed duration. We used a duration of four seconds. Linear velocity amounted to $0.1$ m in the $x$-direction of the robot’s base frame for all pushes. For the straight linear push, all components of the angular velocity vector were set to the zero, while for the two angular pushes, we considered angular velocities of $10^{\circ}/s$ and $-10^{\circ}/s$ respectively, directed around the $z$-axis of the robot’s base frame. In the case of the side link’s contact model, we excluded the angular push directed away from the contact surface from the action set due to its resulting in a loss of contact without applying a push.

At each training contact, we executed each action five times (i.e. rollouts, Sec. IV-C), gathering a set of $2,500$ training pushes, $1,500$ for the front link’s contact model, and $1,000$ for the side link’s contact model.

V-B Test set

We evaluated the learned motion models both in simulation and on a real robot. Considering the simulated environment first, we generated a test set of pushes covering five objects, one being the cube seen during training, and the other four being unseen objects. This design reflects our focus on object transfer. The unseen objects comprise a cuboid with side lengths $0.2$ m and $0.3$ m, an equilateral triangular prism with side length $0.2$ m, a cylinder of radius $0.1$m and height $0.2$ m, and a shape derived from the equilateral triangular prism by rounding off one of its edges. We will refer to the latter object as a rounded triangular prism. All considered objects have the same mass of $0.5$ kg. We chose the objects such that they exhibit variance with respect to the shape, area, and curvature of their surfaces which in turn determine the nature of the contacts that can be generated.

We generated test pushes for the selected objects following the same process used to gather training data. For each test object, we obtained a single shot from a Kinect depth camera. For each of the two learned contact models, we constructed a query density and sampled $50$ query poses. For each query pose, we then applied each action from the corresponding action set four times. For each test push, we sampled the contact friction coefficients from the same distribution used during training. Hence, we obtained a test set of $1,000$ pushes per training object, $600$ for the front link’s contact model and $400$ for the side link’s contact model. Considering all objects, our test set thus comprised $5,000$ pushes. For model selection, we additionally generated a separate validation set of $1,000$ pushes. We tuned the kernel bandwidth parameters on the validation set and kept them constant across all experiments.

Regarding, the evaluation on the real robot, we considered a test set of two boxes. The first, with length $0.35$ m, width $0.16$ m, and height $0.15$ m, we will refer to as the large box. Further, we will refer to the second, with length $0.22$ m, width $0.17$ m, and height $0.12$ m, as the small box. Again, we obtained point clouds of the objects from single shots of a Kinect camera. Subsequently, we estimated each object’s initial pose from the point cloud. First, we approximated the position of its centre of mass (COM) as the centroid of the point cloud. Then, we computed the object’s orientation from the eigenvectors of the point cloud’s covariance matrix. After applying the push, we obtained the object’s final pose from a new shot of the depth camera using the same approach. As in simulation, we considered both contact models (front and side), and generated test contacts from the query density. However, we pruned the angular pushes from the action set, considering only the linear case. Following the described process, we generated $40$ test pushes for this initial proof of concept. We will extend the scope of those experiments in prioritised future work.

V-C Predictions

We now proceed to describe how we generated predictions. For each generated test contact, we attached a set of frames. First, we attached a global object frame at the object’s estimated COM. Both in simulation and on the real robot, we estimated the location of the COM as the centroid of the point cloud.

Subsequently, we determine the robot-object and the object-environment contact frames by using the query density as described in Sec. IV-B. In our experiments, we also aim to investigate the benefit of learning object-environment contacts to improve predictions. Hence, in our results, we denote the predictor based on only robot-object contact information as RO, and the predictors with additional access to three and five object-environment contacts as RO+3OE, and RO+5OE respectively.

Additionally, we varied the number of training pushes available to our models. In simulation, we considered the cases of $100$, $200$, and $500$ training pushes; on the real robot, we considered $500$ pushes only. We thus trained nine different models, namely each of RO, RO+3OE, and RO+5OE with the three considered training set sizes. Based on the available input information, and number of samples, each predictor generated $500$ candidate object motions for each test contact using simulated annealing. Every one of those candidates was generated as the likeliest of $100$ samples obtained from the probability density over object motions, and then optimised in $100$ iterations of the simulated annealing algorithm. Of the $500$ optimised candidates, we kept the $10$ predictions with the maximum likelihood scores. Finally, we computed error statistics for all predictions and compared them to a baseline predictor. We describe our methodology used thereby next.

V-D Baseline predictor

As a performance baseline for comparison, we used a baseline predictor to all test pushes. Although simple, this predictor has several advantages over our proposed models. First, it knows the object’s true COM. Foremost, though, it encodes the planar sliding constraints of our set-up. Thus, the predictor knows that objects will not topple or roll, and that they cannot penetrate the ground. In contrast, our contact-based predictors need to learn those constraints from the data.

We implemented it such that for the $i$th action from the action set, it applies a translation $b$ to the initial pose in world coordinates of the object’s true COM, to which the predictor has access. For the linear push that translation $b$ is computed by transforming the translation given by $a_{i}\cdot l$ from the robot’s base frame into world coordinates. Here, $a_{i}$ is the linear velocity vector of the velocity command corresponding to the $i$th action, and $l$ is the action duration. For angular pushes, we followed the same process but instead of $a_{i}$ compute the translation from the vector $\tilde{a}_{i}$ which is the vector in the $xy$-plane of the robot’s base frame that lies at an angle $\alpha$ to $a_{i}$, where $\alpha$ is the angular velocity around the $z$-axis of the robot’s base frame corresponding to the $i$th action.

V-E Performance measure

For each prediction, we computed both a linear and an angular distance metric with regard to the final object pose. For linear distance $d_{lin}$, we opted for Euclidean distance. For angular distance $d_{ang}$, we used a quaternion distance metric defined as

[TABLE]

where where $q_{test}$ and $q_{pred}$ are unit quaternions representing the final orientation in world coordinates of the object in test sample, and prediction respectively. To obtain a unified distance measure, we combine $d_{lin}$ and $d_{ang}$ by computing the normalised error $d_{norm}$ as

[TABLE]

where $L$ is a normalisation constant. We set $L=\lVert a_{lin}\rVert\cdot l$ where $\lVert a_{lin}\rVert$ is the Euclidean norm of the linear velocity vector $a_{lin}$ representing the linear push from our action set, and $l$ is the duration of the push applied during experiments. We chose that normalisation constant because it constitutes a critical parameter of the problem. All else equal, the longer the push, the more challenging the prediction. In our specific set-up, $L=0.4$.

V-F Results

In simulation and in experiments with the real robot, all of the trained models outperformed the baseline predictor for all test objects with regard to the average normalised error. The lowest overall error value across contact information and sample sizes, with a magnitude of $0.084$, was achieved by RO+3OE with $500$ test samples (see Fig. 4). Breaking this down into its non-normalised components, it is equivalent to an average linear error of $6.171$ cm, and an average angular error of $0.031$. All trained models furthermore exhibited a lower standard deviation of the normalised error than the baseline predictor. Nevertheless, we see great variability in predictions with some of the standard deviations similar in magnitude to the average error itself.

Concerning the size of the training set (Fig. 4, left), our data indicates a relationship between model complexity and sample complexity. For $100$ training samples, the simplest model RO performs best while RO+5OE fares worst. As the number of training pushes increases, the performance of RO remains approximately flat. Meanwhile, the error values for RO+3OE and RO+5OE decrease. At $500$ pushes, RO+3OE surpasses RO as the top performer while while RO+5OE reaches a similar performance level as RO.

Considering differences between objects (Fig. 4, right), we find that, overall, the object-environment contacts do not significantly improve the quality of predictions, besides for the triangular prism. In some cases, discordant outputs from the local predictors diminished performance, which can happen with the PoE. Nevertheless, the predictions are still more accurate than the baseline predictor.

VI DISCUSSION & FUTURE WORK

The experimental results obtained in simulation provide empirical support for the hypothesis that our approach enables object transfer. For objects with sufficiently similar contact geometry, performance is comparable or even better than for the training object. Objects with very different curvatures (the cylinder), and a different area of the support surface (the triangular prism) challenge our predictors. In the first case, the query density is unable to generate sufficiently similar contacts. In the latter case, although similar contacts can be generated, the dynamics of the test object are too different from those encoded by the motion model. In the case of the rounded triangular prism, predictions are accurate as the query density is able to generate contacts on the flat faces while the support surface is large enough to result in motions similar to those of the training cube. One possible approach to increasing prediction accuracy further is to try to increase the generalisation capabilities of the learned models. A more simple, yet promising alternative is learning additional contact and motion models, and selecting the most suitable ones at prediction time. We expect that learning one model for flat surface and one for curved surfaces alone will greatly increase prediction accuracy.

Regarding the high variability of prediction accuracy, we find a large difference in performance between the two contact models. For the front link’s contact model, the average normalised error across all models is $0.072$, compared to a value of $0.113$ for the left link’s contact model. Further analysis of the data shows that for contacts with the left link, depending on the query pose and contact friction, the object came in contact with the robot’s wheels, leading to highly variable results. This is a problem that can be more readily addressed through improved robot design than through modifications of the predictors.

Another reason for variable prediction performance lies in the differing quality of generated query poses. Not all generated query poses are equally similar to the contacts seen during training time. Hence, predictions may require more or less generalisation capability depending on the query pose. Investing more effort into generating optimised query poses may thus be worth the cost depending on the achieved performance yield. Investigating this question is an interesting route for future work.

With regard to performance gains achieved by adding further contact information, our results are mixed. Overall best performance is achieved by RO+3OE but RO performs better for small training sets. Even in the case of $500$ training samples, for some objects, adding contact information indeed worsens performance. To illuminate those results, we note that combining object-environment contacts in a PoE entails the risks of obtaining small numbers. In the extreme case, this may result in numerical instability or predictions of barely diminishable likelihoods. When few kernels support the density, likelihoods are more prone to be small than in the case of abundant training data. Our findings that RO+3OE and RO+5OE improve with the size of the training set are consistent with those observations. Whether this explains all of the observed differences, and whether their accuracy will continue to increase at a similar rate with the number of training pushes, are questions which we aim to address in future work by utilising our full training set.

In the light of this discussion, a particularly interesting result of our experiments is that additional contact information worsens performance for the cylinder in simulation, and for the large and small boxes in experiments with the real robot. Notably, in all cases the objects exhibit surface features which are comparably dissimilar to those seen during training time: the cylinder because of its rounded shape, the test boxes because real point clouds are noisier than the simulated ones seen during training. As motion candidates’ likelihoods are evaluated conditional on surface features, this may exacerbate the aforementioned challenge of small numbers. At this point, we emphasise that this paper represents a first investigation into the proposed approach, and rigorously scrutinising the posed questions will require further experiments which we will conduct in future work. In particular, we aim to train models on real point cloud data to evaluate whether they fare better in predicting real pushes from noisy point clouds.

With regard to limitations of our approach, we already noted that our experiments focus on planar sliding, and have primarily been conducted in simulation. Complementing this with further experiments on real robotic platforms, and with objects that are free to topple and roll, are prioritised items for future work. Alleviating the requirement of learning a separate motion model for each action would furthermore greatly increase the range of applications for our approach. Evaluating the predictive performance of alternative surface features, such as surface material, is another interesting route for further investigations. So is using information obtained from force-torque sensors for prediction. Furthermore, we are interested in learning dynamic contact models, i.e. predicting how contacts will change over time. Finally, we aim to use the presented feature-based predictors as transition models for reinforcement learning.