Combining Physical Simulators and Object-Based Networks for Control

TL;DR

This paper introduces a hybrid physics-neural network model called SAIN that improves the accuracy and data efficiency of robot control involving complex contact dynamics, outperforming existing models in simulation and real-world tests.

Contribution

The paper presents a novel hybrid dynamics model combining physics engines with object-based neural networks for improved robot control.

Findings

Better accuracy in modeling object interactions.

Enhanced data efficiency compared to purely analytical or data-driven models.

Successful generalization to new environments with different objects.

Abstract

Physics engines play an important role in robot planning and control; however, many real-world control problems involve complex contact dynamics that cannot be characterized analytically. Most physics engines therefore employ . approximations that lead to a loss in precision. In this paper, we propose a hybrid dynamics model, simulator-augmented interaction networks (SAIN), combining a physics engine with an object-based neural network for dynamics modeling. Compared with existing models that are purely analytical or purely data-driven, our hybrid model captures the dynamics of interacting objects in a more accurate and data-efficient manner.Experiments both in simulation and on a real robot suggest that it also leads to better performance when used in complex control tasks. Finally, we show that our model generalizes to novel environments with varying object shapes and materials.

Figures

Tables

Table 1. TABLE I: Errors on dynamics prediction in direct-force simulation setup . SAIN achieves the best performance in both position and rotation estimation, compared with methods that rely on physics engines or neural nets alone. The first two metrics are the average Euclidean distance between the predicted and the ground truth object reported as a percentage relative to the initial pose (trans) and as absolute values (pos) in millimeters. The third is the average error of object rotation (rot) in degree.

Models	Error on Object 1/Object 2
	trans (%)	pos (mm)	rot (deg)
Physics	2.82/2.31	6.57/6.05	0.91/0.45
IN	2.09/1.47	5.61/3.79	0.68/0.26
SAIN (ours)	1.62/1.38	4.38/3.34	0.38/0.2

Table 2. TABLE II: Generalization to 3 objects in direct-force simulation setup . SAIN achieves the best generalization in both position and rotation estimation.

Models	Error on Object 1/2/3
	trans (%)	pos (mm)	rot (deg)
Physics	2.79/2.34/2.38	6.53/6.11/6.21	0.89/0.48/0.49
IN	2.12/1.63/1.67	5.68/4.41/4.52	0.70/0.34/0.38
SAIN (ours)	1.68/1.52/1.61	4.54/3.97/4.34	0.41/0.25/0.32

Table 3. TABLE III: Errors on dynamics prediction in robot control simulation setup . SAIN achieves the best performance in both position and rotation estimation.

Models	Error on Object 1/2
	trans (%)	pos (mm)	rot (deg)
Physics	2.52/2.19	6.27/5.81	0.85/0.29
IN	2.13/1.59	5.76/3.84	0.72/0.28
SAIN (ours)	1.82/1.50	4.66/3.47	0.40/0.21

Table 4. TABLE IV: Errors on dynamics prediction in the real world. SAIN obtains the best performance in both position and rotation estimation. Its performance gets further improved after fine-tuning on real data.

Models	Fine-tuning	Error on Object 1/2
		trans (%)	pos (mm)	rot (deg)
Physics	N/A	0.87/1.91	3.06/6.41	0.32/0.17
IN	No	0.86/1.84	2.96/5.75	0.96/0.32
SAIN (ours)	No	0.69/1.06	2.38/3.52	0.43/0.18
IN	Yes	0.63/0.61	2.23/2.05	0.41/0.19
SAIN (ours)	Yes	0.42/0.43	1.50/1.52	0.34/0.17

Equations

$$f_{\theta }^R({\bar{s}}_{t+1},{\hat{s}}_t,a_t)={\hat{s}}_{t+1}\approx s_{t+1},$$

$$f_p({\bar{s}}_t,a_t)={\bar{s}}_{t+1},{\bar{s}}_0={\hat{s}}_0=s_0.$$

$${\theta }^{\ast }=\underset{\theta }{\mathrm{argmin}}\sum _{t=0}^{T-1}\Vert {\hat{s}}_{t+1}-s_{t+1}{\Vert }_2^2+\lambda \Vert \theta {\Vert }_2^2,$$

$$e_t^i$$

$${\hat{v}}_{t+1}^i$$

$${\hat{p}}_{t+1}^i$$

$${\hat{o}}_{t+1}^i$$

$${\bar{s}}_{t+1}$$

$$e_t^i$$

$${\hat{v}}_{t+1}^i$$

$${\hat{p}}_{t+1}^i$$

$${\hat{o}}_{t+1}^i$$

$$\begin{gathered} \frac{1}{T}\sum _{i=1}^2\sum _{t=0}^{T-1}\Vert p_t^i-{\hat{p}}_t^i{\Vert }_2^2+\Vert v_t^i-{\hat{v}}_t^i{\Vert }_2^2+ \\ \Vert \sin (r_t^i)-\sin ({\hat{r}}_t^i){\Vert }_2^2+\Vert \cos (r_t^i)-\cos ({\hat{r}}_t^i){\Vert }_2^2. \end{gathered}$$

Full text

I Introduction

Physics engines are important for planning and control in robotics. To plan for a task, a robot may use a physics engine to simulate the effects of different actions on the environment and then select a sequence of them to reach a desired goal state. The utility of the resulting action sequence depends on the accuracy of the physics engine’s predictions, so a high-fidelity physics engine is an important component in robot planning. Most physics engines used in robotics (such as Mujoco [1] and Bullet [2]) use approximate contact models, and recent studies [3, 4, 5] have demonstrated discrepancies between their predictions and real-world data. These mismatches make contact-rich tasks hard to solve using these physics engines.

One way to increase the robustness of controllers and policies resulting from physics engines is to add perturbations to parameters that are difficult to estimate accurately (e.g., frictional variation as a function of position [4]). This approach leads to an ensemble of simulated predictions that covers a range of possible outcomes. Using the ensemble allows to take more conservative actions and increases robustness, but does not address the limitation of using learned, approximate models [6, 7].

To correct for model errors due to approximations, we learn a residual model between real-world measurements and a physics engine’s predictions. Combining the physics engine and residual model yields a data-augmented physics engine. This strategy is effective because learning a residual error of a reasonable approximation (here from a physics engine) is easier and more sample efficient than learning from scratch. This approach has been shown to be more data efficient, have better generalization capabilities, and outperform its purely analytical or data-driven counterparts [8, 9, 10, 11].

Most residual-based approaches assume a fixed number of objects in the world states. This means they cannot be applied to states with a varied number of objects or generalize what they learn for one object to other similar ones. This problem has been addressed by approaches that use graph-structured network models, such as interaction networks [12] and neural physics engines [13]. These methods are effective at generalizing over objects, modeling interactions, and handling variable numbers of objects. However, as they are purely data-driven, in practice they require a large number of training examples to arrive at a good model.

In this paper, we propose simulator-augmented interaction networks (SAIN), incorporating interaction networks into a physical simulator for complex, real-world control problems. Specifically, we show:

• •

Sample-efficient residual learning and improved prediction accuracy relative to the physics engine,

• •

Accurate predictions for the dynamics and interaction of novel arrangements and numbers of objects, and the

• •

Utility of the learned residual model for control in highly underactuated planar pushing tasks.

We demonstrate SAIN’s performance on the experimental setup depicted in Fig. 1. Here, the robot’s objective is to guide the second disk to a goal by pushing on the first. This task is challenging due to the presence of multiple complex frictional interactions and underactuation [14]. We demonstrate the step-by-step deployment of SAIN, from training in simulation to augmentation with real-world data, and finally control.

II Related Work

II-A Learning Contact Dynamics

In the field of contact dynamics, researchers have looked towards data-driven techniques to complement analytical models and/or directly learn dynamics. For example, Byravan and Fox [15] designed neural nets to predict rigid-body motions for planar pushing. Their approach does not exploit explicit physical knowledge. Kloss et al. [11] used neural net predictions as input to an analytical model; the output of the analytical model is used as the prediction. Here, the neural network learns to maximize the analytical model’s performance. Fazeli et al. [8] also studied learning a residual model for predicting planar impacts. Zhou et al. [16] employed a data-efficient algorithm to capture the frictional interaction between an object and a support surface. They later extended it for simulating parametric variability in planar pushing and grasping [17].

The paper closest to ours is that from Ajay et al. [9], where they used the analytical model as an approximation to the push outcomes, and learned a residual neural model that makes corrections to its output. In contrast, our paper makes two key innovations: first, instead of using a feedforward network to model the dynamics of a single object, we employ an object-based network to learn residuals. Object-based networks build upon explicit object representations and learn how they interact; this enables capturing multi-object interactions. Second, we demonstrate that such a hybrid dynamics model can be used for control tasks both in simulation and on a real robot.

II-B Differentiable Physical Simulators

There has been an increasing interest in building differentiable physics simulators [18]. For example, Degrave et al. [19] proposed to directly solve differentiable equations. Such systems have been deployed for manipulation and planning for tool use [20]. Battaglia et al. [12] and Chang et al. [13] have both studied learning object-based, differentiable neural simulators. Their systems explicitly model the state of each object and learn to predict future states based on object interactions. In this paper, we combine such a learned object-based simulator with a physics engine for better prediction and for controlling real-world objects.

II-C Control with a Learned Simulator

Recent papers have explored model-predictive control with deep networks [21, 22, 23, 24, 25]. These approaches learn an abstract-state transition function, not an explicit model of the environment [26, 27]. Eventually, they apply the learned value function or model to guide policy network training. In contrast, we employ an object-based physical simulator that takes raw object states (e.g., velocity, position) as input. Hogan et al. [28] also learned a residual model with an analytical model for model-predictive control, but their learned model is a task-specific Gaussian Process, while our model has the ability to generalize to new object shapes and materials.

A few papers have exploited the power of interaction networks for planning and control, mostly using interaction networks to help training policy networks via imagination—rolling out approximate predictions [29, 30, 31]. In contrast, we use interaction networks as a learned dynamics simulator, combine it with a physics engine, and directly search for actions in real-world control problems. Recently, Sanchez-Gonzalez et al. [32] also used interaction networks in control, though their model does not take into account explicit physical knowledge, and its performance is only demonstrated in simulation.

III Method

In this section, we describe SAIN’s formulation and components. We also present our Model Predictive Controller (MPC) which uses SAIN to perform the pushing task.

III-A Formulation

Let $S$ be the state space and $A$ be the action space. A dynamics model is a function $f:S\times A\rightarrow S$ that predicts the next state given the current action and state: $f(s,a)\approx s^{\prime},\;\;\text{for all}\;\;s,s^{\prime}\in S,\;\;a\in A$.

There are two general types of dynamics models: analytical (Fig. 2a) and data-driven (Fig. 2b). Our goal is to learn a hybrid dynamics model that combines the two (Fig. 2c). Here, conditioned on the state-action pair, the data-driven model learns the discrepancy between analytical model predictions and real-world data (i.e. the residual). Specifically, let $f_{r}$ represent the hybrid dynamics model, $f_{p}$ represent the physics engine, and $f_{\theta}$ represent the residual component. We have $f_{r}(s,a)=f_{\theta}(f_{p}(s,a),s,a)\approx s^{\prime}$. Intuitively, the residual model refines the physics engine’s guess using the current state and action.

For long-term prediction, let $f^{R}_{\theta}:S\times S\times A\rightarrow S$ represent the recurrent hybrid dynamics model (Fig. 2d). If $s_{0}$ is the initial state, $a_{t}$ the action at time $t$, $\bar{s}_{t}$ the prediction by the physics engine $f_{p}$ at time $t$ and $\hat{s}_{t}$ the prediction at time $t$, then

[TABLE]

For training, we collect observational data $\{(s_{t},a_{t},s_{t+1})\}_{t=0}^{T-1}$ and then solve the following optimization problem:

[TABLE]

where $\lambda$ is the weight for the regularization term.

In this study, we choose to use a recurrent parametric model over a non-recurrent representation for two reasons. First, non-recurrent models are trained on observation data to make single-step predictions. Consequently, errors in prediction compound over a sequence of steps. Second, since these models recursively use their own predictions, the input data given during the simulation phase will have a different distribution than the input data during the training phase. This creates a data distribution mismatch between the training and test phases.

III-B Interaction Networks

We use interaction networks [12] as the data-driven model for multi-object interaction. An interaction network consists of 2 neural nets: $f_{\text{dyn}}$ and $f_{\text{rel}}$. The $f_{\text{rel}}$ network calculates pairwise forces between objects and the $f_{\text{dyn}}$ network calculates the next state of an object, based on the states of the objects it is interacting with and the nature of the interactions.

The original version of interaction networks was trained to make a single-step prediction; for improved accuracy, we extend them to make multi-step predictions. Let $s_{t}=\{o^{1}_{t},o^{2}_{t},\ldots,o^{n}_{t}\}$ be the state at time $t$, where $o^{i}_{t}$ is the state for object $i$ at time $t$. Similarly, let $\hat{s}_{t}=\{\hat{o}^{1}_{t},\hat{o}^{2}_{t},\ldots,\hat{o}^{n}_{t}\}$ be the predicted state at time $t$ where $o^{i}_{t}$ is the predicted state for object $i$ at time $t$. In our work, $o^{i}_{t}=[p^{i}_{t},v^{i}_{t},m^{i},r^{i}]$ where $p^{i}_{t}$ is the pose of object $i$ at time step $t$, $v^{i}_{t}$ the velocity of object $i$ at time step $t$, $m^{i}$ the mass of object i and $r^{i}$ the radius of object i. Similarly, $\hat{o}^{i}_{t}=[\hat{p}^{i}_{t},\hat{v}^{i}_{t},m^{i},r^{i}]$ where $\hat{p}^{i}_{t}$ is the predicted pose of object $i$ at time step $t$ and $\hat{v}^{i}_{t}$ the predicted velocity of object $i$ at time step $t$. Note that we do not predict any changes to static object properties such as mass and radius. Also, we note that while $s_{t}$ is a set of objects, the state of any individual object, $o^{i}_{t}$, is a vector. Now, let $a^{i}_{t}$ be the action applied to object $i$ at time $t$. The equations for the interaction network are:

[TABLE]

III-C Simulator-Augmented Interaction Networks (SAIN)

A simulator-augmented interaction network extends an interaction network, where $f_{\text{dyn}}$ and $f_{\text{rel}}$ now take in the prediction of a physics engine, $f_{p}$. We now learn the residual between the physics engine and the real world. Let $\bar{s}_{t}=\{\bar{o}^{1}_{t},\bar{o}^{2}_{t},\ldots,\bar{o}^{n}_{t}\}$ be the state at time $t$ and $\bar{o}^{i}_{t}$ be the state for object $i$ at time $t$ predicted by the physics engine. The equations for SAIN are

[TABLE]

These equations describe a single-step prediction. For multi-step prediction, we use the same equations by providing the true state $s_{0}$ at $t=0$ and predicted state $\hat{s}_{t}$ at $t>0$ as input.

III-D Control Algorithm

Our action space has two free parameters: the point where the robot contacts the first disk and the direction of the push. In our experiments, a successful execution requires searching for a trajectory of about 50 actions. Due to the size of the search space, we use an approximate receding horizon control algorithm with our dynamics model. The search algorithm maintains a priority queue of action sequences based on the heuristic below. For each expansion, let $s_{t}$ be the current state and $\hat{s}_{t+T}(a_{t},\dots,a_{t+T-1})$ be the predicted state after $T$ steps with actions $a_{t},\dots,a_{t+T-1}$. Let $s_{g}$ be the goal state. We choose the control strategy $a_{t}$ that minimizes the the cost function $||\hat{s}_{t+T}(s_{t},a_{t},\dots,a_{t+T-1})-s_{g}||_{2}$ and insert the new action sequence into the queue.

IV Experiments

We demonstrate SAIN on a challenging planar manipulation task both in simulation and in the real-world. We further evaluate how our model generalizes to handle control tasks that involve objects of new materials and shapes.

IV-A Task

In this manipulation task, we are given two disks with different mass and radii. Our goal is to guide the second disk to a target location, but are constrained to push only the first disk. Here, a point $s$ in the state space is factored into a set of two object states, $s=\{o_{1},o_{2}\}$, where each $o_{i}$ is an element of object state space $O$. The object state includes the mass, 2D position, rotation, velocity, and radius of the disk.

Targets locations are generated at random and divide into two categories: easy and hard. A target location is produced by first sampling an angle $\alpha$ from an interval $U$, then choosing the goal location to be at distance of three times the radius of second disk and at an angle of $\alpha$ with respect to the second disk. In easy pushes, the interval $U$ is $[-\frac{\pi}{6},\frac{\pi}{6}]$. In hard pushes, the interval $U$ is $[-\frac{\pi}{3},-\frac{\pi}{6}]\cup[\frac{\pi}{6},\frac{\pi}{3}]$. A push is considered a success if the distance between the goal location and the pose of the center of mass of the second disk is within $\frac{1}{10}^{th}$ the radius of second disk.

IV-B Simulation Setup

We use the Bullet physics engine [33] for simulation. For each trajectory, we vary the coefficient of friction between the surface and the disks, the mass of the disks and their radius. The coefficient of friction is sampled from $\text{Uniform}(0.05,0.25)$. The mass is sampled from Uniform(0.85kg, 1.15kg) and the radius is sampled from $\text{Uniform}(0.05\text{m},0.06\text{m})$. We always fix the initial position of the first disk to the origin. The other disks are placed in front of the first disk at an angle, randomly sampled from $\text{Uniform}(-\frac{\pi}{3},\frac{\pi}{3})$, and just touches it. We ensure that disks don’t overlap each other. The pusher is placed at back of the first disk at an angle randomly sampled from $\text{Uniform}(-\frac{\pi}{3},\frac{\pi}{3})$, and just touches it. Then the pusher makes a straight line push at an angle, randomly sampled from $\text{Uniform}(-\frac{\pi}{6},\frac{\pi}{6})$, for 2s and covers a distance of about 1cm. We experiment with two different simulation setups: (1) direct-force simulation setup in which we control pusher with external force and (2) robot control simulation setup in which we control the pusher using position control. We use the first setup to show the benefits of SAIN over other models. But in our real world setup, we control the pusher using position-based control. So, we have designed a second simulation setup which matches the real robot and use it to collect pre-training data.

For the direct-force simulation setup, we collect $4500$ pushes with 2 disks for our training set, $500$ pushes with 2 disks and $500$ pushes with 3 disks for our test set. For the robot control simulation setup, we collect $4500$ pushes with 2 disks for our training set and $500$ pushes with 2 disks for our test set.

IV-C Model and Implementation Details

We compare two models for simulation and control: the original interaction networks (IN) and our simulator-augmented interaction networks (SAIN). They share the same architecture. Each consists of two separate neural networks: $f_{\text{dyn}}$ and $f_{\text{rel}}$. Both $f_{\text{dyn}}$ and $f_{\text{rel}}$ have four linear layers with hidden sizes of 128, 64, 32 and 16 respectively. The first three linear layers are followed by a ReLU activation.

Training interaction networks in simulation is straightforward. It is more involved for SAIN, which learns a correction over the Bullet physics engine, so the problem of training “in simulation” is ill-posed. To address this problem, we fix the physics engine in SAIN with mass and radius of disks equaling that of disks in the real world. We also fix the coefficient of friction in the physics engine to an estimated mean of the coefficient of friction of the real world surface across space. The training data instead contain varied mass and radius of both disks, and varied the coefficient of friction between the disks and the surface, and the model is trained to learn the residual.

We use ADAM [34] for optimization with a starting learning rate of 0.001. We decrease it by 0.5 every 2,500 iterations. We train these models for 10,000 iterations with a batch size of 100. Let the predicted 2D position, rotation, and velocity at time $t$ of disk $i$ be $\hat{p}^{i}_{t}$, $\hat{r}^{i}_{t}$ and $\hat{v}^{i}_{t}$, respectively, and the corresponding true values be $p^{i}_{t}$, $r^{i}_{t}$, and $v^{i}_{t}$. Let $T$ be the length of all trajectories. The training loss function for a single trajectory is

[TABLE]

During training, we use a batch of trajectories and take a mean over the loss of those trajectories. We also use $l_{2}$ regularization with $10^{-3}$ as regularization constant.

In practice, we ask the models to predict the change in object states (relative values) rather than the absolute values. This enables them to generalize to arbitrary starting positions without overfitting.

IV-D Search Algorithm

As mentioned in Sec. III-D, an action is defined by the initial position of the pusher and the angle of the push, $\alpha$, with respect to the first disk. After these parameters have been selected, the pusher starts at the initial position and moves at an angle of $\alpha$ with respect to the first disk for $10mm$. We discretize our action space as follows. For selecting $\alpha$, we divide the interval $[\frac{-\pi}{6},\frac{\pi}{6}]$ into six bins and choose their midpoints. For selecting the initial position of the pusher, we choose an angle $\theta$ and place the pusher at edge of first disk at an angle $\theta$ such that the pusher touches the first disk. For selecting $\theta$, we divide the interval $[\frac{-\pi}{3},\frac{\pi}{3}]$ into 12 bins and choose midpoint of one of these bins. Therefore, our action space consists of 72 discretized actions for each time step. We maintain a priority queue of action sequences based on heuristic $h(\hat{p}^{1},\hat{p}^{2},p_{g})$ where $\hat{p}^{i}$ is the predicted 2D position of disk $i$ and $p_{g}$ is the 2D position of goal. $h(\hat{p}^{1},\hat{p}^{2},p_{g})$ is sum of $\|\hat{p}^{2}-p_{g}\|$ and cosine distance between $p_{g}-\hat{p}^{1}$ and $\hat{p}^{2}-\hat{p}^{1}$. The cosine distance serves as a regularization cost to encourage the center of both disks and the goal to stay in a straight line. To prevent the priority queue from blowing up, we do receding horizon greedy search with an initial horizon of 2, and increase it to 3 when the distance between the second disk and goal is less than $10mm$.

IV-E Prediction and Control Results in Simulation

The forward multi-step prediction errors of both interaction networks and SAIN for direct-force simulation setup with 2 disks and 3 disks are reported in Table I and Table II. Note that errors on different objects are separated by $/$ in all the tables. The training data for this setup consist of pushes with only 2 disks. The forward multi-step prediction errors of both interaction networks and SAIN for robot control simulation setup are reported in Table III. Given an initial state and a sequence of actions, the models do forward prediction for the next 200 timesteps, where each time-step is 1/240s. We see that SAIN outperforms interaction networks in both setups. We also list the results of the fixed physics engine used for training SAIN for reference.

We have also evaluated IN and SAIN on control tasks in simulation. We test each model on 25 easy and 25 hard pushes. For these pushes, we set the mass of two disks to 0.9kg and 1kg and their radius to 54mm and 59mm, making them differ them from those used in the internal physics engine of SAIN. This mimics real-world environments, where objects’ size and shape cannot be precisely estimated, and ensures SAIN cannot cheat by just querying the internal simulator. Fig. 3 shows SAIN performs better than IN. This suggests learning the residual not only helps to do better forward prediction, but also benefits control.

IV-F Real-World Robot Setup

We now test our models on a real robot. The setup used for the real experiments is based on the system from the MIT Push dataset [4]. The pusher is a cylinder of radius 4.8mm attached to the last joint of a ABB IRB 120 robot. The position of the pusher is recorded using the robot kinematics. The two disks being pushed are made of stainless steel, have radius of 52.5mm and 58mm, and weight 0.896kg and 1.1kg. During the experiments, the smallest disk is the one pushed directly by the pusher. The position of both disks is tracked using a Vicon system of four cameras so that the disks’ positions are highly accurate. Finally, the surface where the objects lay is made of ABS (Acrylonitrile Butadiene Styrene), whose coefficient of friction is around 0.15. Each push is done at 50mm/s and spans 10mm. We collect 1,500 pushes out of which 1,200 are used for training and 300 for testing.

We evaluate two versions of interaction networks and SAIN. The first is an off-the-shelf version purely trained on synthetic data; the second is one trained on simulated data and later fine-tuned on real data. This helps us understand whether these models can exploit real data to adapt to new environments.

IV-G Results on Real-World Data

Results of forward simulation are shown in Table IV. SAIN outperforms IN on real data. While both models benefit from fine-tuning, SAIN achieves the best performance. This suggests residual learning also generalizes to real data well. All models achieve a lower error on real data than in simulation; this is because simulated data have a significant amount of noise to make the problem more challenging.

We then evaluate SAIN (both with and without fine-tuning) for control, on 25 easy and 25 hard pushes. The results are shown in Fig. 4. The model without fine-tuning achieves 100% success rate on easy pushes and 68% on hard pushes. As shown in the rightmost columns of Fig. 4(a), it sometimes pushes the object too far and gets stuck in a local minimum. After fine-tuning, the model works well on both easy pushes (100%) and hard pushes (96%) (Fig. 4(b)).

While objects of different shapes and materials have different dynamics, the gap between their dynamics in simulation and in the real world might share similar patterns. This is the intuition behind the observation that residual learning allows easier generalization to novel scenarios. Ajay [9] validated this for forward prediction. Here, we evaluate how our fine-tuned SAIN generalizes for control. We test our model on 25 hard pushes with a different surface (plywood, where the original surface is ABS), using the original disks. Our framework achieves successes in 92% of the pushes, where Fig. 5(a) shows qualitative results. We’ve also evaluated our model on another 25 hard pushes, where it pushes the large disk (58mm) to direct the small one (52.5mm). Our framework achieves successes in 88% of the pushes. Fig. 5(b) shows qualitative results. These results suggest that SAIN can generalize to solve control tasks with new object shapes and materials.

V Conclusion

We have proposed a hybrid dynamics model, simulator-augmented interaction networks (SAIN), combining a physical simulator with a learned, object-centered neural network. Our underlying philosophy is to first use analytical models to model real-world processes as much as possible, and learn the remaining residuals. Learned residual models are specific to the real-world scenario for which data is collected, and adapt the model accordingly. The combined physics engine and residual model requires little need for domain specific knowledge or hand-crafting and can generalize well to unseen situations. We have demonstrated SAIN’s efficacy when applied to a challenging control problem in both simulation and the real world. Our model also generalizes to setups where object shape and material vary and has potential applications in control tasks that involve complex contact dynamics.

**Acknoledgements. ** This work is supported by NSF #1420316, #1523767, and #1723381, AFOSR grant FA9550-17-1-0165, ONR MURI N00014-16-1-2007, Honda Research, Facebook, and Draper Laboratory.