Object Reconfiguration with Simulation-Derived Feasible Actions

TL;DR

This paper introduces a simulation-based planning approach for 3D object reconfiguration that automatically discovers feasible actions through physics simulation, eliminating manual encoding and improving planning efficiency.

Contribution

It presents a novel method embedding a physics simulator within a motion planner to implicitly determine valid actions, streamlining the reconfiguration process.

Findings

Successfully generates physically valid rearrangement trajectories

Reduces manual effort in specifying action semantics

Works across diverse 3D object reconfiguration scenarios

Abstract

3D object reconfiguration encompasses common robot manipulation tasks in which a set of objects must be moved through a series of physically feasible state changes into a desired final configuration. Object reconfiguration is challenging to solve in general, as it requires efficient reasoning about environment physics that determine action validity. This information is typically manually encoded in an explicit transition system. Constructing these explicit encodings is tedious and error-prone, and is often a bottleneck for planner use. In this work, we explore embedding a physics simulator within a motion planner to implicitly discover and specify the valid actions from any state, removing the need for manual specification of action semantics. Our experiments demonstrate that the resulting simulation-based planner can effectively produce physically valid rearrangement trajectories for a range of 3D object reconfiguration problems without requiring more than an environment description and start and goal arrangements.

Tables

Table 1. TABLE I: Run-time metrics of SAST across the test problems. Success rate refers to the proportion of runs where SAST finds a solution within the given time limit. For successful runs, Solve Time refers to the time taken to find a solution; Solution Length refers to the solution length, in terms of the number of objects moved; Num. Nodes refer to the total number of tree vertices created by SAST during search. Mean values are shown, with standard error shown in parentheses.

	Reverse	Transform	Rotate
Success Rate	0.96 (0.03)	0.96 (0.03)	1.00 (0)
Solve Time (s)	16.5 (0.9)	55.0 (5.6)	61.5 (6.6)
Solution Length	25.6 (0.7)	23.7 (0.8)	15.2 (0.5)
Num. Nodes	44.1 (2.4)	86.2 (9.6)	57.4 (6.0)

Table 2. TABLE II: Solution simplification results. Simplify Time is the time taken for the simplification procedure to terminate. Simplified Solution Length is the length of the simplified solutions, in terms of the number of objects moved. Improvement is the percentage of the original solution length that simplification eliminated. Mean values are shown, with standard error in parentheses.

	Reverse	Transform	Rotate
Simplify Time (s)	26.5 (1.4)	41.8 (2.7)	28.0 (2.0)
Simplified Solution Length	12.1 (0.1)	12.0 (0.1)	8.0 (0.1)
Improvement (%)	51.5 (1.2)	46.8 (1.7)	44.9 (1.7)

Table 3. TABLE III: Solution feasibility with and without low-level motion checking during the arrangement tree search. We show the mean proportion of solutions with feasible low-level motions; the standard error is in parentheses.

	Reverse	Transform	Rotate
w/ Motion Checks	1.00 (0)	1.00 (0)	1.00 (0)
w/o Motion Checks	0.60 (0.07)	0.42 (0.07)	0.56 (0.07)

Equations

$$\text{TRANSIT(}{\alpha }_{\mathrm{start}},q_{\mathrm{start}},q_1\text{)}$$

$$\to$$

Full text

I Introduction

Robot manipulation planning is primarily a problem of finding a sequence of valid actions that move a set of target objects to a given goal configuration. Actions are valid if they respect the problem’s constraints, which may be task-specific (e.g., keeping a glass of water upright or maintaining dynamic stability of a stack of objects) or implicit and arising from the environment (e.g., avoiding collisions).

Most approaches to manipulation planning (e.g., [1, 2, 3, 4, 5, 6, 7]) rely on explicitly specified problem constraints through formal languages like Linear Temporal Logic (LTL) [8] and the Planning Domain Definition Language (PDDL) [9, 10] or through natural language [11]. These manually created specifications identify both the valid, meaningful subsets of the state space and the valid transitions between these subsets. The resulting transition system guides a search for a sequence of valid actions that perform the given task when executed by the robot.

However, such specifications are onerous and error-prone to construct [12], and may not capture the full set of possible actions. They must not only define the valid dynamics for a problem’s environment, but also be rich enough to describe a wide range of problems and goals. Furthermore, problem specifications are not unique and the choice of specification can impact planning performance [13, 14, 15, 16]. Conversely, some manipulation planners forgo full generality for simplified problem specification and improved performance [1, 2]. These planners tend to be restricted to planar tabletop object rearrangement or similar problems [17].

We propose a middle ground: planners that can solve a broad set of classes of manipulation planning problems with no more problem specification than a typical low-level motion planning problem. Our insight is that the necessary transition systems can be implicitly defined through an environment simulator, reducing the manual specification burden.

This paper contributes a novel perspective on manipulation problem specification and manipulation planner design. This perspective centers around embedding an environment simulation in a sampling-based planning algorithm as an implicit specification of a problem’s valid transition system. In support of these ideas, we contribute

1. the arrangement space, a novel planning space representing object arrangements and dynamically discovered low-level robot motions moving between them,

2. Stable Arrangement Search Trees(SAST), an arrangement-space planner using embedded environment simulators to discover valid action sequences and the associated low-level motions (Sec. IV-A), and

3. a procedure to simplify the solutions found in the arrangement space.

Concretely, we investigate the use of an embedded off-the-shelf physics simulator [18, 19] in SAST to efficiently find statically stable, collision-free states for 3D object reconfiguration problems without manually specifying action semantics. This setting is a specific instance of the broader manipulation planning paradigm that we propose.

We demonstrate that our proposed framework can efficiently solve 3D object reconfiguration problems with physical constraints without requiring more than an environment description and start/goal configurations to specify a problem. These results argue for the viability of a family of planners based upon implicitly simulator-defined transition systems.

II Background and Related Work

This paper proposes a novel perspective on planning with embedded simulators that combines and extends earlier uses of simulation in robot planning and control (Sec. II-A). SAST, an example of this perspective in practice, is an efficient sampling-based planning algorithm (Sec. II-B) that builds upon ideas from tabletop rearrangement planning and integrated task and motion planning (Sec. II-C) to solve dynamically-constrained object reconfiguration problems.

II-A Simulation in robotics

Simulation is widely used in robot control and learning. Model-predictive control (MPC) simulates control trajectories forward through time to choose optimal inputs [20]. Recent work improves MPC performance by integrating differentiable physics simulation [21, 22]. Efficient simulation for training [23, 24] has been core to learning-based methods for robot control [25, 26, 27], despite the challenge of translating controllers from simulation to the real world [28].

However, as noted by [29], simulation for planning is under-studied. Prior work combining manipulation planning and simulation restricts to specific motion primitives [30, 31] or 2D settings [32]; we operate in a 3D workspace and dynamically discover the valid motions available to the robot via a bi-level search over object arrangements and robot motions.  [29] studied efficient planning with simulators by selectively simulating actions.  [33] improved the long-horizon planning efficiency of a Monte-Carlo Tree Search-based planner by integrating parallel simulation.  [34, 35] use simulators with different precisions and interleaved simulation and execution to improve manipulation planning performance and robustness. These approaches complement SAST. We propose that an embedded simulator can be an effective implicit specification of a problem’s constraints.

II-B Sampling-based motion planning

Sampling-based motion planning (SBMP) is a family of efficient robot motion planning techniques based upon constructing approximations of a robot’s high-dimensional configuration space [36] from sampled configurations [37, 38, 39, 40]. Most SBMP algorithms operate by building up a graph [37] or tree [38] of valid configurations connected by short valid motions. RRTConnect [39] is among the fastest SBMP algorithms for many problems, due to its technique of growing two trees of valid motions, one from each of the start and the goal, toward each other using a local “extension” planner to control the trees’ growth.

SAST adapts the high-level planning loop of RRTConnect to search an expansive space of stable object arrangements (Sec. IV-A) using a simulation-based extension planner (Sec. IV-C).

II-C Object reconfiguration

Object reconfiguration has been studied in contexts including manipulation planning [4, 5, 41, 42], rearrangement planning [43, 1, 2, 44], and integrated task and motion planning (TAMP) [10]. These approaches span an axis ranging from problem specialization (i.e., planar rearrangement planners [1, 2, 43]) to relative generality (i.e., full TAMP solving [3, 6, 7]). This axis also corresponds to the relative specification effort for each planner: a measure of the work a user must do to provide a given planner with the information it needs to operate. Planar rearrangement planners typically only specify the desired object arrangement (as well as the environment geometry), and exploit their assumption of planar problems to find solutions faster. TAMP solvers also rely on symbolic action specifications, mechanisms for discovering states that satisfy action preconditions, and more (e.g., explicit problem constraint specifications) [10]. We strike a balance: simulators still require manual effort to create, but are more broadly reusable across problems and domains than the specifications and samplers required by most TAMP solvers. Simulators can also implicitly encode a more general set of constraints than most rearrangement solvers, allowing for richer problems. Further, as progress in learning problem-specific dynamics models advances [45, 46, 47, 48], the effort required to create simulators for planning will decrease.

SAST, like [2], relies on an arrangement-aware extension primitive to find valid action sequences.  [32] also proposes a rearrangement planner incorporating a simplified 2D physics model to evaluate a predefined set of rearrangement actions. Similarly, [49] explores kinodynamic planning for planar rearrangement with a focus on reacting to unexpected events during rearrangement plan execution, and using a heuristic-based task specification. SAST uses full 3D physics, does not predefine motion primitives, and models dynamic constraints such as stability. In future work, synergistically combining SAST with the techniques of [32, 49] could allow SAST to use richer non-prehensile motions for manipulating objects.

III Problem Formulation

We demonstrate implicit constraint definition via embedded simulation in a specific application: 3D object reconfiguration with stability constraints, using pick-and-place actions.

Consider a 3D workspace containing movable rigid-body objects, $o\in\mathcal{O}$, and a known set of posed static obstacle geometries. Objects have known 3D geometries and poses in $\operatorname{SE}(3)$. An arrangement assigns a pose to each object:

Definition 1: Arrangement

An arrangement, $\alpha$, prescribes a pose, $\operatorname{\alpha}[o]\in\operatorname{SE}(3)$, to each object in the workspace. Denote the arrangement space, the set of all arrangements, as $\mathcal{A}$, and let $\alpha\setminus o$ be arrangement $\alpha$ with object $o\in\mathcal{O}$ removed from consideration.

Arrangements may be valid or invalid. Valid arrangements are those that are both collision-free and statically stable.

Definition 2: Valid arrangement

Let $\operatorname{\texttt{CollisionFree}}(\cdot)$ be a collision test for arrangements, such that $\operatorname{\texttt{CollisionFree}}(\alpha)=\texttt{True}$ if $\alpha$ has no objects in collision. Similarly, let $\operatorname{\texttt{Stable}}(\alpha)$ be a static stability test for arrangements, such that $\operatorname{\texttt{Stable}}(\alpha)=\texttt{True}$ if $\alpha$ is statically stable after a fixed duration.

An arrangement, $\alpha\in\mathcal{A}$, is valid if and only if $\operatorname{\texttt{CollisionFree}}(\alpha)=\texttt{True}$ and $\operatorname{\texttt{Stable}}(\alpha)=\texttt{True}$. We evaluate $\operatorname{\texttt{CollisionFree}}(\cdot)$ via a physics simulator’s collision checker. We check $\operatorname{\texttt{Stable}}(\cdot)$ by stepping the simulator for a fixed number of time steps and verifying that all objects’ displacements remain below a small heuristic threshold.

Let $r$ be a robot arm with a static base and joint configuration space $\mathcal{Q}$ [36]. The arm is capable of two classes of motion: Transit motions move the empty end effector along a collision-free path between two workspace poses. Transfer motions grasp a target object and move it and the end effector to a new pose along a collision-free path [4].

Definition 3: Transit motions

A transit motion $\texttt{TRANSIT(}{\alpha,q_{i},q_{j}}\texttt{)}$ is a continuous motion of the robot arm from initial configuration $q_{i}\in\mathcal{Q}$ to $q_{j}\in\mathcal{Q}$ that is collision-free with respect to $\alpha$.

Definition 4: Transfer motions

A transfer motion $\texttt{TRANSFER(}{\alpha_{i},o,q,q^{\prime},\alpha_{j}}\texttt{)}$ is a continuous motion of the robot arm, holding object $o\in\mathcal{O}$, from $q\in\mathcal{Q}$ to $q^{\prime}\in\mathcal{Q}$, that is collision-free with respect to $\alpha_{i}\setminus o$. $q$ and $q^{\prime}$ must place object $o$ at $\operatorname{\alpha_{i}}[o]$ and $\operatorname{\alpha_{j}}[o]$, respectively.

Note that these motion classes do not predefine concrete motion primitives or actions. We are now equipped to formally state the object reconfiguration problem:

Definition 5: Object Reconfiguration Problem

Given an initial valid arrangement (Def. 2), $\alpha_{\mathrm{start}}\in\mathcal{A}$, robot configuration, $q_{\mathrm{start}}\in\mathcal{Q}$, and valid goal arrangement $\alpha_{\mathrm{goal}}\in\mathcal{A}$, the object reconfiguration problem is to find a sequence of objects and robot configurations, $[q_{1},o_{1},q^{\prime}_{1},\ldots,q_{n},o_{n},q^{\prime}_{n}]$ and corresponding alternating TRANSIT and TRANSFER motions such that the sequence:

[TABLE]

is valid and $\alpha_{n}=\alpha_{\mathrm{goal}}$, where $\alpha_{i}$ is the arrangement after executing the $i$-th TRANSFER motion.

This problem formulation is similar to that of [2], but adds a 3D workspace and consideration of stability constraints.

IV Approach

We propose to solve the reconfiguration problem with a bidirectional tree search algorithm, SAST, that operates in a given problem’s arrangement space. SAST resembles RRTConnect, but operates in the arrangement space with a novel extension operator that exploits an embedded physics simulator (Sec. IV-C) to automatically discover valid actions.

IV-A Stable Arrangement Search Trees (SAST)

SAST initializes two trees in the arrangement space, one rooted at the start arrangement, $\alpha_{\mathrm{start}}$, and the other at the goal arrangement, $\alpha_{\mathrm{goal}}$. Vertices in these trees represent valid arrangements (Def. 2); edges represent transformations between valid arrangements. In this work, we consider pick-and-place transformations which move exactly one object. Given two valid arrangements $\alpha_{i-1}$ and $\alpha_{i}$, a connecting edge can be described as $(q_{i},o_{i},q_{i}^{\prime})$. This transformation corresponds to a TRANSIT motion of the robot to $q_{i}$, followed by a stable TRANSFER motion moving $o_{i}$ from its pose in $\alpha_{i-1}$ to $\alpha_{i}$ by grasping $o_{i}$ and moving the robot from $q_{i}$ to $q_{i}^{\prime}$. Edges are bidirectional: the reverse transformation from $\alpha_{i}$ to $\alpha_{i-1}$ corresponds to a TRANSIT motion to $q_{i}^{\prime}$, followed by a stable TRANSFER motion of $o_{i}$ from its pose in $\alpha_{i}$ to $\alpha_{i-1}$ by grasping $o_{i}$ and moving the robot from $q_{i}^{\prime}$ to $q_{i}$. In the arrangement space representation, a solution to a reconfiguration problem is a path of edges that connect $\alpha_{\mathrm{start}}$ to $\alpha_{\mathrm{goal}}$.

Planning starts from the tree rooted at $\alpha_{\mathrm{start}}$. Each iteration of the planning loop samples a random arrangement $\alpha_{\mathrm{rand}}$ and finds its closest neighbor, $\alpha_{\mathrm{nearest}}$ in the current tree (Alg. 1, Algs. 1 and 1). This is done via spatial lookup on a GNAT [50] with arrangement distance defined as the summed $\operatorname{SE}(3)$ distance111$\operatorname{SE}(3)$ distance is the sum of the Euclidean distance of the translational components and the angular distance of the rotational components. between the respective poses of each object in the two arrangements. SAST then attempts to Extend the tree from $\alpha_{\mathrm{nearest}}$ toward $\alpha_{\mathrm{rand}}$ by growing a sequence of edges according to Alg. 3. If the resulting sequence of edges is non-empty (Alg. 1, Alg. 1), we try to Connect the other tree to the terminal vertex of the extended trajectory (Alg. 1, Alg. 1). This is done (Alg. 2) by repeatedly extending the closest arrangement on the other tree to the terminal vertex, until either the connection succeeds or until the extension fails. If connection succeeds, SAST has found a solution and terminates. Otherwise, it swaps the trees and repeats the planning loop.

IV-B Sampling stable arrangements

The SampleArrangement subroutine (Fig. 3) samples a valid arrangement for use with Extend. Here, we leverage the embedded physics simulator to find stable arrangements. First, SampleArrangement picks uniform-random 3D poses for each object within the workspace bounds, using rejection sampling to ensure that the objects do not intersect. Then, it simulates the dynamics of the arrangement forward for a fixed number of small timesteps, checking at fixed intervals if the objects have maintain zero displacement since the previous interval. If so, the arrangement resulting from the applied dynamics is kinematically valid and statically stable, and is returned as a result. Otherwise, this process repeats until a valid sample is found. SampleArrangement is easy to parallelize—our implementation of SAST uses multiple threads to sample stable arrangements.

Uniform-random initial pose sampling trades off performance for ease of specification, avoiding the specialized samplers used by TAMP solvers to find states on low and zero-measure state manifolds.

IV-C Generating valid transformation actions

The Extend subroutine (Alg. 3) searches for a sequence of valid edges that transform a given arrangement $\alpha$ of $k$ objects into a target arrangement $\alpha^{\prime}$. A major contribution of our work is to use an embedded physics simulator in Extend to reason about the validity of these transformations. The simulator allows us to treat the physics of the environment as an implicit specification of the valid transformation actions from any state. We also ensure that a valid transformation has a valid instantiation with the robot by motion planning for its associated TRANSIT and TRANSFER motions.

Extend starts by selecting a random order to move the objects222We choose a random order for simplicity, but could substitute a more sophisticated permutation selector for performance. and setting the current arrangement, $\alpha_{\mathrm{cur}}$ to the given start arrangement, $\alpha$. It then tries to move each object in the chosen order to its target position in the given target arrangement, $\alpha^{\prime}$, while maintaining stability of the other objects.

For each object, $o_{i}$, Extend creates a new arrangement $\alpha_{\mathrm{next}}$ equal to $\alpha_{\mathrm{cur}}$ with $o_{i}$ at its pose in $\alpha^{\prime}$ and samples collision free robot configurations grasping $o_{i}$’s pose in $\alpha_{\mathrm{cur}}$ and at $\alpha_{\mathrm{next}}$, using the same grasp. Then, it checks that:

1. $\alpha_{\mathrm{next}}$is collision-free,

2. $\alpha_{\mathrm{cur}}\setminus o_{i}$is stable, allowing $o_{i}$ to be moved, and

3. $\alpha_{\mathrm{next}}\setminus o_{i}$is also stable, allowing $o_{i}$ to be moved in the reverse transformation

. If these conditions are met, Extend attempts to find a valid TRANSIT motion between the preceding configuration of $\alpha_{\mathrm{cur}}$333If $\alpha_{\mathrm{cur}}=\alpha_{\mathrm{start}}$, we select $q_{0}$ as $q_{\mathrm{prev}}^{\prime}$. If $\alpha_{\mathrm{cur}}=\alpha_{\mathrm{goal}}$, we skip this check since there is no constraint on the robot configuration at the goal. and the sampled grasp for $o_{i}$’s pose in $\alpha_{\mathrm{cur}}$, and a valid TRANSFER motion between the sampled grasps for $o_{i}$’s pose in $\alpha_{\mathrm{cur}}$ and $\alpha_{\mathrm{next}}$, respectively, using a standard motion planner (Alg. 3, Algs. 3 and 3). These motions are considered infeasible if the sub-planner fails to find a solution within a predefined timeout.

If Extend finds the requisite TRANSIT and TRANSFER motions, then it either adds the discovered edge to the current tree (Alg. 3, Algs. 3, 3 and 3) and continues with the next object and $\alpha_{\mathrm{cur}}=\alpha_{\mathrm{next}}$, or attempts to connect the newly-reached arrangement to the other tree (Alg. 3, Algs. 3, 3, 3 and 3) if $\alpha_{\mathrm{next}}$ is the target arrangement and a connection is desired. In the latter case, Extend returns the target arrangement (Alg. 3, Alg. 3); otherwise, it iterates through the remaining objects and returns the last reached arrangement.

IV-D Solution simplification

Although unnecessary for completeness, SAST applies heuristic simplifications to solutions to improve their quality.

If an object $o$ has been moved twice along a solution trajectory, one of these motions may be unnecessary. We can remove the first motion by altering the second motion to move $o$ starting from the first motion’s starting pose. Similarly, we can remove the second motion by altering the first motion to move $o$ to the second motion’s ending pose. Both cases modify the pose of $o$ in the arrangements between the first and second motions. This requires recomputing the grasps and planning motions for these intermediate arrangements to validate the altered arrangement trajectory. In a third case, motions may also be removed if the pickup and placement locations of the object are exactly the same.

SAST iterates through these three simplification cases on solutions, rechecking for stability and recomputing the TRANSIT and TRANSFER motions after each modification to ensure that the solution remains feasible. This simplification process continues until no potentially redundant actions remain. Note that this heuristic set is non-exhaustive and does not guarantee optimal motions.

V Experiments

We evaluate SAST on a set of 3D tabletop rearrangement problems (Fig. 4)—Reverse (a-b), Transform (c-d), and Rotate (e-f). These problems involve using a single-arm manipulator to reconfigure cubes from one 3D structure to another. They require reasoning about the physical constraints between the cubes, as well as with the environment. The solutions are non-trivial, in that the robot must choose and move the objects through intermediate arrangements to achieve its goal. In addition, some problems contain obstacles such as tiles and bumps which complicate the validity of actions. Grounding these details in order to apply contemporary approaches would be tedious and challenging.

V-A Implementation

We use DART [19] as our embedded physics simulator and plan TRANSIT and TRANSFER motions via Robowflex [51] with the Open Motion Planning Library (OMPL) [52]. All experiments ran on an AMD 5900x desktop CPU with $12$ cores at $4.8$ GHz, using $6$ parallel threads for sampling stable arrangements and inverse kinematics.

V-B Planning performance

We applied SAST to each test problem for $50$ trials with a maximum timeout of $300$ seconds per trial. In each trial, we randomize the start and goal positions of the structure to rearrange, together with the the obstacle positions.

Table I shows that SAST was almost always able to find a solution within the stipulated time limit. Solution times were also reasonable, taking not more than a minute per successful run despite having to invoke the simulator repeatedly for collision and static stability checking, and having to integrate the low-level motion planning of the TRANSIT and TRANSFER motions. The sizes of the search trees, in terms of the number of nodes, were also small, indicating that a sparse coverage of arrangements was sufficient to identify a solution.

Across each problem and trial, we only had to specify the geometry and positions of the obstacles (steps and bumps) and the start and goal arrangement poses of the objects. This highlights the strength of our approach in using the physics simulator to automatically derive action validity without requiring any manual, explicit specification.

Solution lengths, however, often require about twice the optimal number of steps. This is because SAST, like RRTConnect, is non-optimizing.

V-C Simplification performance

The results in Table I do not use the solution simplification heuristics of Sec. IV-D. Table II shows the results of applying these heuristics to the solutions found by each successful run, indicating that solution simplification usually terminated within $40$ seconds. Most of the additional time comes from rechecking for stability and replanning for the low-level TRANSIT and TRANSFER motions, required whenever two actions moving the same object merge. This is done up to $\mathcal{O}(n^{3})$ times in the number of actions in the initial solution.

Simplification usually decreased solution length by roughly half, reaching or coming close to the optimal solution length. Fig. 5 shows an example of a simplified Rotate solution.

V-D How important is integrating motion planning?

SAST verifies the feasibility of each TRANSIT and TRANSFER motion by planning a valid trajectory in the robot’s configuration space for the motion. To investigate the impact of this integrated verification, we conducted an ablation experiment by removing these low-level feasibility checks. We find solutions in terms of sequences of object arrangements and object grasps, assuming that the transformations between arrangements are feasible. After finding a full solution, we attempt to compute a motion plan for each of the associated low-level motions to check solution validity.

Table III shows a substantial drop in solution feasibility when low-level motion checks are skipped. Indeed, TRANSIT motions require checking that an object is reachable with a given grasp without the manipulator colliding with the other objects; TRANSFER motions require checking that an object can be pulled away without the object or the manipulator intersecting with the other nearby objects. The problems we consider have environment obstacles (table, tiles, and bumps) that do not interfere much with the robot’s motion—in more constrained environments, such as in a cupboard or drawer, we could expect feasibility to worsen.

VI Discussion

This work contributes a novel perspective on manipulation planning that embeds a simulator to implicitly encode the valid actions and states for a problem domain. We demonstrate this perspective for 3D object reconfiguration planning, where we are able to efficiently find statically stable object configurations that would otherwise be onerous to specify.

SAST currently uses random sampling and extension to grow the arrangement space graph, but informed approaches like Expansive Space Trees [53] or Monte Carlo Tree Search [33] may discover solutions faster. SBMP advances such as biased samplers [54, 55] and optimizing planners [56, 57, 58, 59] may also complement embedded-simulator planning.

Embedded-simulator planning is broadly applicable outside object reconfiguration, which poses several directions for future work. How can we use simulators to encode constraints beyond stability, such as orientation or contact dynamics? Similarly, what are the precise requirements for an embedded simulator? For some problems, precise physics simulation may be unnecessary; for others, non-standard physics can encode problem constraints. Further, how well do plans found via embedded simulation transfer to the real world?

Finally, we wish to explore richer uses of the embedded simulator, including combining differentiable simulation with optimization techniques, to broaden the manipulation problem classes that we can efficiently solve.