NeuralSVCD for Efficient Swept Volume Collision Detection

TL;DR

NeuralSVCD introduces a neural network-based approach for swept volume collision detection that improves both accuracy and efficiency, enabling safer and faster robotic manipulation in complex environments.

Contribution

The paper presents NeuralSVCD, a neural encoder-decoder architecture that balances efficiency and accuracy in SVCD by leveraging shape and temporal locality.

Findings

Outperforms existing SVCD methods in accuracy

Achieves higher computational efficiency

Demonstrates robustness across various scenarios

Abstract

Robot manipulation in unstructured environments requires efficient and reliable Swept Volume Collision Detection (SVCD) for safe motion planning. Traditional discrete methods potentially miss collisions between these points, whereas SVCD continuously checks for collisions along the entire trajectory. Existing SVCD methods typically face a trade-off between efficiency and accuracy, limiting practical use. In this paper, we introduce NeuralSVCD, a novel neural encoder-decoder architecture tailored to overcome this trade-off. Our approach leverages shape locality and temporal locality through distributed geometric representations and temporal optimization. This enhances computational efficiency without sacrificing accuracy. Comprehensive experiments show that NeuralSVCD consistently outperforms existing state-of-the-art SVCD methods in terms of both collision detection accuracy and computational efficiency, demonstrating its robust applicability across diverse robotic manipulation scenarios. Code and videos are available at https://neuralsvcd.github.io/.

Tables

Table 1. Table 1: Motion planning performance over 500 random trials per task, reporting average success rate (Suc Rate), planning time (Time), and maximum penetration depth (Max Pen).

Task	Robot	Motion planner	Suc Rate (%)	Time (s)	Max Pen (mm)
dish insertion	UR5 (6 DoF)	cuRobo-sphere-50	73.6	9.99	3.73
		cuRobo-sphere-400	95	12.35	0.5
		cuRobo-NeuralSVCD-discrete	8.5	26.62	52.13
		cuRobo-NeuralSVCD	99.3	9.64	0.01
bimanual insertion	ARMADA [25] (12 DoF)	cuRobo-sphere-100	70.4	5.27	3.64
		cuRobo-sphere-1200	84.9	12.45	1.03
		cuRobo-NeuralSVCD-discrete	82.2	3.26	2.72
		cuRobo-NeuralSVCD	92.8	2.92	0.33
mining site navigation	mobile UR5 (9 DoF)	cuRobo-sphere-50	73.8	11.67	10.82
		cuRobo-sphere-2000	83.8	60.91	5.32
		cuRobo-NeuralSVCD-discrete	1.0	11.92	86.15
		cuRobo-NeuralSVCD	92.2	9.84	3.81

Table 2. Table 2: Hyperparameter Sampling Ranges for SVCD Baselines. U U (min, max) denotes uniform distribution

SVCD Method	Hyperparameter	Sampling Range
Convex Cell / GJK	Trajectory Discretization Number	$U(2,256)\subset \mathbb{Z}$
	Activation Length	$U(-0.01,0.01)\subset ℝ$
Sphere / Sphere-Mesh Distance	Trajectory Discretization Number	$U(2,64)\subset ℝ$
	Activation Length	$U(-0.01,0.01)\subset ℝ$
	Voxel Size (Sphere Approximation)	$U(0.01,1)\subset ℝ$
	Number of Surface Points (Sphere Approximation)	$U(0,100)\subset \mathbb{Z}$
	Mesh Simplification Voxel Factor	$U(2,32)\subset \mathbb{Z}$
NeuralSVCD	Trajectory Discretization Number	$U(2,100)\subset \mathbb{Z}$
	Number of Collision Pairs After Broad-Phase Filtering	$U(2,256)\subset \mathbb{Z}$

Table 3. Table 3: Hyperparameter settings for the bimanual insertion task across motion planners.

Hyperparameter	cuRobo- sphere- 100	cuRobo- sphere- 1200	cuRobo- NeuralSVCD	cuRobo- NeuralSVCD- discrete
Number of initial seeds	2
MPPI - interpolation number	4			100
MPPI - iteration	40
MPPI - number of samples	200			50
MPPI - number of control points	5
LBFGS - interpolation number	2			100
LBFGS - iteration	10
LBFGS - number of control points	10
${\alpha }_{vel}$	10.0
${\alpha }_{col}$	1.0
${\alpha }_{acc}$	2
${\alpha }_{jerk}$	10
Activation distance	0.010		-	-
Number of sphere for robot	891		-	-
Number of sphere for grasping object	100	1200	-	-

Table 4. Table 4: Hyperparameter settings for the mining site navigation task across motion planners.

Hyperparameter	cuRobo- sphere- 50	cuRobo- sphere- 2000	cuRobo- NeuralSVCD	cuRobo- NeuralSVCD- discrete
Number of initial seeds	2
MPPI - interpolation number	3			100
MPPI - iteration	40
MPPI - number of samples	400			50
MPPI - number of control points	8
LBFGS - interpolation number	8			100
LBFGS - iteration	10
LBFGS - number of control points	3
${\alpha }_{vel}$	1
${\alpha }_{col}$	5
${\alpha }_{acc}$	10
${\alpha }_{jerk}$	50
Activation distance	0.040	0.040	-	-
Number of sphere for robot	397		-	-
Number of sphere for grasping object	50	2000	-	-

Table 5. Table 5: Hyperparameter settings for the dish insertion task across motion planners.

Hyperparameter	curobo- sphere- 50	curobo- sphere- 400	curobo- NeuralSVCD	curobo- NeuralSVCD- discrete
Number of initial seeds	2
MPPI - interpolation number	4			50
MPPI - iteration	40
MPPI - number of samples	200			50
MPPI - number of control points	5
LBFGS - interpolation number	2			50
LBFGS - iteration	10
LBFGS - number of control points	10
${\alpha }_{vel}$	10.0
${\alpha }_{col}$	1.0
${\alpha }_{acc}$	2
${\alpha }_{jerk}$	10
Activation distance	0.040	0.040	-	-
Number of sphere for robot	397		-	-
Number of spheres for grasping object	50	400	-	-

Equations

$$\underset{t\in [0,1]}{\min }\text{dist}(p_i^{\text{static}},\tau (t)\cdot p_j^{\text{mov}})-r_i^{\text{static}}-r_j^{\text{mov}},$$

$$C_{traj}({\tau }_{control})=C_{smooth}({\tau }_{control})+{\alpha }_{col}{C}_{SVCD}({\tau }_{control})$$

$$\Phi (x)=\frac{1}{max(\Vert z_1\Vert ,\Vert z_2\Vert )}(\varphi (x{)}^{-1}\cdot x),$$

Full text

Keywords: Neural swept-volume collision detection, Motion planning

1 Introduction

Robotic motion planning in unstructured environments, such as inserting dishes into a rack, assembling parts, or navigating in a tight space, requires accurate collision checking. Traditional discrete collision checkers sample a finite set of waypoints along a candidate path, but this can overlook collisions that occur between samples, known as “tunneling” errors. In contrast, SVCD prevents tunneling problems by using the swept volume of a path of the object to check collision, making it more practical, especially in tight‐tolerance tasks [1]. See Figure 1. Formally, given an object (or robot) trajectory $\tau:[0,1]\to SE(3)$ and geometric models of the objects, SVCD algorithms output the maximum penetration depth. See Figure 2, (a) for an illustration.

SVCD algorithms typically have two design components: the geometric representation of entities and a collision detector tailored for the representation. For instance, previous works [2, 3] discretize trajectories, construct convex hulls using vertices of a pair of waypoints in the trajectory, then apply the Gilbert–Johnson–Keerthi (GJK) algorithm [4] designed for convex shapes to each convex hull (Figure 2, (b)). Ideally, we would parallelize the collision checks on convex hulls using a GPU, but because GJK involves branching in computation, this is difficult and it must iterate through each convex hull to check collisions [5], which makes it inefficient. In contrast, collision detection between spheres can be computed in parallel on a GPU, as it only involves evaluating distances between sphere centers. Recent approaches [6, 7, 8] exploit this property to approximate complex shapes with multiple spheres, discretize trajectories at a high resolution, and use parallelized sphere-based collision checks at all waypoints (see Figure 2 (c)). However, this approximation sacrifices accuracy, leading to inaccurate collision detection.

Instead, we propose a neural network (NN)-based SVCD algorithm using an encoder-decoder architecture. Specifically, the encoder processes pairs of object shapes, poses, and trajectory data to produce neural representations of the shapes, while the decoder predicts collision outcomes directly from these representations. This offers two primary advantages. First, by training an encoder-decoder network for collision prediction, the encoder learns a latent representation that preserves only the relevant features, rather than a complete mesh. Second, since the decoder is a NN, it can leverage GPU parallelization. However, there are two challenges involved in this approach. First, we have a limited amount of 3D asset data [9, 10], but we need to make the model generalize across a variety of novel shapes. Second, even for the same shapes, you can create a large variety of object trajectories, where even seemingly similar trajectories could have different collision outcomes, especially in tight spaces.

To overcome these limitations, we leverage two essential insights: shape locality and temporal locality. Shape locality addresses the scarcity of 3D asset data by recognizing that local geometric features remain relatively consistent, even when global shapes differ significantly, as shown in Figure 3 (Left). So, if we restrict the neural decoder’s input to local object features that are relevant for collision rather than global object geometry, the model would generalize collision predictions across unseen shapes. Temporal locality is the observation that collisions are generally influenced by short trajectory segments rather than an entire path, as depicted in Figure 3 (right). Consequently, if we limit the decoder’s input to these relevant trajectory segments instead of complete trajectories, our approach would be more robust across diverse trajectories.

Based on these insights, we propose NeuralSVCD, which uses a novel geometric representation designed to leverage shape locality and temporal locality. Specifically, we represent a shape as a set of uniformly sampled points on the object and encode the local shape around each point into a latent representation using a neural network. The entire shape, then, is represented with a collection of these local latent representations. A decoder, instead of taking the entire object shapes as inputs, only takes a pair of these local representations to check if there is a collision between these two local shapes. However, naively checking a collision this way would require evaluating $N^{2}$ pairs of local representations continuously over the trajectory, where $N$ is the number of local representations. To avoid excessive collision checks, we propose instead to first perform a broad-phase collision detection by approximating local shapes with spheres. Then, we transform the spheres along the trajectory to find the time and pair of spheres that intersect. This process yields the specific pair of local latent vectors and the corresponding collision time along the trajectory. Then, we run the collision decoder on pairs whose corresponding spheres are in collision, supplying the corresponding locally linearized trajectory segment. This allows us to limit the decoder’s input to these relevant trajectory segments and local shapes rather than complete trajectories and global shapes.

We benchmark our approach against multiple state-of-the-art SVCD baselines, including mesh-based and sphere-based methods, demonstrating significant improvements in both swept volume collision detection accuracy and runtime. When integrated into the cuRobo trajectory optimizer and evaluated on three challenging robotics tasks, NeuralSVCD achieves higher success rates and reduced computation times compared to state-of-the-art approaches.

2 Related works

2.1 Explicit shape representations for swept volume collision detection

Continuous collision detection (CCD) and SVCD both answer whether a moving object intersects a static obstacle along a given trajectory. CCD additionally returns the time of first contact (TOC), which is the first time at which the two meshes collide [11, 12, 13, 14]. SVCD instead outputs the maximum penetration depth encountered over the entire motion. SVCD involves two critical challenges: (i) reconstructing the swept volume of the moving object and (ii) accurately measuring its penetration depth against the static mesh.

There are lines of work that reconstruct swept volume as triangle mesh [15, 16, 17, 18, 19]. Even though they can generate accurate swept volumes, computing penetration depth against a non-convex mesh of the static object is slow due to exhaustive triangle–edge pair tests. To accelerate queries, an alternative approach forgoes exact swept-volume reconstruction and instead approximates the swept volume with a collection of convex hulls [3, 2], which enables use of GJK [4]. We obtain the swept volume by convex‐decomposing both object meshes, discretizing the trajectory into segments, and for each segment and convex part, building the convex hull of its vertices at that segment’s start and end poses (see Fig. 2, second column). After reconstructing the swept volume, the penetration depth is calculated by applying the GJK algorithm for each pair of convex cells from different objects. This hull-based approach is exact only under pure translation, and GJK’s iterative, branch-heavy algorithm hinders parallel performance, making it inefficient for GPU-accelerated pipelines.

An alternative for efficient penetration‐depth estimation decomposes the moving object into a set of spheres [6]. In CuRobo, these spheres query signed distances against the static mesh in parallel on the GPU. Penetration is measured by discretizing the trajectory into piecewise‐linear segments, selecting all spheres that intersect the environment along each segment, and summing their penetration depths (Fig. 2, third column). Although this approach scales linearly with the number of spheres, its geometric fidelity suffers for thin or highly concave shapes, as accurately covering narrow features or deep cavities requires a prohibitively large number of spheres, as demonstrated in our motion planning experiments.

2.2 Implicit shape representations for swept volume collision detection

To support general trajectories, recent works use the implicit function, which is a function that outputs the degree of distance to the surface from a given point, taking negative values within the object and positive values outside. The swept volume is represented by an implicit function that, for any point in space, takes the smallest value of the object’s original implicit function observed over the entire motion, evaluated with respect to time along the trajectory [20]. This process is referred to as time optimization because it determines the timestep at which maximum penetration occurs between a point and a moving object along its trajectory. Penetration depth between the swept volume and the static object is computed by evaluating the swept volume’s implicit function on the static object’s surface and taking the pointwise minimum. This method supports arbitrary trajectories and any implicit representation. However, accurate penetration‐depth estimation via implicit functions requires dense surface sampling and a separate time‐optimization solution for each sample, which becomes prohibitively expensive.

Instead, we can use an object collision network, which predicts contact probability based on the shape and pose of two objects [5, 21, 22], rather than an implicit function, to accelerate the time optimization process. Time optimization, which originally finds the moment of maximal penetration between a moving object and a single surface point, can be reframed as locating the time of maximum penetration between the moving mesh and an entire static object (Fig. 2, fourth column). To our knowledge, ours is the first SVCD approach to fuse an object-collision network into this time-optimization loop for defining the swept volume.

3 Neural swept volume collision detection

Swept volume collision detection (SVCD) involves identifying potential collisions between a static environment and the swept volumes generated by objects moving along specified trajectories. Formally, the swept volume (SV) of an object is defined as the spatial region occupied by its geometry as it continuously moves along a trajectory $\tau:[0,1]\rightarrow SE(3)$. Given meshes for both a static scene (stationary environment) and an object at a canonical pose, SVCD determines if there is a collision between the object moving through $\tau$ and the scene. Throughout this paper, the subscript $mesh_{static}$ denotes a representation of the mesh of a static object that remains fixed during the collision detection process, and the subscript $mesh_{mov}$ specifically indicates representation of a moving object at its canonical pose, prior to their transformation along trajectory $\tau$.

3.1 Distributed latent scene representation from neural encoder

Given meshes $\text{mesh}_{\text{mov}}$ and $\text{mesh}_{\text{static}}$, our first step is to encode these meshes into distributed latent object representations $\mathcal{Z}$. To leverage shape locality effectively, we adopt a distributed latent representation strategy. Specifically, we first sample surface points from each mesh and select $N$ representative points $\{p_{i}\}_{i=1}^{N}$ using furthest point sampling (FPS), ensuring uniform coverage across the surface. Each sampled surface point is then assigned to its nearest representative point $p_{i}$. For each representative point $p_{i}$, we input its assigned local points into a neural network to generate a localized latent shape representation $z_{i}\in\mathbb{R}^{K}$. This process yields distributed pairs $\{(p_{i},z_{i})\}_{i=1}^{N}$.

For efficient broad-phase collision detection, we define bounding spheres around each representative point $p_{i}$ with radius $r_{i}\in\mathbb{R}$. We define the radius $r_{i}$ for each point based on the distance to its nearest neighboring representative point: $r_{i}=\alpha\min_{j\in[1,N],j\neq i}\text{dist}(p_{i},p_{j})$, where $\alpha>1$, ensuring each sphere encloses the local geometry near its representative point. Consequently, our final distributed latent object representation is $\mathcal{Z}=\{(p_{i},z_{i},r_{i})\}_{i=1}^{N}$, illustrated in the upper part of Figure 4.

3.2 Two-Step SVCD via Broad-phase Filtering and Neural Refinement

Given distributed representations of the moving object $Z_{\text{mov}}=\{(p_{i}^{\text{mov}},z_{i}^{\text{mov}},r_{i}^{\text{mov}})\}_{i=1}^{N}$ at the initial trajectory position $\tau(0)$, static scene $Z_{\text{static}}=\{(p_{j}^{\text{static}},z_{j}^{\text{static}},r_{j}^{\text{static}})\}_{j=1}^{N}$, and a trajectory $\tau:[0,1]\rightarrow SE(3)$, we propose a two-step SVCD approach to improve computational efficiency. Our method comprises a broad-phase that quickly identifies potential collision candidates, followed by a narrow-phase employing a neural SVCD decoder for precise collision evaluation.

Broad-phase with Sphere Approximation: Performing collision detection between the static object and the swept volume of one moving object typically involves checking all possible pairs of local representations, leading to $N^{2}$ checks continuously over the trajectory, which is computationally intensive. To reduce complexity, we first use a broad-phase collision detection to identify potential collision pairs and their corresponding times efficiently. Specifically, we use bounding spheres with radii $r_{i}^{\text{static}}$ for the static object and $r_{j}^{\text{mov}}$ for the moving object. Throughout this section, the subscript $\cdot_{i}$ denotes elements of the static representation $\mathcal{Z}_{\text{static}}$, and the subscript $\cdot_{j}$ denotes elements of the moving representation $\mathcal{Z}_{\text{mov}}$. We evaluate potential collisions by checking overlaps between each static sphere and the corresponding swept volume formed by each moving sphere as it follows the trajectory $\tau(t)$. For each pair $(i,j)$ within the complete set $\mathcal{P}_{\text{all}}=\{(i,j)\mid i,j\in[1,N]\}$, we perform the following numerical optimization (e.g., via Newton’s method):

[TABLE]

where $\text{dist}(\cdot,\cdot)$ denotes the Euclidean distance between points, and $\tau(t)\cdot p_{j}^{\text{mov}}$ represents the transformed position of $p_{j}^{\text{mov}}$ under the trajectory $\tau(t)\in SE(3)$. We denote by $t^{\dagger}_{ij}$ the time along the trajectory at which this quantity is minimized—that is, the moment when the two bounding spheres are in closest proximity. Negative results from Eq. (1) indicate possible collisions, forming the reduced candidate set of $(i,j,t^{\dagger}_{ij})$, which requires further detailed evaluation in the narrow-phase. Additionally, solving this optimization provides the argmin solution—the pseudo-critical collision time $t^{\dagger}$—which serves as an approximation to the true critical collision time $t^{*}$, derived from simplified spherical approximations rather than exact geometrical representations.

Narrow-phase with Neural Decoder: For each candidate pair $(i,j,t^{\dagger}_{ij})$ identified in the broad-phase, we refine collision predictions using a neural SVCD decoder $f_{SVCD}$. The decoder takes as input the static representation $(p_{i}^{\text{static}},z_{i}^{\text{static}})$ and the swept volume representation generated by $(p_{j}^{\text{mov}},z_{j}^{\text{mov}})$ along the trajectory $\tau$. To exploit temporal locality, we avoid using the entire trajectory, focusing instead on a local trajectory segment around $t^{\dagger}_{ij}$, we approximate the local trajectory using a linear, first-order Taylor expansion parameterized by $\xi^{t^{\dagger}_{ij}}\in\mathbb{R}^{6}$, representing linear and angular velocities at $t^{\dagger}_{ij}$, along with the pose $\tau(t^{\dagger}_{ij})$. We define the local swept volume segment using $(p_{j}^{\text{mov}},z_{j}^{\text{mov}})$ transformed by this local trajectory approximation. The neural decoder $f_{SVCD}$ then processes inputs $(p_{i}^{\text{static}},z_{i}^{\text{static}})$, $(p_{j}^{\text{mov}},z_{j}^{\text{mov}})$, $\xi^{t^{\dagger}_{ij}}$, and $\tau(t^{\dagger}_{ij})$ to predict the collision probability.

We assume that the pseudo-critical collision time $t^{\dagger}_{ij}$ sufficiently approximates the true critical collision time $t^{*}$, which is time where decoded collision probability is maximized, ensuring that the local swept volume defined by $\xi^{t^{\dagger}_{ij}}$ and $\tau(t^{\dagger}_{ij})$ adequately captures the critical collision region for accurate predictions. Specifically, the neural decoder $f_{SVCD}$ receives the inputs $(p_{i}^{\text{static}},z_{i}^{\text{static}})$, $(p_{j}^{\text{mov}},z_{j}^{\text{mov}})$, $\xi^{t^{\dagger}_{ij}}$, and $\tau(t^{\dagger}_{ij})$. We train $f_{SVCD}$ to directly predict collision probabilities from these inputs. Practically, $f_{SVCD}$ consists of multiple multilayer perceptrons (MLPs) that process concatenated inputs and output collision probabilities within $[0,1]$. Final collision predictions for rigid bodies are obtained by applying max pooling across all candidate pair probabilities identified during the broad-phase. This inference procedure is illustrated in the lower portion of Figure 4.

3.3 Training Encoder and Decoder

In this section, we describe our procedure for training both the shape encoder and the neural SVCD decoder. Our training dataset consists of 2556 carefully curated meshes sourced from established 3D asset collections, including GoogleScannedObjects [9], ObjaverseXL [10], and NOCS [23]. We create training datapoints by randomly pairing a static mesh $\text{mesh}_{\text{static}}$ with a canonical moving mesh $\text{mesh}_{\text{mov}}$. These pairs undergo random transformations, and we sample trajectories $\tau(\cdot):[0,1]\rightarrow SE(3)$. For training simplicity, we restrict trajectories to linear paths by randomly selecting directions in $\mathbb{R}^{6}$ and an initial pose $\tau(0)\in SE(3)$. Ground-truth collision labels $y_{SVCD}$ are computed using the FCL library [24]. Thus, each datapoint is formally defined as $d:=(\text{mesh}_{\text{static}},\text{mesh}_{\text{mov}},\tau(\cdot),y_{SVCD})$. To ensure balanced training, we apply rejection sampling to maintain an equal proportion of collision and non-collision instances, resulting in our final training dataset $\mathcal{D}$. The training objective is defined as $\sum_{d\in\mathcal{D}}L_{SVCD}(d),$ where $L_{SVCD}$ represents the binary cross-entropy loss between the predicted collision outcomes and the ground-truth labels $y_{SVCD}$. To enable NN to output the degree of penetration instead of the predicted binary label, we add a regularization loss. See Appendix A for further details.

4 Experiments

Our experiments are designed to test two main hypotheses: (1) NeuralSVCD outperforms the state-of-the-art methods for SVCD in terms of both accuracy and computational efficiency. (2) When integrated into a motion planning framework, our SVCD algorithm leads to higher success rates and better efficiency compared to systems employing the baseline SVCD implementations.

4.1 Swept Volume Collision Detection Accuracy

To quantify the accuracy of the SVCD, we perform a comparative experiment. We use $9$-DOF UR5 robot with mobile base as moving robot (3-base, 6-arm). For each trial, we randomly assign the initial and goal configurations within a bounded workspace and select a static object from a predefined pool, assigning it a random pose. Visualization of the problem case is shown in Figure 6. To evaluate generalization, we partition the obstacles into two asset pools: In‑Domain (ID)—objects that were present during training—and Out‑of‑Domain (OOD)—novel shapes unseen at training time. Near-contact situations are critical for collision-free planning in confined spaces. As in [21], we adjust the pose of the static object to ensure a small penetration depth. A detailed problem-generation process is described in the Appendix B.

We compare NeuralSVCD with seven baselines:

• •

Convex Cell/GJK: Following [3, 2], we convex-decompose moving and static meshes. The continuous variant discretizes trajectories into segments and forms convex hulls of each moving object part at segment endpoints. The discrete variant unions convex parts at discrete waypoints. Both use GJK collision detection on convex cell pairs.

• •

Sphere/Sphere-Mesh Distance: As in [6], the moving object mesh is approximated by spheres (Appendix B). The continuous variant samples sphere paths along trajectory segments, while the discrete variant samples at waypoints. Collision detection involves sphere–mesh distance queries.

• •

Implicit‐function/time optimization: Following [20], we define the moving object by its signed‐distance implicit function and densely sample surface points on the static mesh. For each point, we solve a time-optimization problem to determine the timestamp at which the trajectory achieves maximum penetration. This method produces accurate ground-truth collision labels, provided that a sufficient number of surface points are sampled.

• •

NeuralSVCD** - Discrete:** This variant uses the same encoder-decoder but replaces time optimization with fixed waypoint sampling: the trajectory is approximated by a set of discrete configurations, and collision checks are evaluated only at those points.

• •

NeuralSVCD** - Global Representation:** This variant replaces multiple local latent representations $z$ with a single global latent representation.

We randomly sample hyperparameters (e.g., interpolation length; see Table 2) and measure collision-detection accuracy and per-query inference time. Figure 5 shows that NeuralSVCD outperforms all baselines: it attains $90\%$ accuracy in $6.4\times 10^{-5}\,\mathrm{s}$ and peaks at $95.6\%$ (ID) and $94.9\%$ (OOD). In contrast, convex-cell/GJK continuous and discrete variants are $120\times$ and $590\times$ slower (due to GPU-unfriendly branching), while sphere-mesh methods require $24\times$ more time in the continuous case and never reach $90\%$ in the discrete case (owing to the large number of spheres and interpolations). NeuralSVCD using global representation achieves only $81\%$ (ID) and $76\%$ (OOD) of maximum success rate. NeuralSVCD’s discrete variant degrades maximum success rate by $1.7\%$ (ID) and $2.3\%$ (OOD), and 5% lower success rate at same inference time of $6.4\cdot 10^{-5}$s in both ID and OOD settings because grazing contacts require finer discretization.

4.2 Motion planning

To demonstrate improvements in motion planning performance, we integrate NeuralSVCD into the cuRobo trajectory optimization framework [6], which we refer to as cuRobo-NeuralSVCD. Given meshes of the static scene, robot links, grasped objects, and specified start and goal robot configurations, the motion planning algorithm generates collision-free trajectories between these configurations. Initially, we represent the trajectory as a spline with a fixed number of control points. The initial trajectory is a linear interpolation between the initial and goal states. We then apply an optimization-based motion planner with an objective function from cuRobo but with our custom collision cost.

We compare two baselines that use the same planner but different collision detectors. The first, cuRobo-sphere, employs sphere approximations for SVCD; we test it with varying sphere counts per task (denoted cuRobo-sphere-number of spheres) to explore the accuracy–efficiency trade-off. The second, cuRobo-NeuralSVCD-discrete, uses our discrete SVCD variant (Section 4.1), approximating trajectories by fixed waypoints and performing collision checks only at those configurations.

We evaluate performance on three distinct motion‐planning tasks—dish insertion, bimanual peg assembly, and mining‐site navigation—as illustrated in Figure 7. These tasks, which require navigating tight spaces and interacting with shallow geometries, underscore the necessity of SVCD throughout the entire trajectory. Each task is defined by initial and goal configurations for the robot, a set of robot links, a set of static obstacles, and hyperparameters for the optimization, such as a limit on the number of optimization iterations. For a detailed description, see C.

We evaluate performance by measuring success rate, execution time, and maximum penetration depth over 500 different problem instances for each task. A task is considered successful if no collisions occur along the entire trajectory and the goal is reached. The aggregated results are presented in Table 1. Across all three tasks, cuRobo-NeuralSVCD consistently achieves higher success rates, faster runtimes, and lower maximum penetration depths than both cuRobo‐sphere and cuRobo‐NeuralSVCD‐discrete. Although increasing sphere counts in cuRobo-sphere can match our success rate, it incurs at least $1.3\times$ and up to $6.2\times$ longer computation time. Adequate on bimanual insertion, the discrete variant falls below $10\%$ in complex or large‐scale scenarios, whereas our continuous approach maintains a robust $92.2\%$ success. These results highlight NeuralSVCD’s versatility and safety across diverse robot platforms and environments (see Appendix D).

5 Conclusion

We introduced NeuralSVCD, a distributed latent representation and neural swept-volume collision detector that predicts collision occurrence. By decomposing each object into a collection of locally optimized latent representations and pairing them with a neural decoder, our approach (i) captures only those surface features that matter for collision, (ii) exploits massive GPU parallelism through purely tensor operations, and (iii) generalizes across previously unseen shapes and trajectories by leveraging shape and temporal locality.

Experiments on swept-volume collision detection demonstrate that NeuralSVCD deliver up to a 24× speed-up over sphere-based GPU methods and more than a 100× speed-up over mesh-based GJK—while simultaneously improving accuracy by $\approx$ 5–15 % even with the shapes unseen during training. When embedded in a trajectory-optimization–based motion planner, these gains result in higher success rates, lower penetration depths, and shorter planning times across tasks that span tight-tolerance assembly, dual-arm manipulation, and mobile manipulation in a cluttered mining tunnel.

6 Limitation

6.1 Limitations in collision distance prediction

Trajectory optimization methods, such as TrajOpt [3] and cuRobo [6], rely on collision distance metrics that indicate penetration depth (negative values) during collisions and separation distances (positive values) when no collision occurs. In contrast, our method’s function $f_{SVCD}$ is trained primarily as a binary classifier, outputting collision probabilities. Therefore, we use logits—computed as the inverse sigmoid of these probabilities—as approximate surrogates for collision distances. While this approximation enables a smooth transition between collision and non-collision states and facilitates effective gradient computations during optimization, deviations from true collision distances can occasionally lead gradient-based trajectory optimizers to local minima. In practice, a two-stage trajectory optimization approach combining MPPI and L-BFGS has proven effective at mitigating the issue of local minima, as validated through experiments in three challenging motion planning scenarios. This observation aligns with findings reported by cuRobo [6].

6.2 Application to raw sensory observations

The current framework assumes access to the full mesh of the scene, including robot links, grasped objects and static environments. In practical real-world applications, obtaining such detailed representations is challenging. Recent advances in learning latent shape representations from raw sensory inputs, such as point clouds [26] or RGB images [27], indicate that it is possible to estimate object representation $\mathcal{Z}$ from these sources. This finding opens an exciting avenue for future research.

Acknowledgments

This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant and National Research Foundation of Korea (NRF) funded by the Korea government(MSIT) (No.2019-0-00075, Artificial Intelligence Graduate School Program(KAIST)), (No.2022-0- 00311, Development of Goal-Oriented Reinforcement Learning Techniques for Contact-Rich Robotic Manipulation of Everyday Objects), (No. 2022-0-00612, Geometric and Physical Commonsense Reasoning based Behavior Intelligence for Embodied AI), (No. RS-2024-00359085, Foundation model for learning-based humanoid robot that can understand and achieve language commands in unstructured human environments), (No. RS-2024-00509279, Global AI Frontier Lab).

Appendix A Implementation Details in training

From given training dataset $\mathcal{D}$, the training objective is $\sum_{d\in\mathcal{D}}L_{SVCD}(d)$, which is the binary cross-entropy loss between predicted collision outcomes and ground-truth labels $y_{SVCD}$. However, binary classification loss alone results in almost zero gradients when two objects are deeply penetrated into each other because the probability is close to 1 and nearly unchanged. This is undesirable because The training objective is defined as \sum_{d\in\mathcal{D}}\big{[}L_{SVCD}(d)\big{]}, where $L_{SVCD}$ is the binary cross-entropy loss between predicted collision outcomes and ground-truth labels $y_{SVCD}$. , and $L_{reg}$ is a regularization loss given by: L_{reg}(d)=\Bigl{(}\lVert\nabla f_{SVCD}(d)\rVert-1\Bigr{)}^{2}, encouraging smooth gradient variations beneficial for stable trajectory optimization [28].

Appendix B Implementation Details in SVCD Accuracy

Problem Generation. First, we compute the shortest distance vector $\delta$ between the moving robot and the fixed object by checking the penetration depth between the robot in a large number of intermediate configurations and the fixed object. The obstacle is then translated by $\delta+\epsilon$, where $\epsilon\sim\mathcal{N}(0,0.03^{2})$ represents a small perturbation. After translating the fixed object, we calculate the penetration depth as we did previously to check the collision label. During problem generation, rejection sampling is used to balance positive and negative collision labels.

Sphere Approximation. To approximate the shape of a moving object with a set of spheres, we follow strategies from cuRobo. A set of spheres is constructed by combining spheres placed inside the object from voxelized interior regions with spheres centered at uniformly sampled surface points. For this process, we can vary the size of the voxel and the number of sampled surface points as hyperparameters.

Appendix C Domain Description for Motion Planning

Dish insertion: A fixed‐base UR5 must insert three distinct dishes (sampled without replacement from six shapes) into a rack. We randomly place the rack in a collision‐free pose and sample a collision‐free robot start configuration. Hard-coded target poses specify both the end effector and the final location of each dish. After each insertion, physics is enabled to let the dish settle; once settled, it becomes a static obstacle for subsequent insertions.

Bimanual insertion: The ARMADA system, with two 6 DoF arms, performs a peg‐into‐slot assembly. We generate collision-free start and goal configurations via rejection sampling: for each trial, we sample a peg pose, grasp pose, and peg–hole shape, solve the inverse kinematics for joint angles, and discard any samples that collide.

Mining site navigation: A UR5 on a mobile base carries a pickaxe through a synthetic mining tunnel, avoiding wagons and beams. In each episode, we randomly select four meshes from eight obstacle candidates and place them at fixed scene locations. The robot’s start configuration is sampled from a bounded, collision‐free region, while the pickaxe’s goal location remains constant.

Appendix D Implementation Details in Motion Planning

To demonstrate improvements in motion planning performance, we incorporate NeuralSVCD into a trajectory optimization framework modeled after cuRobo [6]. In this framework, trajectories are parameterized using splines with a fixed number of control points. Our method adopts a two-stage optimization process. First, a particle-based solver (MPPI [29]) is employed to promote exploration, and then the trajectory is refined using L-BFGS [30].

Initially, we represent the trajectory as a spline with a fixed number of control points $\tau_{control}\in\mathbb{R}^{M\times F}$, where $M$ is a number of control points, and $F$ is a number of actuation of the robot. We then apply an optimization-based motion planner with an objective function from cuRobo but with custom collision cost:

[TABLE]

where $C_{SVCD}$ and $\alpha_{col}$ denotes SVCD cost computed using NeuralSVCD and its coefficient. For $C_{smooth}=\alpha_{vel}C_{vel}+\alpha_{acc}C_{acc}+\alpha_{jerk}C_{jerk}$, where each term indicates velocity, acceleration, and jerk minimization cost with pre-defined coefficient. For cuRobo-NeuralSVCD and cuRobo-NeuralSVCD-discrete, we modify the collision cost term in the loss function (2) by replacing it with collision logits computed by $f_{ccd}$. For each task, we use different hyperparameters. See Table 3, 4, 5. We also modify the calculating gradient during L-BFGS to be the bundled gradient [31], which combines gradient evaluations from multiple samples or time steps into a single aggregated update to improve optimization efficiency and stability, benefiting all baselines.

Appendix E Applying transformation to the latent vectors

Another critical consideration in designing latent representations and the neural SVCD decoder is the application of rigid-body transformations. During collision evaluation, each robot link must be positioned according to specific trajectory configurations, requiring transformations described by SE(3)—a combination of rotations and translations. Using explicit mesh representations, applying transformations is straightforward: one simply applies transformations directly to mesh vertices before checking for collisions. However, when using latent representations $z$ for robot links, applying SE(3) transformations is not trivial, as latent representations do not inherently support direct geometric transformations.

One option is to use neural network-based transformation operators [32], which receive both the transformation and $z$ and output the transformed $z$. However, they do not guarantee the rigid-body transformation properties required. For instance, applying a 180° rotation about the z-axis twice should reproduce the original representation, a consistency that such operators cannot reliably ensure.

To overcome these limitations, we introduce an analytical transformation operation within a high-dimensional latent space that explicitly preserves rigid transformation properties. Applying a rigid-body transformation $T\in SE(3)$ analytically involves separately treating translation and rotation. Given $T=(R,t)$, with $R\in SO(3)$ being a rotation matrix and $t\in\mathbb{R}^{3}$ a translation vector, we define the transformation of each point-latent pair as: $T\cdot(p_{i},z_{i}):=(R\cdot p_{i}+t,\;D(R)\cdot z_{i})$ where $D(R)\in\mathbb{R}^{K\times K}$ is a rotation operator applied directly within the latent space, explicitly constructed following the equivariant rotation formulation proposed in [33]. This operator $D(R)$ ensures that latent representations transform consistently with geometric rotations, preserving rigid-body properties. Intuitively, this means translations shift only the spatial coordinates $p_{i}$, while rotations alter the latent encodings $z_{i}$, effectively rotating the local geometric feature descriptions encoded within each latent vector.

Appendix F Implementation of encoder and neural SVCD decoder

For any two meshes and their corresponding moving direction $\xi$, the SVCD result is invariant under both SE(3) transformations and scaling. In other words, if a consistent transformation $T$ is applied to the meshes and the moving direction or if a uniform scale $s$ is applied, the collision detection outcome remains unchanged. We leverage this property when designing our neural CCD decoder. The inputs to the neural CCD decoder, $f_{SVCD}$, include the latent representations $z_{i}$, positions $p_{i}$, transformations $T_{i}$, and twists $\xi_{i}$ (which contain both linear and angular velocities) at critical collision times determined during the broad-phase.

Building on ideas from [33, 34, 35], we first preprocess these inputs to achieve invariance with respect to additional transformations and scaling. The preprocessing pipeline is illustrated in Figure 8. Let the combined input be $x=[\xi;z_{1};z_{2};p_{1};p_{2}]$. We define the preprocessing function as

[TABLE]

with $\phi:\mathbb{R}^{M}\to SE(3)$ such that $\phi(T\cdot x)=T\cdot\phi(x)\quad\forall T\in SE(3)$, where $M$ is the dimension of $x$ and $\lVert z\rVert$ is Euclidean norm of $z$. It is straightforward to verify that $\Phi$ is invariant under any transformation in $SE(3)$ and scale. We define $\phi$ analytically by setting its translation component to $-(p_{1}+p_{2})$ and aligning its rotation $R\in SO(3)$ with the line connecting $p_{1}$ and $p_{2}$, thereby rigidly attaching the transformation to the pair of bodies. This construction ensures the property $\phi(T\cdot x)=T\cdot\phi(x)$ holds.

The preprocessed inputs are then passed through multiple multilayer perceptrons (MLPs). The network outputs a scalar logit that represents a binary collision prediction (with positive logits indicating a collision). Finally, after predicting collisions for every $p_{i}$ and $p_{j}$ pair identified during the broad-phase, we apply max pooling to obtain the final logits indicating a collision between the rigid bodies.

The shape encoder processes an input point cloud to produce $N$ local representations $\{(z_{i},p_{i})\}_{i=1}^{N}$. The sample positions $\{p_{i}\}$ are selected via Furthest Point Sampling (FPS), and the corresponding latent vectors $\{z_{i}\}$ are learned by a neural network. Concretely, every surface point is assigned to its nearest sample $p_{i}$, and all points in each partition are fed—independently but using shared weights—through the encoder to generate the local $z_{i}$. To ensure compatibility with our latent-space transformations, we adopt the FER-VN-OccNet encoder architecture from [33].