GeoNet: Deep Geodesic Networks for Point Cloud Analysis

TL;DR

GeoNet is a novel deep learning architecture that models the intrinsic surface structure of point clouds using geodesic information, enhancing various 3D point cloud analysis tasks.

Contribution

Introduces GeoNet, the first deep network to learn surface geodesic structures from point clouds, and demonstrates its effectiveness in multiple downstream tasks.

Findings

Improves state-of-the-art in point upsampling

Enhances normal estimation accuracy

Boosts performance in mesh reconstruction and shape classification

Abstract

Surface-based geodesic topology provides strong cues for object semantic analysis and geometric modeling. However, such connectivity information is lost in point clouds. Thus we introduce GeoNet, the first deep learning architecture trained to model the intrinsic structure of surfaces represented as point clouds. To demonstrate the applicability of learned geodesic-aware representations, we propose fusion schemes which use GeoNet in conjunction with other baseline or backbone networks, such as PU-Net and PointNet++, for down-stream point cloud analysis. Our method improves the state-of-the-art on multiple representative tasks that can benefit from understandings of the underlying surface topology, including point upsampling, normal estimation, mesh reconstruction and non-rigid shape classification.

Tables

Table 1. Table 1: Neighborhood geodesic distance estimation MSE (x100) on the heldout ShapeNet training-category samples. We compare with KNN-Graph based shortest path methods under different choices of K 𝐾 K values. Euc represents the difference between Euclidean distance and geodesic distance. MSE(s) are reported under multiple radius ranges r 𝑟 r . v1 takes uniformly distributed point sets with 512 points as input, and v2 uses randomly distributed point clouds. v3 is tested using point clouds that have 2048 uniformly distributed points.

		K-3	K-6	K-12	Euc	GeoNet
v1	r $⩽$ 0.1	8.75	8.97	9.04	9.06	5.67
	r $⩽$ 0.2	16.22	17.33	17.90	18.16	9.25
	r $⩽$ 0.4	15.15	16.80	17.88	18.95	9.75
v2	r $⩽$ 0.1	11.71	11.49	11.55	11.57	7.06
	r $⩽$ 0.2	19.22	17.76	18.28	18.56	9.74
	r $⩽$ 0.4	21.03	17.19	18.20	19.44	10.04
v3	r $⩽$ 0.1	13.28	14.23	14.62	14.78	10.86
	r $⩽$ 0.2	14.85	17.27	18.54	19.49	13.61
	r $⩽$ 0.4	13.48	16.10	17.72	19.68	14.73

Table 2. Table 2: Geodesic neighborhood estimation MSE (x100) on the leftout ShapeNet categories. v1 takes uniformly distributed point sets with 512 points as input, and v2 uses randomly distributed point clouds. v3 is tested using point clouds that have 2048 uniformly distributed points.

		K-3	K-6	K-12	Euc	GeoNet
v1	r $⩽$ 0.1	8.81	9.01	9.05	9.06	7.52
	r $⩽$ 0.2	11.84	12.88	13.49	13.75	11.44
v2	r $⩽$ 0.1	10.52	10.21	10.25	10.26	8.94
	r $⩽$ 0.2	15.02	12.99	13.59	13.86	11.69
v3	r $⩽$ 0.1	11.82	12.39	12.65	12.75	10.88
	r $⩽$ 0.2	11.80	12.84	13.55	14.50	12.26

Table 3. Table 3: Point cloud upsampling results on both the heldout training-category samples and the unseen ShapeNet categories. MSE(s) (x10000) are scaled for better visualization.

		MSE	EMD	CD
Training	PU-Net	7.14	8.06	2.72
	PUF	6.23	7.62	2.46
Leftout	PU-Net	12.38	11.43	3.98
	PUF	9.55	8.90	3.27

Table 4. Table 4: Point cloud normal estimation accuracy (%) on the Shrec15 dataset under multiple angle thresholds.

	$⩽$ 2.5∘	$⩽$ 5∘	$⩽$ 10∘	$⩽$ 15∘
PCA	6.16$\pm$0.01	14.85$\pm$0.02	27.16$\pm$0.17	34.17$\pm$0.28
PointNet++	12.81$\pm$0.18	33.37$\pm$0.92	61.58$\pm$2.02	75.49$\pm$1.95
POF	16.26$\pm$0.30	39.02$\pm$1.09	66.98$\pm$1.46	79.66$\pm$1.21

Table 5. Table 5: Point cloud normal estimation accuracy (%) on the ShapeNet dataset for both heldout training-category samples and leftout categories.

		$⩽$ 2.5∘	$⩽$ 5∘	$⩽$ 10∘	$⩽$ 15∘
Training	PCA	5.33	10.11	18.52	24.82
	PointNet++	30.68	43.19	55.91	62.30
	POF	32.04	45.02	57.52	63.62
Leftout	PCA	5.24	10.59	18.99	25.17
	PointNet++	17.35	28.82	43.26	51.17
	POF	19.13	31.83	46.22	53.78

Table 6. Table 6: Point cloud classification of non-rigid shapes on the Shrec15 dataset.

	Input feature	Accuracy (%)
PointNet++	XYZ	73.56
POF	XYZ	94.67
DeepGM	Intrinsic features	93.03

Table 7. Table 7: Noisy point clouds classification accuracy (%). We add Gaussian noise of 0.8% to 1.2% of unit ball radius.

	Gaussian Noise Level
	0.8%	0.9%	1.0%	1.1%	1.2%
PointNet++	70.54	69.27	67.83	65.66	62.38
POF	91.89	90.93	89.40	87.72	84.98

Equations

$$\begin{array}{r}{L}_{ge{o}_l}=\sum _{x_i\in \chi }\sum _{x_j\in B_{r_l}(x_i)}\frac{\left|g_{ij}-\hat{g}(x_i,x_j)\right|}{N{K}_l}\end{array}$$

$$\begin{array}{r}L=L_{geo}+\lambda L_{task}\end{array}$$

$$\begin{array}{r}{L}_{up}(\theta )=L_{EMD}(P,\hat{P})+{\lambda }_1{L}_{rep}(\hat{P})+{\lambda }_2{\Vert \theta \Vert }^2\end{array}$$

$$\begin{array}{r}{L}_{EMD}(P,\hat{P})=\underset{\varphi :\hat{P}\to P}{\min }\sum _{p_i\in \hat{P}}{\Vert p_i-\varphi (p_i)\Vert }^2\end{array}$$

$$\begin{array}{r}{L}_{rep}(\hat{P})=\sum _{p_i\in \hat{P}}\sum _{p_j\in {\tilde{P}}_i}\eta (\Vert p_i-p_j\Vert )\omega (\Vert p_i-p_j\Vert )\end{array}$$

$$\begin{array}{r}{L}_{normal}=\sum _{x_i\in \chi }\sum _{j=1}^3\frac{\left|n_i^{(j)}-\hat{n}(x_i{)}^{(j)}\right|}{3N}\end{array}$$

$$\begin{array}{r}{L}_{cls}=-\sum _{c=1}^S{y}_{c}log(p_c(\chi ))\end{array}$$

Full text

1 Introduction

Determining neighborhood relationship among points in a point cloud, known as topology estimation, is an important problem since it indicates the underlying point cloud structure, which could further reveal the point cloud semantics and functionality. Consider the red inset in Fig. 1: the two clusters of points, though seemingly disconnected, should indeed be connected to form a chair leg, which supports the whole chair. On the other hand, the points on opposite sides of a chair seat, though spatially very close to each other, should not be connected to avoid confusing the sittable upper surface with the unsittable lower side. Determining such topology appears to be a very low-level endeavor but in reality it requires global, high-level knowledge, making it a very challenging task. Still, from the red inset in Fig. 1, we could draw the conclusion that the two stumps are connected only after we learn statistical regularities from a large number of point clouds and observe many objects of this type with connected elongated vertical elements extending from the body to the ground. This motivates us to adopt a learning approach to capture the topological structure within point clouds.

Our primary goals in this paper are to develop representations of point cloud data that are informed by the underlying surface topology as well as object geometry, and propose methods that leverage the learned topological features for geodesic-aware point cloud analysis. The representation should capture various topological patterns of a point cloud and the method of leveraging these geodesic features should not alter the data stream, so our representation can be learned jointly and used in conjunction with the state-of-the-art baseline or backbone models (e.g. PU-Net, PointNet++ [37, 27, 28]) that feed the raw data through, with no information loss to further stages of processing.

For the first goal, we propose a geodesic neighborhood estimation network (GeoNet) to learn deep geodesic representations using the ground truth geodesic distance as supervision signals. As illustrated in Fig. 2, GeoNet consists of two modules: an autoencoder that extracts a feature vector for each point and a geodesic matching (GM) layer that acts as a learned kernel function for estimating geodesic neighborhoods using the latent features. Due to the supervised geodesic training process, intermediates features of the GM layer contain rich information of the point cloud topology and intrinsic surface attributes. We note that the representation, while trained on geodesic distances, does not by construction produce geodesics (e.g. symmetry, triangle inequality, etc.). The goal of the representation is to inform subsequent stages of processing of the global geometry and topology, and is not to conduct metric computations directly.

For the second task, as shown in Fig. 3, we propose geodesic fusion schemes to integrate GeoNet into the state-of-the-art network architectures designed for different tasks. Specifically, we present PU-Net fusion (PUF) for point cloud upsampling, and PointNet++ fusion (POF) for normal estimation, mesh reconstruction as well as non-rigid shape classification. Through experiments, we demonstrate that the learned geodesic representations from GeoNet are beneficial for both geometric and semantic point cloud analyses.

In summary, in this paper we propose an approach for learning deep geodesic-aware representations from point clouds and leverage the results for various point set analyses. Our contributions are:

• •

We present, to the best of our knowledge, the first deep learning method, GeoNet, that ingests point clouds and learns representations which are informed by the intrinsic structure of the underlying point set surfaces.

• •

To demonstrate the applicability of learned geodesic representations, we develop network fusion architectures that incorporate GeoNet with baseline or backbone networks for geodesic-aware point set analysis.

• •

Our geodesic fusion methods are benchmarked on multiple geometric and semantic point set tasks using standard datasets and outperform the state-of-the-art methods.

2 Related work

We mainly review traditional graph-based methods for geodesic distance computation, as well as general works on point cloud upsampling, normal estimation, and non-rigid shape classification, as we are unaware of other prior works on point cloud-based deep geodesic representation learning.

Geodesic distance computation. There are two types of methods: some allow the path to traverse mesh faces [30, 26, 6, 12, 32, 35, 7] for accurate geodesic distance computation, while others find approximate solutions via shortest path algorithms constrained on graph edges [10, 11, 19]. For the first type, an early method [30] suggests a polynomial algorithm of time $O(n^{3}logn)$ where $n$ is the number of edges, but their method is restricted to a convex polytope. Based on Dijkstra’s algorithm [10], [26] improves the time complexity to $O(n^{2}logn)$ and extends the method to an arbitrary polyhedral surface. Later, [6] proposes an $O(n^{2})$ approach using a set of windows on the polyhedron edges to encode the structure of the shortest path set. By filtering out useless windows, [35] further speeds up the algorithm. Then [7] introduces a heat method via solving a pair of standard linear elliptic problems. As for graph edge-based methods, typical solutions include Dijkstra’s [10], Floyd-Warshall [11] and Johnson’s algorithms [19], which have much lower time complexity than the surface traversing methods. For a 20000-vertex mesh, computing its all-pair geodesic distances can take several days using [35] while [19] only uses about 1 minute on CPU. When a mesh is dense, the edge-constrained shortest path methods generate low-error geodesic estimates. Thus in our work, we apply [19] to compute the ground truth geodesic distance.

Point upsampling. Previous methods can be summarized into two categories. i) Optimization based methods [1, 23, 18], championed by [1], which interpolates a dense point set from vertices of a Voronoi diagram in the local tangent space. Then [23] proposes a locally optimal projection (LOP) operator for point cloud resampling and mesh reconstruction leveraging an $L_{1}$ median. For improving robustness to point cloud density variations, [18] presents a weighted LOP. These methods all make strong assumptions, such as surface smoothness, and are not data-driven, and therefore have limited applications in practice. ii) Deep learning based methods. To apply the (graph) convolution operation, many of those methods first voxelize a point cloud into regular volumetric grids [34, 33, 16, 8] or instead use a mesh [9, 36]. While voxelization introduces discretization artifacts and generates low resolution voxels for computational efficiency, mesh data can not be trivially reconstructed from a sparse and noisy point cloud. To directly upsample a point cloud, PU-Net [37] learns multilevel features for each point and expands the point set via a multibranch convolution unit implicitly in feature space. But PU-Net is based on Euclidean space and thus does not leverage the underlying point cloud surface attributes in geodesic space, which we show in this paper are important for upsampling.

Normal estimation. A widely used method for point cloud normal estimation is to analyze the variance in a tangential plane of a point and find the minimal variance direction by Principal Component Analysis (PCA) [17, 20]. But this method is sensitive to the choice of the neighborhood size, namely, large regions can cause over-smoothed surfaces and small ones are sensitive to noises. To improve robustness, methods based on fitting higher-order shapes have been proposed [13, 4, 2]. However, these methods require careful parameter tuning at the inference time and only estimate normal orientation up to sign. Thus, so far robust estimation for oriented normal vectors using traditional methods is still challenging, especially across different noise levels and shape structures. There are only few data-driven methods that are able to integrate normal estimation and orientation alignment into a unified pipeline [14, 28]. They take a point cloud as input and directly regress oriented normal vectors, but these methods are not designed to learn geodesic topology-based representations that capture the intrinsic surface features for better normal estimation.

Non-rigid shape classification. Classifying the point cloud of non-rigid objects often consists of two steps: extracting intrinsic features in geodesic space and applying a classifier (e.g. SVM, MLP, etc.). Some commonly used features include wave kernel signatures [3], heat kernel signatures [31], spectral graph wavelet signatures [25], Shape-DNA [29], etc. For example [24] uses geodesic moments and stacked sparse autoencoders to classify non-rigid shapes, such as cat, horse, spider, etc. The geodesic moments are feature vectors derived from the integral of the geodesic distance on a shape, while stacked sparse autoencoders are deep neural networks consisting of multiple layers of sparse autoencoders. However, the above methods all require knowing graph-based data, which is not available from widely used sensors (e.g. depth camera, Lidar, etc.) for 3D data acquisition. Though PointNet++ [28] is able to directly ingest a point cloud and conduct classification, it is not designed to model the geodesic topology of non-rigid shapes and thus its performance is inferior to traditional two-step methods which heavily reply on the offline computed intrinsic surface features.

3 Method

3.1 Problem Statement

$\chi=\{x_{i}\}$ denotes a point set with $x_{i}\in{\mathbb{R}}^{d}$ and $i=1,\dots,N$. Although the problem and the method developed are general, we focus on the case $d=3$ using only Euclidean coordinates as input. A neighborhood subset within radius $r$ from a point $x_{i}$ is denoted $B_{r}(x_{i})=\{x_{j}|d_{E}(x_{i},x_{j})\leq r\}$ where $d_{E}(x_{i},x_{j})\in{\mathbb{R}}$ is the Euclidean (embedding) distance between $x_{i}$ and $x_{j}$. The cardinality of $B_{r}(x_{i})$ is $K$. The corresponding geodesic distance set around $x_{i}$ is called $G_{r}(x_{i})=\{g_{ij}=d_{G}(x_{i},x_{j})|x_{j}\in B_{r}(x_{i})\}$ where $d_{G}\in{\mathbb{R}}$ means the geodesic distance. Our goal is to learn a function $f:x_{i}\mapsto G_{r}(x_{i})$ that maps each point to (an approximation of) the geodesic distance set $G_{r}(x_{i})$ around it.

3.2 Method

We introduce GeoNet, a network trained to learn the function $f$ defined above. It consists of an autoencoder with skip connections, followed by a multi-scale Geodesic Matching (GM) layer, leveraging latent space features $\{\psi(x_{i})\}\subseteq{\mathbb{R}^{3+C}}$ of the point set. GeoNet is trained in a supervised manner using ground truth geodesic distances between points in the set $\chi$. To demonstrate the applicability of learned deep geodesic-aware representations from GeoNet, we test our approach on typical tasks that require understandings of the underlying surface topology, including point cloud upsampling, surface normal estimation, mesh reconstruction, and non-rigid shape classification. To this end, we leverage the existing state-of-the-art network architectures designed for the aforementioned problems. Specifically, we choose PU-Net as the baseline network for point upsampling and PointNet++ for other tasks. The proposed geodesic fusion methods, called PU-Net fusion (PUF) and PointNet++ fusion (POF), integrate GeoNet with the baseline or backbone models to conduct geodesic-aware point set analysis.

3.3 Geodesic Neighborhood Estimation

As illustrated in Fig. 2, GeoNet consists of two modules: an autoencoder that extracts a feature vector $\psi(x_{i})$ for each point $x_{i}\in\chi$ and a GM layer that acts as a learned geodesic kernel function for estimating $G_{r}(x_{i})$ using the latent features.

Feature Extraction. We use a variant of PointNet++, which is a point set based hierarchical and multi-scale function, for feature extraction. It maps an input point set $\chi$ to a feature set $\{\varphi(x_{i})|x_{i}\in\widetilde{\chi}\}$ where $\varphi(x_{i})\in{\mathbb{R}^{3+\widetilde{C}}}$ is a concatenation of the $xyz$ coordinates and the $\widetilde{C}$ dimensional embedding of $x_{i}$, and $\widetilde{\chi}$ is a sampled subset of $\chi$ by farthest-point sampling. To recover features $\{\psi(x_{i})\}$ for the point cloud $\chi$, we use a decoder with skip connections. The decoder consists of recursively applied tri-linear feature interpolators, shared fully connected (FC) layers, ReLU and Batch Normalization. The resulting $(N,3+C)$ tensor is then fed into the GM layer for geodesic neighborhood estimation.

Geodesic Matching. We group the latent features $\psi(x_{i})$ into neighborhood feature sets $F_{r_{l}}(x_{i})=\{\psi(x_{j})|x_{j}\in B_{r_{l}}(x_{i})\}$, under multiple radius scales $r_{l}$. At each scale $r_{l}$ we set a maximum number of neighborhood points $K_{l}$, and thus produce a tensor of dimension $(N,K_{l},3+C)$. The grouped features, together with the latent features, are sent to a geodesic matching module, where $\psi(x_{i})$ is concatenated with $\psi(x_{j})$ for every $x_{j}\in B_{r_{l}}(x_{i})$. The resulting feature $\xi_{ij}\in{\mathbb{R}^{3+2C}}$ becomes the input to a set of shared FC layers with ReLU, Batch Normalization and Dropout. As demonstrated in [15], the multilayer perceptron (MLP) acts as a kernel function that maps $\xi_{ij}$ to an approximation of the geodesic distance, $\hat{g}_{ij}$. Finally, the GM layer yields $G_{r_{l}}(x_{i})$ for each point of the input point cloud $\chi$. We use a multi-scale $L_{1}$ loss $L_{geo}=\sum_{l}L_{geo_{l}}$ to compare the ground truth geodesic distances to their estimates:

[TABLE]

3.4 Geodesic Fusion

To demonstrate how the learned geodesic representations can be used for point set analysis, we propose fusion methods based on the state-of-the-art (SOTA) network architectures for different tasks. For example, PU-Net is the SOTA upsampling method and thus we propose PUF that uses PU-Net as the baseline network to conduct geodesic fusion for point cloud upsampling. With connectivity information provided by the estimated geodesic neighborhoods, our geodesic-fused upsampling network can better recover topological details, such as curves and sharp structures, than PU-Net. We also present POF leveraging PointNet++ as the fusion backbone, and demonstrate its effectiveness on both geometric and semantic tasks where PointNet++ shows the state-of-the-art performance.

PU-Net Geodesic Fusion. A PUF layer, as illustrated in Fig. 3 (top), takes a $(N,d)$ point set as input and sends it into two branches: one is a multi-scale Euclidean grouping layer, and the other is GeoNet. At each neighborhood scale $r_{l}$, the grouped point set $B_{r_{l}}(x_{i})$ is fused with the estimated geodesic neighborhood $G_{r_{l}}(x_{i})$ to yield $S_{r_{l}}(x_{i})=\{(x_{j},g_{ij})|x_{j}\in B_{r_{l}}(x_{i})\}$ with $(x_{j},g_{ij})\in{\mathbb{R}^{d+1}}$. Then the $(N,K_{l},d+1)$ fused tensor is fed to a PointNet to generate a $(N,C_{l})$ feature tensor which will be stacked with features from other neighborhood scales. The remaining layers are from PU-Net. As indicated by the red rectangles in Fig. 3, the total loss has two weighted terms:

[TABLE]

where $L_{geo}$ is for GeoNet training (1), $\lambda$ is a weight and $L_{task}$, in general, is the loss for the current task that we are targeting. In this case, the goal is point cloud upsampling: $L_{task}=L_{up}(\theta)$ where $\theta$ indicates network parameters. PUF upsampling takes a randomly distributed sparse point set $\chi$ as input and generates a uniformly distributed dense point cloud $\hat{P}\subseteq{\mathbb{R}^{3}}$. The upsampling factor is $\alpha=\frac{\left|P\right|}{\left|\chi\right|}$:

[TABLE]

in which the first term is the Earth Mover Distance (EMD) between the upsampled point set $\hat{P}$ and the ground truth dense point cloud $P$:

[TABLE]

where $\phi:\hat{P}\rightarrow P$ indicates a bijection mapping.

The second term in (3) is a repulsion loss which promotes a uniform spatial distribution for $\hat{P}$ by penalizing close point pairs:

[TABLE]

where $\widetilde{P}_{i}$ is a set of k-nearest neighbors of $p_{i}$, $\eta(r)=-r$ penalizes close pairs $(p_{i},p_{j})$, and $\omega(r)=e^{-r^{2}/h^{2}}$ is a fast-decaying weight function with some constant $h$ [18, 23].

PointNet++ Geodesic Fusion. Fig. 3 (bottom) illustrates the PointNet++ based fusion pipeline. Due to task as well as architecture differences between PU-Net and PointNet++, we make following changes to PUF to design a suitable fusion strategy that leverages PointNet++. First, for multi-scale grouping, we use the learned geodesic neighborhoods $\hat{G}_{r}(x_{i})$ instead of Euclidean ones. Geodesic grouping brings attention to the underlying surfaces as well as structures of the point cloud. Second, while the PUF layer fuses estimated $\hat{G}_{r}(x_{i})=\{\hat{g}_{ij}=\hat{d}_{G}(x_{i},x_{j})|x_{j}\in B_{r}(x_{i})\}$, where $\hat{g}_{ij}\in{\mathbb{R}}$, of each neighborhood point set $B_{r}(x_{i})$ into the backbone network, the POF layer uses the latent geodesic-aware features $\widetilde{\xi}_{ij}\in{\mathbb{R}^{\widetilde{C}}}$ extracted from the second-to-last FC layer in GeoNet. Namely, $\widetilde{\xi}_{ij}$ is an intermediate high-dimensional feature vector from $\xi_{ij}$ to $\hat{g}_{ij}$ via FC layers, and therefore it is better informed of the intrinsic point cloud topology. Third, in PointNet++ fusion we apply the POF layer in a hierarchical manner, leveraging farthest-point sampling. Thus, the learned features encode both local and global structural information of the point set. The total loss for POF also has two parts: One is for GeoNet training and the other is for the task-at-hand. We experiment on representative tasks that can benefit from understandings of the topological surface attributes. We use the $L_{1}$ error for point cloud normal estimation:

[TABLE]

in which $n_{i}\in{\mathbb{R}^{3}}$ is the ground truth unit normal vector of $x_{i}$, and $\hat{n}(x_{i})\in{\mathbb{R}^{3}}$ is the estimated normal. We then use the normal estimation to generate mesh via Poisson surface reconstruction [21]. To classify point clouds of non-rigid objects, we use cross-entropy loss:

[TABLE]

where $S$ is the number of non-rigid object categories, and $c$ is class label; $y_{c}\in\{0,1\}$ is a binary indicator, which takes value $1$ if class label $c$ is ground truth for the input point set. $p_{c}(\chi)\in{\mathbb{R}}$ is the predicted probability w.r.t. class $c$ of the input point set.

3.5 Implementation

For GeoNet training, the multiscale loss $L_{geo_{l}}$ is enforced at three radius ranges: $0.1$, $0.2$ and $0.4$. We use Adam [22] with learning rate 0.001 and batchsize 3 for 8 epochs. To train the geodesic fusion networks, we set the task term weight $\lambda$ as 1, and use Adam with learning rate 0.0001 and batchsize 2 for around 300 to 1500 epochs depending on the task and the dataset. Source code in Tensorflow will be made available upon completion of the anonymous review process.

4 Experiments

We put GeoNet to the test by estimating point cloud geodesic neighborhoods. To demonstrate the applicability of learned deep geodesic-aware representations, we also conduct experiments on down-stream point cloud tasks such as point upsampling, normal estimation, mesh reconstruction and non-rigid shape classification.

4.1 Geodesic Neighborhood Estimation

In Tab. 1 (v1) we show geodesic distance set, $G_{r}(x_{i})$, estimation results on the ShapeNet dataset [5] using point clouds with 512 uniformly distributed points. Mean-squared errors (MSE) are reported under multiple radius scales $r$ w.r.t. $x_{i}\in\chi$. GeoNet demonstrates consistent improvement over the baselines. Representative results are visualized in Fig. 4. Our method captures various topological patterns, such as curved surfaces, layered structures, inner/outer parts, etc.

Generality. We test GeoNet’s robustness under different point set distributions and sizes. In Tab. 1 (v2) we use point clouds with 512 randomly distributed points as input. We also test on dense point sets that contain 2048 uniformly distributed points in Tab. 1 (v3). Our results are robust to different point set distributions as well as sizes. To show the generalization performance, in Tab. 2 we report results on the leftout ShapeNet categories. Our method performs better on unseen categories, while KNN-Graph based shortest path approaches suffer from point set distribution randomness, density changes and unsuitable choices of K values.

4.2 Point Cloud Upsampling

We test PUF on point cloud upsampling and present results in Tab. 3. We compare against the state-of-the-art point set upsampling method PU-Net on three metrics: MSE, EMD as well as the Chamfer Distance (CD). Our method outperforms the baseline under all metrics by 9.25% average improvement on the heldout training-category samples. Since geodesic neighborhoods are better informed of the underlying point set topology than Euclidean ones, PUF upsampling produces less outliers and recovers more details in Fig. 5, such as curves and sharp structures.

Generality. To analyze outlier robustness (i.e. points with large reconstruction errors), we plot top-k MSE in Fig. 6. Our method generates fewer outliers on both the heldout training-category samples and the unseen categories. We also report quantitative results on the leftout categories in Tab. 3. Again, PUF significantly surpasses the state-of-the-art upsampling method PU-Net under three different evaluation metrics.

4.3 Normal Estimation and Mesh Reconstruction

For normal estimation we apply PointNet++ geodesic fusion, POF, then we conduct Poisson mesh reconstruction leveraging the estimated normals. Quantitative results for normal estimation on the Shrec15 dataset and the ShapeNet dataset are given in Tab. 4 and Tab. 5, respectively. We compare our method with the traditional PCA algorithm as well as the state-of-the-art deep learning method PointNet++. Our results outperform the baselines by around 10% relative improvement. In Fig. 8, we visualize typical normal estimation errors, showing that PointNet++ usually fails at high-curvature and complex-surface regions. For further evidence, we visualize Poisson mesh reconstruction in Fig. 7 using the estimated normals.

Generality. In Tab. 5 we evaluate normal estimation performance on the leftout ShapeNet categories. Our method has higher accuracy over competing methods under multiple angle thresholds. Though trained with point clouds of 2048 points, POF is also tested on denser input. In Fig. 9 we take point clouds with 8192 points as input, and visualize the normal estimation and mesh reconstruction results, which shows that our method generalizes to dense point clouds without re-training and produces fine-scaled mesh.

4.4 Non-rigid Shape Classification

Results of non-rigid shape classification are reported in Tab. 6. While POF and PointNet++ only take point cloud-based $xyz$ Euclidean coordinates as input, DeepGM requires offline computed intrinsic features from mesh data in the ground truth geodesic metric space. Though using less informative data, our method has higher classification accuracy than other methods, which further demonstrates that the proposed geodesic fusion architecture, POF, is suitable for solving tasks that require understandings of the underlying point cloud surface attributes.

Generality. We add Gaussian noise of different levels to the input and conduct noisy point clouds classification. Comparisons are shown in Tab. 7. POF outperforms PointNet++ under several noise levels. Our method also demonstrates better noise robustness. It shows a 10.24% decrease in relative accuracy at the maximum noise level, while PointNet++ decreases by up to 15.20%.

4.5 Failure Modes

Failure cases of geodesic neighborhood estimation are shown in Fig. 10. Due to large ratios between length and width/height, after normalizing a stick-shaped object (e.g. rocket, knife, etc.) into a unit ball we need high precision small values to represent its point-pair geodesic distance along the width/height sides. Since stick-shaped objects like rocket and knife only take up a small portion of the training data, GeoNet tends to make mistakes for heldout samples from these categories at inference time. We have not found additional failure cases, and quantitative improvements continue to take effect due to rich surface-based topological information learned during the geodesic-supervised training process.

5 Conclusion

We have presented GeoNet, a novel deep learning architecture to learn the geodesic space-based topological structure within point clouds. The training process is supervised by the ground truth geodesic distance and therefore the learned representations reflect the intrinsic structure of the underlying point set surfaces. To demonstrate the applicability of such a topology estimation network, we also propose fusion methods to incorporate GeoNet into computational schemes that involve the standard backbone architectures for point cloud analysis. Our method is tested on both geometric and semantic tasks and outperforms the state-of-the-art methods, including point upsampling, normal estimation, mesh reconstruction and non-rigid shape classification.