Parametric Shape Modeling and Skeleton Extraction with Radial Basis Functions using Similarity Domains Network

TL;DR

This paper introduces a novel approach using Similarity Domains Networks with radial basis functions for shape modeling and skeleton extraction from images, demonstrating the effectiveness of SDs in neural network frameworks.

Contribution

It presents a new method combining SDs and RBFs within neural networks for shape analysis and skeleton extraction, advancing shape modeling techniques.

Findings

SDNs effectively model pixel-based images with SDs

Learned SDs can accurately extract shape skeletons

The approach enhances shape analysis with neural networks

Abstract

We demonstrate the use of similarity domains (SDs) for shape modeling and skeleton extraction. SDs are recently proposed and they can be utilized in a neural network framework to help us analyze shapes. SDs are modeled with radial basis functions with varying shape parameters in Similarity Domains Networks (SDNs). In this paper, we demonstrate how using SDN can first help us model a pixel-based image in terms of SDs and then demonstrate how those learned SDs can be used to extract the skeleton of a shape.

Tables

Table 1. Table 1 : Bin centers for the quantized foreground shape parameters ( σ i 2 subscript superscript 𝜎 2 𝑖 \sigma^{2}_{i} ) and the total number of shape parameters that fall in each bin for the image in Fig.  3(a) .

Bin Center:	9.93	29.12	48.32	67.51	86.71	105.90	125.09	144.29	163.48	182.68
Total Counts:	591	18	7	3	2	4	0	0	1	3

Equations

$$\overline{y}=sign(f(\mathbf{x}))\text{ and }f(\mathbf{x})=\sum _{i=1}^k{\alpha }_i{y}_i{K}_{\sigma i}(\mathbf{x},{\mathbf{x}}_{\mathbf{i}}),$$

$$K_{\sigma i}(\mathbf{x},{\mathbf{x}}_{\mathbf{i}})=\exp (-\parallel \mathbf{x}-{\mathbf{x}}_{\mathbf{i}}{\parallel }^2/{\sigma }_i^2)$$

$$\underset{\alpha }{\max }Q(\alpha )=\sum _{i=1}^n{\alpha }_i-\frac{1}{2}\sum _{i=1}^n\sum _{j=1}^n{\alpha }_i{\alpha }_j{y}_i{y}_j{K}_{\sigma ij}({\mathbf{x}}_{\mathbf{i}},{\mathbf{x}}_{\mathbf{j}}),$$

$$\text{subject to:}\sum _{i=1}^n{\alpha }_i{y}_i=0,C\ge {\alpha }_i\ge 0\text{ for }i=1,2,...,n,$$

$$\text{ and }{K}_{\sigma ij}({\mathbf{x}}_{\mathbf{i}},{\mathbf{x}}_{\mathbf{j}})<T,\text{ if }{y}_i{y}_j=-1,\forall i,j$$

$$\overline{y}=+1,if\parallel \mathbf{x}-{\mathbf{x}}_{\mathbf{i}}\parallel <\sqrt{a{\sigma }_i^2},\exists {\mathbf{x}}_{\mathbf{i}}\in S_1$$

$$otherwise\overline{y}=-1,$$

Full text

1 Introduction

Recent advances in deep learning moved attention to the neural networks based solutions for shape understanding, shape analysis and parametric shape modeling. Radial basis networks (RBNs) are a particular set of neural networks using radial basis function (RBF) kernels and in this paper, we introduce a novel shape modeling algorithm based on RBNs. RBFs have been used in the literature for many classification tasks including the original LeNET architecture [12]. While RBFs are useful in modeling surfaces and classification tasks as in [18, 11, 22, 5, 17, 15], there are many challenges associated with utilizing RBFs in neural networks for parametric shape modeling. Two of those challenges include: (I) estimating the optimal number of RBFs (e.g., the number of circles in our figures) to be used in the network along with their optimal center values, and (II) estimating the optimal RBF kernel parameters by relating them to shapes geometrically. The kernel parameters are typically known as the scale or the shape parameter (representing the radius of a circle in this paper) and used interchangeably in the literature. The standard RBNs as defined in [13] applies the same kernel parameter to each and all basis functions used in the architecture. Recent literature focused on using multiple kernels with their own kernel parameters as in [9] and [1]. While the idea of utilizing different kernels with different parameters has been heavily studied in the literature under the ”Multiple Kernel Learning” (MKL) framework as formally modeled in [1], there are not many efficient approaches and available implementations focusing on utilizing multiple kernels with their own parameters in RBNs for shape modeling. Recently, the work in [16] combined the optimization advances achieved in the kernel machines domain with the radial basis networks and introduced a novel algorithm for shape analysis. In this paper, we call that algorithm as ”Similarity Domains Network” (SDN) and discuss its benefits from both shape analysis (see Figure 1) and skeleton extraction perspectives. As we demonstrate in this paper, the computed SDs of SDN can be used to obtain both parametric models for shapes via its SDs and their skeletons without requiring large training samples.

2 Related Work

In this paper, we propose using SDs for both parametric shape modeling and for extracting the skeleton. Our proposed algorithm: SDN is related to both RBNs and kernel machines. Skeleton extraction has been widely studied in the literature as in [7, 21, 20, 8]. However, in this paper, we mainly discuss and present our novel algorithm from the RBNs perspective. In the past, the RBN related research mostly focused on computing the optimal single kernel parameter (i.e., the scale or shape parameter) to be used in all of the RBFs used in the network as in [14, 4]. While the parameter computation for multiple kernels have been heavily studied under the MKL framework in the literature (for examples, see the survey papers: [6, 10]), the computation of multiple kernel parameters in RBNs has been mostly studied under two main approaches: using optimization or using heuristic methods. For example, in [3], the authors proposed using multiple scales as opposed to using a single scale value in RBNs. Their approach utilizes first computing the standard deviation of each cluster (after applying a k-means like clustering on the data) and then using a scaled version of those standard deviations of each cluster as the shape parameter for each RBF in the network. The work in [2] also used a similar approach by using the root-mean-square-deviation (RMSD) value between the RBF centers and the data value for each RBF in the network. The authors used a modified orthogonal least squares (OLS) algorithm to select the RBF centers. The work in [9] used k-means algorithm on the training data to choose k centers and used those centers as RBF centers. Then it used separate optimizations for computing the kernel parameters and the kernel weights (see next chapter for the formal definitions). Using additional optimization steps for different set of parameters is costly and makes it harder to interpret those parameters and to relate them to shapes geometrically and accurately. As an alternative solution, the work in [16] proposed a geometric approach by using the distance between the data samples as a geometric constraint. In [16], the author did not use the well known MKL model. Instead, he defined interpretable similarity domains concept using RBFs and developed his own optimization approach with geometric constrains similar to the original Sequential Minimal Optimization (SMO) algorithm [19]. Consequently, the SDN algorithm combines both RBN and kernel machine concepts to develop a novel algorithm with geometrically interpretable kernel parameters. In this paper, we propose using SDN for parametric shape modeling and skeleton extraction. Unlike the existing work, instead of applying an initial k-means algorithm or OLS algorithm to compute the kernel centers separately or using multiple cost functions, SDN chooses the RBF centers and their numbers automatically via its sparse modeling and uses a single cost function to be optimized with its geometric constraint. That is where SDN differs from other similar RBN works as they would have issues on computing all those parameters within a single optimization step while automatically adjusting the number of RBFs used in the network sparsely.

3 Similarity Domains Network

RBNs typically include a single hidden layer using radial basis functions as activation functions and the hidden layer uses $n$ different RBFs. The illustration of SDN as a radial basis network is given in Figure 2. In the figure, the hidden layer uses all of the $n$ training data as an RBF center and then through the sparse optimization, it selects a subset of the training data (e.g., subset of pixels for shape modeling). SDN represents the decision boundary as a weighted combination of Similarity Domains (SDs). A Similarity Domain is a $d$ dimensional sphere in the $d$ dimensional feature space. Each similarity domain is centered at an RBF center and modeled with a Gaussian RBF in SDN. SDN estimates the label $y$ of a given input vector x as $\overline{y}$ as shown below:

[TABLE]

where the scalar $\alpha_{i}$ is a nonzero weight fo the RBF center ${\bf{x}_{i}}$, $y_{i}\epsilon\{-1,+1\}$ the class label of the training data and $k$ the total number of RBF centers. $K$(.) is the Gaussian RBF kernel defined as:

[TABLE]

where $\sigma_{i}$ is the shape parameter for the center ${\bf{x}_{i}}$. The centers are automatically selected among the training data during the training via the following cost function:

[TABLE]

where $T$ is a constant value assuring that the RBF function yields a smaller value for any given pair of samples from different classes. The shape parameter $\sigma{ij}$ is defined as $\sigma{ij}=min(\sigma_{i},\sigma_{j})$. Further details on SDs and SDN formulation can be found in [16].

4 Parametric Shape Modeling with SDN

The Gaussian RBFs and their shape parameters can be used for parametric modeling of the shapes. For that, we can save and use only the foreground (the shape’s) centers and their shape parameters to obtain a one class classifier. The computed centers of SDN can be grouped as $C_{1}=\bigcup\limits_{i=1,y_{i}\in{+1}}^{s_{1}}\mathbf{x_{i}}$ and $C_{2}=\bigcup\limits_{i=1,y_{i}\in{-1}}^{s_{2}}\mathbf{x_{i}}$, where $s_{1}+s_{2}=k$, $s_{1}$ is the total number of centers from the (+1) class and $s_{2}$ is the total number of centers from the (-1) class. Since the Gaussian kernel functions now represent local SDs geometrically, the original decision function $f(\mathbf{x})$ can now be approximated by using only $C_{1}$ (or by using only $C_{2}$). Therefore, we define the one-class approximation by using only the centers and their associated kernel parameters from the $C_{1}$ for any given $\mathbf{x}$ as follows:

[TABLE]

where the SD radius for the $i^{th}$ center $\mathbf{x_{i}}$ is defined as $\sqrt{a\sigma^{2}_{i}}$ and $a$ is a domain specific constant. One class approximation examples are given in Figure 1(b) where we used only the SDs from the foreground to reconstruct the altered image.

5 Extracting the Skeleton from SDs

Once learned and computed by the SDN, the Similarity Domains (SDs) can be used to obtain a representation of a shape’s skeleton. For that purpose, we first bin the computed shape parameters ($\sigma^{2}_{i}$) into $m$ bins (in our experiments $m$ is set to 10). Since typically the majority of the similarity domains lay around the object (or shape) boundary, they appear in small values. Eliminating them at first, gives us a lesser number of SDs to consider for skeleton extraction. After eliminating those small SDs and their computed parameters with a simple thresholding process, we connect the centers of the remaining SDs by tracing the overlapping SDs. In the case of remaining non-overlapping SDs, we connect the closest SDs.

6 Experiments

Here, we demonstrate how to use SDN for parametric shape learning from a given single input image. Since it is hard to model shapes with the standard RBNs, and since there is no good RBN implementation was available to us, we did not use any RBN network in our experiments. The standard RBNs (as discussed earlier) have many issues and many individual steps to compute the RBN parameters including the total number of RBF centers and finding the center values along with the computation of the shape parameters at those centers. However, comparison of kernel machines (SVM) and SDN on shape modeling was already studied in the literature before (see [16]). Therefore, in this section, we focus on parametric shape modeling and skeleton extraction from SDs by using SDNs. All the images are resized to fit into the figures.

6.1 Parametric Shape Modeling with SDs

We first demonstrate visualizing the computed shape parameters of SDN on a sample image in Figure 3. Figure 3(a) shows the original input image. We used each image pixel’s 2D coordinate as the training input, and its color (being black or white) as the training labels. SDN is trained at T=0.05. SDN learned and modeled the shape and reconstructed it with zero pixel error by using 1393 SDs. Pixel error is the total number of wrongly classified pixels in the image. Figure 3(b) visualizes all the computed shape parameters of the RBF centers of SDN as circles and Figure 3(c) visualizes the ones for the foreground only. The radius of a circle in all figures is computed as $\sqrt{a\sigma^{2}_{i}}$ where $a=2.85$. We found the value of $a$ through a heuristic search and noticed that 2.85 suffices for all the shape experiments that we had. There are total of 629 foreground RBF centers computed by SDN (only 2.51% of all the input image pixels).

6.2 Skeleton Extraction From the SDs

Next, we demonstrate the skeleton extraction from the computed similarity domains as a proof of concept. Extracting the skeleton from the SDs as opposed to extracting it from the pixels, simplifies the computations as SDs are only a small portion of the total number of pixels (reducing the search space). To extract the skeleton from the computed SDs, we first quantize the shape parameters of the object into 10 bins and then starting from the largest bin, we select the most useful bin value to threshold the shape parameters. The remaining SD centers are connected based on their overlapping similarity domains. If multiple SDs overlap inside the same SD, we look at their centers and we ignore the SDs whose centers fall within the same SD (accepted the original SD center). That is why some points are not considered as a part of the skeleton in Figure 4. First row in Figure 4 demonstrates the remaining SD centers and their radiuses at various thresholds. The second row in the figure visualizes the extracted skeletons (shown as a blue line) from the SDs as explained in Section 5. Another example is shown in Figure 5. The learned SDs are thresholded and the corresponding skeleton as extracted from the remaining SDs are visualized as a blue line.

7 Conclusion

In this paper, we introduced how the computed SDs of the SDN algorithm can be used to extract skeleton from shapes for the first time as a proof of concept. Instead of using and processing all the pixels to extract the skeleton of a shape, we propose to use SDs (a subset of the pixels) to extract the skeleton. The RBF shape parameters of SDN are used to define SDs and they can be used to model a shape as described in Section 4 and as visualized in our experiments. While the presented skeleton extraction algorithm is a naive solution to demonstrate the use of SDs, future work will focus on presenting more elegant solutions to extract the skeleton from SDs. SDN is a novel classification algorithm and has potential in many shape analysis applications besides the skeleton extraction. A shape can be modeled parametrically by using SDNs via shape parameters and RBF centers. A further reduction in parameters can be obtained with one class classification approximation of SDN as shown in Eq. 4. SDN can parametrically model a given single shape without requiring or using large datasets.

Acknowledgement

We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Quadro P6000 GPU used for this research.