Skill Acquisition via Automated Multi-Coordinate Cost Balancing

TL;DR

This paper introduces MCCB, a framework for learning movement skills from demonstrations by encoding in multiple coordinates, optimizing a convex cost function, and learning coordinate weights, demonstrated on handwriting and complex skills.

Contribution

MCCB is a novel framework that encodes demonstrations in multiple differential coordinates and learns optimal weights for balancing their importance.

Findings

Effective in reproducing handwriting and complex skills

Outperforms baseline methods in skill acquisition tasks

Demonstrates robustness across diverse datasets

Abstract

We propose a learning framework, named Multi-Coordinate Cost Balancing (MCCB), to address the problem of acquiring point-to-point movement skills from demonstrations. MCCB encodes demonstrations simultaneously in multiple differential coordinates that specify local geometric properties. MCCB generates reproductions by solving a convex optimization problem with a multi-coordinate cost function and linear constraints on the reproductions, such as initial, target, and via points. Further, since the relative importance of each coordinate system in the cost function might be unknown for a given skill, MCCB learns optimal weighting factors that balance the cost function. We demonstrate the effectiveness of MCCB via detailed experiments conducted on one handwriting dataset and three complex skill datasets.

Tables

Table 1. TABLE I: Orange check marks denote the most relevant coordinate and green check marks denote the best performing method.

	Single Coordinate			Multi-Coordinate
	Cartesian	Tangent	Laplacian	Uniform W.	MCCB
Handwriting		✓ ✓	✓ ✓	✓	✓
Picking		✓			✓
Pressing	✓				✓
Pushing			✓		✓

Equations

$$\mathbf{\Delta }=\mathbf{L}\mathbf{X}$$

$$\mathbf{L}=\left[\begin{array}{cccccc}1& -1& 0& \dots & \dots & 0\\ -0.5& 1& -0.5& 0& \dots & 0\\ 0& -0.5& 1& -0.5& \dots & 0\\ ⋮& & \ddots & \ddots & \ddots & ⋮\\ 0& \dots & 0& -0.5& 1& -0.5\\ 0& \dots & \dots & 0& -1& 1\end{array}\right]$$

$$\mathbf{\Gamma }=\mathbf{G}\mathbf{X}$$

$$\mathbf{G}=\left[\begin{array}{cccccc}-1& 1& 0& \dots & \dots & 0\\ 0& -1& 1& 0& \dots & 0\\ 0& 0& -1& 1& \dots & 0\\ ⋮& & \ddots & \ddots & \ddots & ⋮\\ 0& \dots & 0& 0& -1& 1\\ 0& \dots & \dots & 0& 0& -1\end{array}\right]$$

$$\hat{x}(t)=\mathbb{E}[x|t]=\sum _{k=1}^{K_C}{h}_C^k(t)(A_C^{k}t+b_C^k)$$

$${\hat{\Sigma }}_x(t)=Var[x|t]=\sum _{k=1}^{K_C}{h_C^k}^2({\Sigma }_x^k-{\Sigma }_{x,t}^k({\Sigma }_t^k{)}^{-1}{\Sigma }_{t,x})$$

$$\hat{\gamma }(t)=$$

$$\hat{\delta }(t)=$$

$${\hat{\Sigma }}_{\gamma }(t)=$$

$${\hat{\Sigma }}_{\delta }(t)=$$

$${\mathbf{X}}_r=$$

$$+w_L{J}_L(\mathbf{X})$$

$$s.t.P_x\mathbf{X}={\mathbf{X}}^{\ast }$$

$$J_C(\mathbf{X})=$$

$$J_G(\mathbf{X})=$$

$$J_L(\mathbf{X})=$$

$${\hat{w}}_C=\frac{{\alpha }_C}{{\beta }_C};{\hat{w}}_G=\frac{{\alpha }_G}{{\beta }_G};{\hat{w}}_L=\frac{{\alpha }_L}{{\beta }_L}$$

$${\beta }_i=\frac{{\sum }_{j=1}^N{J}_i({\mathbf{X}}_d^j)}{{\sum }_l{\sum }_{j=1}^N{J}_l({\mathbf{X}}_d^j)},\forall i,l=\{C,G,L\}$$

$$\{{\alpha }_C,{\alpha }_G,{\alpha }_L\}=$$

$$s.t.\sum _i{\alpha }_i=1,\forall i=\{C,G,L\}$$

$${\mathbf{X}}_r^j=$$

$$+\left(\frac{{\alpha }_L}{{\beta }_L}\right)J_L(\mathbf{X})$$

$$s.t.P_x\mathbf{X}={\mathbf{X}}_j^{\ast }$$

Full text

I Introduction

The next generation of robots, that can operate in and adapt to unstructured and dynamic environments, must possess a diverse set of skills. However, it is implausible to pre-program robots with a library of all required skills. Learning from Demonstration (LfD) [1, 2] is a paradigm that aims to equip robots with the ability to learn efficiently from demonstrations provided by humans. Existing work in trajectory-based LfD has contributed a wide range of mathematical representations that encode skills from human demonstrations and then reproduce the learned skills at runtime. Proposed representations include Spring-damper systems with forcing functions [3], Gaussian Mixture Models (GMMs) [4, 5, 6], Neural Networks (NNs) [7, 8], Gaussian Processes (GPs) [9, 10, 11], and geometric objects [12], among others. Each of these representations is used to encode the demonstrations in a predefined space or coordinate system (e.g., Cartesian coordinates). In other words, a single best coordinate system for any given skill is assumed to both exist and be known. However, as we show in this work, the assumption that a single best coordinate system exists for each task does not hold. Further, encoding in only a single coordinate system prohibits the model from capturing some of the geometric features that underlie a demonstrated skill.

In this work, we contribute a learning framework that encodes demonstrations simultaneously in multiple coordinates, and balances the relative influences of the learned models in generating reproductions. The proposed framework, named Multi-Coordinate Cost Balancing (MCCB), encodes demonstrations in three differential coordinates: Cartesian, tangent, and Laplacian (Section III-A). Simultaneously learning in these three coordinates allows our method to capture all of the underlying geometric properties that are central to a given skill. MCCB encodes the joint density of the time index and the demonstrations in each differential coordinate frame using a separate statistical model. Thus, given any time instant, we are able to readily obtain the conditional mean and covariance in each coordinate system (Section III-B). MCCB generates reproductions by solving an optimization problem with a blended cost function that consists of one term per coordinate. Each term penalizes deviations from the norm, weighted by the inverse of the expected variance in the corresponding coordinate system (Section III-C). Further, we subject the optimization problem to linear constraints on the reproductions, such as initial, target, and via point constraints. Our constrained optimization problem is convex with respect to the reproduction and hence can be solved efficiently.

A major hurdle in learning a wide variety of skills, without significant parameter tweaking, is that the relative importance of each differential coordinate (or the geometric feature) in encoding a given task is unknown ahead of time. For instance, consider the problem of encoding the demonstrations illustrated in Fig. 1. Using any one coordinate system in isolation, even when the most suitable one is known, does not yield good reproductions (the red, brown, and green dashed lines). To alleviate this problem, MCCB preferentially weights the costs defined in each coordinate (Fig. 2). Importantly, MCCB learns the optimal weights directly from the demonstrations without making task-dependent assumptions. To this end, MCCB solves a meta optimization problem that aims to minimize reproduction errors (Section III-D). As shown by the solid blue lines in Fig. 1, a cost function that optimally balances the costs in each coordinate yields better reproductions than any single-coordinate method.

In summary, we contribute a unified task-independent learning framework that (1) encodes demonstrations simultaneously in multiple differential coordinates, (2) defines a blended cost function that incentivizes conformance to the norm in each coordinate system while considering expected variance, and (3) learns optimal weights directly from the demonstrations to balance the relative influence of each differential coordinate in generating reproductions. Further, MCCB is compatible with and complementary to several existing LfD methods that utilize different statistical representations and coordinate systems [13, 14, 12, 11, 10, 15, 16].

II Related Work

Learning from demonstration has attracted a lot of attention from researchers in the past few decades. While several categories of LfD methods exist [1], our work falls under the category of trajectory-based LfD. In this category, demonstrations take the form of trajectories and the methods aim to synthesize trajectories that accurately reproduce the demonstrations.

Dynamical systems-based trajectory learning methods, such as [5, 6, 7], encode demonstrations using statistical dynamical systems and generate reproductions by forward propagating the dynamics. While such deterministic methods exhibit several advantages, such as convergence guarantees and robustness to perturbations, they are restricted to learning in a single coordinate system and ignore inherent uncertainties in the demonstrations. They incentivize conformance to the norm even when demonstrations exhibit high variance.

Trajectory optimization methods, such as [17] and [18], focus on geometric features by minimizing costs specified using predefined norms. An optimization framework proposed in [19] attempts to adapt multiple demonstrations to new initial and target locations by minimizing the distance between the demonstrations and the reproduction according to a learned Hilbert space norm. Indeed, learning an appropriate Hilbert space norm is related to finding an appropriate coordinate system based on the demonstrations. However, similar to the dynamical systems-based methods, the methods in [17, 18, 19] are restricted to a single predefined or learned coordinate system and do not explicitly model and utilize the inherent time-dependent variations in the demonstrations.

Probabilistic trajectory-learning methods, such as [10, 11] and [14], on the other hand, capture and utilize the variation observed in the demonstrations. However, these methods are also restricted to encoding demonstrations in a single predefined coordinate system that is assumed to be known.

Our design of the costs in each differential coordinate is inspired by the minimal intervention principle [13] that takes variance into account. While the approach in [13] does encode demonstrations in different frames of references, all the frames are restricted to Cartesian coordinates or orientation space. Furthermore, all the relevant frames for a given task are also expected to be provided by the user.

The motion planning framework in [15], complementary to our approach, utilizes a blended cost function, the construction of which is guided by probability distributions learned from the demonstrations. This framework incentivizes factors such as smoothness, manipulability, and obstacle avoidance, but is restricted to the Cartesian coordinate system. MCCB, on the other hand, encodes demonstrations in multiple differential coordinates and learns to optimally balance their relative influences, but does not consider factors such as manipulability and obstacle avoidance.

Differential coordinates have been extensively used in the computer graphics community [20, 21]. Prior work in trajectory learning that incorporates differential coordinates includes the Laplacian trajectory editing (LTE) algorithm [16]. Using Laplacian coordinates, the LTE algorithm adapts a single demonstration to new initial, target, and via points while preserving the shape. However, the LTE algorithm does not reason about the relative importances of multiple coordinates.

III Methodology

The section describes the technical details of MCCB and its work flow as illustrated in Fig. 2.

III-A Differential Coordinate Transformations

In this section, we define the differential coordinates and their corresponding transformations used in MCCB.

Cartesian: Let a discrete finite-length trajectory in $n$-dimensional Cartesian coordinates be denoted by $\bm{X}=[x(1)\ x(2)\cdots x(T)]^{\top}\in\mathbb{R}^{T\times n}$ and let $x(t)\in\mathbb{R}^{n}$ denote a discrete sample at time index $t$. This trajectory can be represented using a graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$ where $\mathcal{V}$ is the set of vertices representing the samples in the trajectory and $\mathcal{E}$ is the set of edges that represent the connections between the samples in the trajectory. The neighborhood $\mathcal{N}_{t}$ of each vertex $\mathcal{V}_{t}$ is defined by the set of adjacent vertices $\mathcal{V}_{t}^{\prime}$. In the case of discrete-time trajectories, the edges between any given vertex and its two neighbors are assumed to carry unit weights, while all other edges carry zero weights.

Laplacian: It is known that the discrete Laplace-Beltrami operator for the trajectory $\bm{X}$ provides the Laplacian coordinate $\delta(t)$ as ${\delta(t)\triangleq\sum_{t^{\prime}\in\mathcal{N}_{t}}\frac{1}{\sum_{t^{\prime}\in\mathcal{N}_{t}}1}(x(t)-x(t^{\prime}))}$ [20]. Note that the above relationship can be written as a linear differential operator in matrix form

[TABLE]

where $\bm{\Delta}=[\delta(1)\ \delta(2)\cdots\delta(T)]^{\top}\in\mathbb{R}^{T\times n}$ is the trajectory in the Laplacian coordinates, and $\bm{L}\in\mathbb{R}^{T\times T}$, called the graph Laplacian, is given by

[TABLE]

As pointed out in [16], the Laplacian coordinates have meaningful geometric interpretations. Specifically, the Laplacian coordinates can be seen as the discrete approximations of the derivative of the unit tangent vectors of an arc-length parametrized continuous trajectory. In other words, the Laplacian coordinates measure the deviation of each sample from the centroid of its neighbors.

Tangent: While the Laplacian coordinates are discrete approximations of second order differential transformations, a discrete approximation of the first differential transformation is possible. Consider such a first order transformation using first order finite differences defined as ${\gamma(t)\triangleq(x(t+1)-x(t))}$, where $\gamma(t)$ is called the tangent coordinate. The matrix form of the above relationship results in a linear differential operator given by

[TABLE]

where $\bm{\Gamma}=[\gamma(1)\ \gamma(2)\cdots\gamma(T)]^{\top}\in\mathbb{R}^{T\times n}$ is the trajectory in the tangent coordinates and $\bm{G}\in\mathbb{R}^{T\times T}$, called the graph incidence matrix, is given by

[TABLE]

Similar to the Laplacian coordinates, the tangent coordinates have geometric interpretations. Specifically, the tangent coordinates can be seen as discrete approximations of the un-normalized tangent vectors of an arc-length parametrized continuous trajectory, i.e., the tangent coordinates measure the local direction of motion at each sample of the trajectory.

In our work, we assume that a set of $N$ demonstrations in the Cartesian coordinates are available. Let the $j$th demonstration be denoted by $\bm{X}_{d}^{j}=[x_{d}^{j}(1)\ x_{d}^{j}(2)\cdots x_{d}^{j}(T)]^{\top}\in\mathbb{R}^{T\times n}$. Note that if the raw demonstrations are of varying duration in time, we perform time alignment using dynamic time warping. MCCB transforms each obtained demonstration $\bm{X}_{d}^{j}$ into a trajectory in the tangent coordinates (denoted by $\bm{\Gamma}_{d}^{j}$) and a trajectory in Laplacian coordinates (denoted by $\bm{\Delta}_{d}^{j}$) using (1) and (3), respectively.

III-B Encoding in Multiple Differential Coordinates

This section defines the costs associated with each coordinate. With the demonstrations available in all three differential coordinates, we employ three independent Gaussian mixture models (GMMs)111MCCB does not rely on the use of GMMs and any statistical representation that can provide the conditional estimates will suffice. to approximate the joint probability densities of time and the samples in each coordinate system.

The GMM associated with the Cartesian coordinates attempts to approximate the joint density of $t$ and $x$ using a finite number of Gaussian basis functions as follows ${\mathcal{P}(t,x;\theta_{C})=\sum_{k=1}^{K_{C}}\mathcal{P}(k)\mathcal{P}(t,x|k)}$, where $K_{C}$ is the number of Gaussian basis functions, $\mathcal{P}(k)=\pi_{C}^{k}$ is the prior associated with the $k$th basis function, $\theta_{C}=\{\mu_{C}^{1}\cdots\mu_{C}^{K_{C}},\Sigma_{C}^{1}\cdots\Sigma_{C}^{K_{C}},\pi_{C}^{1}\cdots\pi_{C}^{K_{C}}\}$ is the set of parameters of the GMM, and $\mathcal{P}(t,x|k)$ is the conditional probability density given by ${\mathcal{P}(t,x|k)\sim\mathcal{N}\left(\begin{bmatrix}t\\ x\end{bmatrix};\mu_{C}^{k},\Sigma_{C}^{k}\right)}$, where $\mu_{C}^{k}=\begin{bmatrix}\mu_{t}^{k}\\ \mu_{x}^{k}\end{bmatrix}$ is the mean and $\Sigma_{C}^{k}=\begin{bmatrix}\Sigma_{t}^{k}&\Sigma_{t,x}^{k}\\ \Sigma_{x,t}^{k}&\Sigma_{x}^{k}\end{bmatrix}$ is the covariance matrix of the $k$th Gaussian basis function.

We learn the parameters $\theta_{C}$ of the model using the Expectation-Maximization algorithm based on the demonstrations $\{\bm{X}_{d}^{j}\}_{j=1}^{N}$. Given the learned model and a time instant, the expected value of the conditional density $\mathcal{P}(x|t)$ is given by Gaussian mixture regression (GMR) [22] as follows

[TABLE]

where $h_{C}^{k}(t)=\frac{\mathcal{P}(k)\mathcal{P}(t|k)}{\sum_{i=1}^{K_{C}}\mathcal{P}(i)\mathcal{P}(t|i)}$, $A_{C}^{k}=\Sigma_{x,t}^{k}(\Sigma_{t}^{k})^{-1}$, $b^{k}=\mu_{x}^{k}+(t-\mu_{t}^{k})$, and the conditional covariance is given by

[TABLE]

Similar to the GMM learned in the Cartesian coordinates, we learn a second GMM in the tangent coordinates based on the demonstrations $\{\bm{\Gamma}_{d}^{j}\}_{j=1}^{N}$, and a third GMM in the Laplacian coordinates based on the demonstrations $\{\bm{\Delta}_{d}^{j}\}_{j=1}^{N}$. The expected values of the conditional densities $\mathcal{P}(\gamma|t)$ and $\mathcal{P}(\delta|t)$ are given by

[TABLE]

and the corresponding conditional expectations are given by

[TABLE]

where the variables in (7)-(10) with subscripts $G$ and $L$ correspond to the tangent and Laplacian coordinates, respectively, and are defined similarly to the ones in (5)-(6).

III-C Imitation via Optimization

In this section, we explain the design of our multi-coordinate cost function. MCCB generates reproductions by solving a constrained optimization problem given by

[TABLE]

where $\bm{X}_{r}\in\mathbb{R}^{T\times n}$ is the reproduction, $w_{C},\ w_{G},\ w_{L}\in\mathbb{R}^{+}$ are positive weights; $J_{C},\ J_{G},\ J_{L}:\mathbb{R}^{T\times n}\rightarrow\mathbb{R}^{+}$ are cost functions in the Cartesian, tangent, and Laplacian coordinates, respectively; $P_{x}\in\mathbb{R}^{m\times T}$ and $\bm{X}^{*}\in\mathbb{R}^{m\times n}$ define $m\in\mathbb{Z}^{+}$ linear constraints on $\bm{X}_{r}$. In practice, $m<<n$ and we use the linear constraints to enforce constraints on initial, target, and via points.

We define the cost function in each coordinate system as follows

[TABLE]

where $\bm{\hat{\Sigma}_{\bm{X}}},\bm{\hat{\Sigma}_{\Gamma}},\bm{\hat{\Sigma}_{\Delta}\in\mathbb{R}^{nT\times nT}}$ denote the block diagonal matrices formed with the conditional covariances $\hat{\Sigma}_{x}(t),\hat{\Sigma}_{\gamma}(t),$ and $\hat{\Sigma}_{\delta}(t)$, respectively, for all values of $t$. Further, the notation $(:)$ denotes vectorization - for instance, $\bm{X}(:),\hat{\bm{X}}(:)\in\mathbb{R}^{nT}$ denote the vectorized trajectories formed by vertically stacking $x(t)$ and $\hat{x}(t)$ for all values of $t$, respectively. Note that we construct the trajectories $\Gamma$ and $\Delta$ in (14) and (15) from $\bm{X}$ via the linear operators defined in (3) and (1), respectively. MCCB penalizes deviations from the conditional mean in each coordinate system. However, deviations are penalized less (more) severely if high (low) variance is observed in the demonstrations at any given time.

III-D Automated Cost Balancing

In order to obtain reproductions that successfully imitate demonstrations of a wide variety of skills, the weights $w_{C},\ w_{G},$ and $w_{L}$ have to be chosen with care. Indeed, they preferentially weight the costs defined in each differential coordinate and thereby manipulate the relative incentive for successful imitation in each coordinate system.

We learn these weights directly from the available demonstrations. Note that, for known weights, the constrained optimization problem in (11) is convex in $\bm{X}$. We estimate the weights in the following form

[TABLE]

where $\beta_{C},\ \beta_{G},\ \beta_{L}\in(0,1]$, such that $\sum_{i}\beta_{i}=1$, are positive scaling factors used to correct for inherent differences in the magnitudes of the costs, and $\alpha_{C},\ \alpha_{G},\ \alpha_{L}\in[0,1]$, such that $\sum_{i}\alpha_{i}=1$, are positive weights used to preferentially weight the cost defined in each coordinate system. MCCB estimates the scaling factors $\beta_{i}$’s as follows

[TABLE]

With the scaling factors compensating the inherent scale difference in the costs, we compute the preferential weighting factors $\alpha_{i}$’s that minimize reproduction error. To this end, we formulate the following meta optimization problem

[TABLE]

where $\mathrm{SSE}(\cdot)$ denotes the sum of squared errors computed over time, and $\bm{X}_{r}^{j}$ is the solution to the following optimization problem

[TABLE]

where $P_{x}\bm{X}=\bm{X}_{j}^{*}$ denotes specific linear constraints pertaining to the demonstration $\bm{X}_{d}^{j}$, such as initial, target, and via points. Solving the above meta-optimization problem results in the preferential weights $\alpha_{i}$’s that minimize reproduction errors of the solutions generated by the original constrained optimization problem in (11)-(12).

IV Experimental Evaluation

This section describes the design and discusses the results of four experiments conducted to evaluate MCCB. In each experiment, we compared the performances of the following approaches:

Cartesian-coordinates: $w_{C}=1$, $w_{G}=0$, $w_{L}=0$ 2. 2.

Tangent-coordinates: $w_{C}=0$, $w_{G}=1$, $w_{L}=0$ 3. 3.

Laplacian-coordinates: $w_{C}=0$, $w_{G}=0$, $w_{L}=1$ 4. 4.

Uniform weighting: $w_{C}=1/3$, $w_{G}=1/3$, $w_{L}=1/3$ 5. 5.

MCCB: $w_{C}=\hat{w_{C}}$, $w_{G}=\hat{w_{G}}$, $w_{L}=\hat{w_{L}}$

We measured the performance of each approach by the following geometric and kinematic metrics: Swept Error Area (SEA) [23], Sum of Squared Errors (SSE), Dynamic Time Warping Distance (DTWD), and Frechet Distance (FD) [24]. These metrics allow us to evaluate different aspects of each method’s performance. The SEA and SSE metrics penalize both spatial and temporal misalignment, and thus evaluate kinematic performance. On the other hand, the DTWD and FD metrics penalize spatial misalignment while disregarding time misalignment, and thus evaluate geometric performance. Further, the SEA, SSE, and DTWD metrics evaluate aggregate performance by summing over or averaging across all the samples of each reproduction. The FD metric, on the other hand, computes the shortest possible cord length required to connect the demonstration and the reproduction in space while allowing time re-parametrization of either trajectory, and thus measures maximal deviation in space. Note that the SEA metric is restricted to 2-dimensional data, so we only report it for one of our experiments.

In all the experiments, we used the position constraints in (12) to enforce both initial and end point constraints uniformly across all the methods being compared. Further, we uniformly set the number of Gaussian basis functions to five across all the coordinates and all the experiments.

IV-A Handwriting Skill

This experiment evaluates MCCB on the publicly available LASA human handwriting library [5], that consists of handwriting motions collected from pen input using a Tablet PC. The library contains a total of 25 handwriting motions, each with 7 demonstrations.

Fig. 3 shows that MCCB yields reproductions that are qualitatively similar to the demonstrations while satisfying the end-point constraints across all motions. As shown in Fig. 4, quantitative analysis indicates that MCCB ($\bar{\alpha}_{C}=0.1814,\ \bar{\alpha}_{G}=0.4958,\ \bar{\alpha}_{L}=0.3228$)222Weighting factors averaged over all 25 skills in the LASA dataset and three of the four baselines performed comparably with respect to the SEA, FD, SSE, and DTWD metrics, while the Cartesian baseline performed poorly in comparison. This is consistent with the fact that the demonstrations within the LASA dataset emphasize strong similarities in shape.

IV-B Picking Skill

The second experiment evaluates the performance of MCCB in a picking task (Fig. 5). The data consists of six kinesthetic demonstrations, each a 3-dimensional robot end-effector position trajectory recorded as a human guided the robot in picking up two magnets atop two blocks. We enforced two via-point constraints (one at each picking point) in addition to the end-point constraints.

As shown in Fig. 6(a), MCCB generated reproductions that are qualitatively similar to the demonstrations while satisfying all the position constraints. Quantitative evaluations reveal that learning in tangent coordinates yielded better reproductions than learning in Cartesian and Laplacian coordinates (Fig. 7). This was expected since the demonstrations of this task, much like the LASA dataset, emphasize shape similarity. Further, MCCB ($\alpha_{C}=0.2362,\ \alpha_{G}=0.5451,\ \alpha_{L}=0.2187$) yielded the best performance, with respect to all three metrics. In fact, uniform weighting yielded poorer results, with respect to all three metrics, than when considering only the tangent coordinates. The results of this experiment show that while multi-coordinate methods can yield strong performance, it is critical that we balance the weights appropriately.

IV-C Pressing Skill

In this experiment, we evaluated MCCB’s ability to learn pressing skills (Fig. 5). The data consists of six kinesthetic demonstrations, each a 3-dimensional robot end-effector position trajectory recorded as a human guided the robot in pressing two cylindrical pegs into their respective holes.

As shown in Fig. 6(b), MCCB successfully reproduced the demonstrations. Note that MCCB is capable of automatically capturing and reproducing the consistencies across the demonstrations in certain regions without any position constraints. Fig. 8 illustrates the performance of MCCB and the baselines with respect to three different metrics. Learning in Cartesian coordinates resulted in the better performance compared to learning in tangent and Laplacian coordinates. Quantitative evaluations further demonstrate that MCCB ($\alpha_{C}=0.6735,\ \alpha_{G}=0.2034,\ \alpha_{L}=0.1231$) consistently yielded the best performance with respect to all three metrics. The results of this experiment, in light of the results in Section IV-B, suggest that the relative importance of each of the differential coordinates vary across different skills.

IV-D Pushing Skill

The final experiment evaluates the performance of MCCB in a pushing task (Fig. 5). The data consists of six kinesthetic demonstrations, each a 3-dimensional robot end-effector position trajectory recorded as a human guided the robot in sliding closed the lid of a wooden box.

As shown in Fig. 6(c), MCCB successfully generated reproductions that are similar to the demonstrations. As evidenced by quantitative evaluations in Fig. 9, encoding demonstrations in the Laplacian coordinates yielded better performance, with respect to all three metrics, when compared to learning only in either of the other two coordinates, while, MCCB ($\alpha_{C}=0.0123,\ \alpha_{G}=0.045,\ \alpha_{L}=0.9427$) consistently outperformed all the other approaches. Note that learning in the Laplacian coordinates alone resulted in better performance than uniformly weighting of all the coordinates. These results are consistent with the results from the previous sections and indicate that MCCB yields consistently good performance. The results are summarized in Table I.

V Discussion and Conclusion

We introduced MCCB, a learning framework for encoding demonstrations in multiple differential coordinates, and automated balancing of costs defined in those coordinates. As shown in Table I, we demonstrated that the relative effectiveness of each coordinate system is not consistent across a variety of tasks since any given skill might be better suited for learning in one (or more) coordinate system(s). Furthermore, uniform weighting of costs in different coordinates does not consistently yield the best results across different skills. Indeed, uniform weighting, in some cases, yielded poorer performances compared to when only one coordinate system was used. On the other hand, MCCB learned to balance the costs and consistently yielded the best performance. Since the weights are learned directly from the demonstrations, MCCB makes no task-specific assumptions and does not require tedious parameter tuning. Note that although we used GMMs as the base representation in this work, MCCB is agnostic to the statistical model used to encode the demonstrations in each coordinate system, and thus can be combined with other techniques, such as [13, 14, 12, 11, 10, 15, 16]. Furthermore, MCCB can be extended to include more coordinate systems that capture additional trajectory features.

ACKNOWLEDGMENT

This research is supported in part by NSF NRI 1637758.