Adaptive Critic Based Optimal Kinematic Control for a Robot Manipulator

TL;DR

This paper introduces a novel adaptive critic control method for robot manipulators that guarantees convergence to optimality and stability, validated through simulations and real-world experiments with a 6-DOF robot.

Contribution

It proposes a new critic weight update law within the SNAC framework that ensures convergence and stability in kinematic control tasks.

Findings

Guaranteed convergence to optimal cost

Stable control of robot manipulator

Validated with real robot experiments

Abstract

This paper is concerned with the optimal kinematic control of a robot manipulator where the robot end effector position follows a task space trajectory. The joints are actuated with the desired velocity profile to achieve this task. This problem has been solved using a single network adaptive critic (SNAC) by expressing the forward kinematics as input affine system. Usually in SNAC, the critic weights are updated using back propagation algorithm while little attention is given to convergence to the optimal cost. In this paper, we propose a critic weight update law that ensures convergence to the desired optimal cost while guaranteeing the stability of the closed loop kinematic control. In kinematic control, the robot is required to reach a specific target position. This has been solved as an optimal regulation problem in the context of SNAC based kinematic control. When the robot is required to follow a time varying task space trajectory, then the kinematic control has been framed as an optimal tracking problem. For tracking, an augmented system consisting of tracking error and reference trajectory is constructed and the optimal control policy is derived using SNAC framework. The stability and performance of the system under the proposed novel weight tuning law is guaranteed using Lyapunov approach. The proposed kinematic control scheme has been validated in simulations and experimentally executed using a real six degrees of freedom (DOF) Universal Robot (UR) 10 manipulator.

Equations

$$x(t)=f(\theta (t))$$

$$\dot{x}(t)=J\dot{\theta }(t)$$

$$\dot{x}(t)=Ju(t)$$

$$e(t)=x(t)-x_d(t)$$

$$\dot{e}=\dot{x}(t)=Ju(t)$$

$$V(e(t))={\int }_t^{\infty }e(t{)}^{T}Qe(t)+u(t{)}^{T}Ru(t)dt$$

$$H(e(t),u(t),\nabla V(t))=$$

$$+u(t{)}^{T}Ru(t)$$

$$u^{\ast }(e)=\frac{-1}{2}{R}^{-1}{J}^T\nabla V^{\ast }(e)$$

$$0={\nabla V^{\ast }}^T\dot{e}(t)+e(t{)}^{T}Qe(t)+{u^{\ast }(t)}^{T}R{u}^{\ast }(t)$$

$$V^{\ast }(e(t))=W_c^T{\sigma }_c(e)+{\epsilon }_c$$

$$\nabla V^{\ast }=\nabla {\sigma }_c^T{W}_c+\nabla {\epsilon }_c$$

$${\hat{V}}^{\ast }={\widehat{W_c}}^T{\sigma }_c(e)$$

$$\nabla {\hat{V}}^{\ast }=\nabla {\sigma }_c^T\widehat{W_c}$$

$$u^{\ast }(e)=\frac{-1}{2}{R}^{-1}{J}^T(\nabla {\sigma }_C^{T}Wc+\nabla {\epsilon }_c)$$

$${\hat{u}}^{\ast }(e)=\frac{-1}{2}{R}^{-1}{J}^T\nabla {\sigma }_c^T\widehat{W_c}$$

$$\dot{e}=-\frac{1}{2}J{R}^{-1}{J}^T\nabla {\sigma }_c^T\widehat{W_c}$$

$$\dot{J_s(e)}=\nabla J_s(e{)}^T\dot{e}<0$$

$$\begin{gathered} \nabla J_s(e{)}^T\dot{e}=-\nabla J_s(e{)}^{T}M\nabla J_s(e) \\ \le -{\lambda }_{min}(M)\parallel \nabla J_s(e){\parallel }^2 \end{gathered}$$

$$\dot{\widehat{W_c}}=\alpha \nabla {\sigma }_{c}J{R}^{-1}{J}^T\nabla J_s(e)$$

$$L(t)=\frac{1}{4\alpha }{\widetilde{W_c}}^T\widetilde{W_c}+J_s(e)$$

$$\dot{L}(t)=\frac{1}{2\alpha }{\widetilde{W_c}}^T\dot{\widetilde{W_c}}+\nabla J_s^T\dot{e}$$

$$\dot{L}(t)=\frac{-1}{2}{\widetilde{W_c}}^T\nabla \sigma J{R}^{-1}{J}^T\nabla J_s+\nabla J_s^T(\dot{e})$$

$${\dot{e}}^{\ast }-\dot{e}=$$

$$\dot{L}(t)=\nabla J_s^T{\dot{e}}^{\ast }+\frac{1}{2}\nabla J_s^{T}J{R}^{-1}{J}^T\nabla {\epsilon }_c$$

$$\dot{L}\le -{\lambda }_{min}(M)\parallel \nabla J_s{\parallel }^2+\frac{1}{2}\parallel \nabla J_s\parallel {\lambda }_J^2\parallel R^{-1}\parallel {\lambda }_{\nabla \epsilon }$$

$$\parallel \nabla J_s\parallel \ge \frac{1}{2{\lambda }_{min}(M)}{\lambda }_J^2\parallel R^{-1}\parallel {\lambda }_{\nabla \epsilon }$$

$$\parallel u^{\ast }(e)-{\hat{u}}^{\ast }(e)\parallel \le \frac{1}{2}\parallel R^{-1}\parallel {\lambda }_J({\lambda }_{\sigma }{\lambda }_{\widetilde{W_c}}+{\lambda }_{\nabla \epsilon })\stackrel{\Delta }{=}{\lambda }_u$$

$$\ddot{L}={\dot{e}}^T{\dot{e}}^{\ast }+e^T{\ddot{e}}^{\ast }+\frac{1}{2}{\dot{e}}^{T}J{R}^{-1}{J}^T\nabla {\epsilon }_c+e^T\dot{J}{R}^{-1}{J}^T\nabla {\epsilon }_c+$$

$$\frac{1}{2}{e}^{T}J{R}^{-1}{J}^T{\nabla }^2{\epsilon }_c\dot{e}$$

$$\dot{x_d}(t)=\phi (x_d(t))$$

$$\dot{e}(t)=-\phi (x_d(t))+Ju(t)$$

Full text

I Introduction

Modern robotic systems are becoming complex as the degrees of freedom are increasing to address the complex application scenario such as agriculture, health-care and ware-house automation. A mobile manipulator with Shadow hand mounted on it has 28 degrees of freedom. The inverse kinematic solution to such systems can not be solved analytically. Thus neural and fuzzy neural network based schemes have become popular to design kinematic control[1, 2, 3, 4]. These approaches can not learn the inverse kinematics while optimizing a global cost function. This is very important when one deals with redundant manipulators. One of the approaches to design optimal kinematic control is to use the Hamilton-Jacobi-Bellman (HJB) formulation [5, 6]. The analytical solution of the HJB equation is still a major challenge and as such these solutions are obtained off-line. The real time approximate solutions are known as approximate dynamic programming and are presented in [7, 8, 9]. In these approaches, the framework uses two neural networks - one for action and the other critic. Action network learns to actuate an optimal policy while critic network evaluates the cost function through learning.

The single network adaptive critic (SNAC) is introduced [10] where critic is updated using back-propagation.

In [11], the kinematic control of a robot manipulator using this SNAC framework was presented. The block diagram of the kinematic control scheme for a robot manipulator using SNAC is shown in Fig. 1. As these schemes used back-propagation for critic weight updates, after repeated training, one could show the convergence to near optimal cost through extensive simulations. There exists very few works in the literature that has shown the convergence to the optimal cost in an analytical manner along with the proof of stability. In this work, we are solving the kinematic control problem of any $n$-DOF robot manipulator where the tasks of reaching a fixed target position and following a time varying task space trajectory have been solved in the framework of optimal regulation and optimal tracking respectively using SNAC. A simple and novel critic weight update rule which ensures that the closed loop system is stable has been proposed. The optimal kinematic control policy using HJB formulation has been developed in the framework of optimal regulation and optimal tracking. Former make the robot to reach fixed target position and later make the robot to follow the task-space time varying trajectory. The analytical proof of stability and convergence has been done using Lyapunov approach. As the degrees of freedom of robotic systems are increasing, learning based strategies will play more important role as compared to model based analytics [12],[13]. Thus the proposed approach has significant relevance in this context. The relevant literature is further scrutinized.

The optimization based kinematic control has been accomplished using local optimization for finding an instantaneous optimal solution [14, 15]. It is centered on the Jacobian pseudo-inverse and null space. The redundancy is resolved by including some constraints into the direct kinematic model or by projecting a particular solution onto the Jacobian null space. The other approach is a global optimization which uses an integral type performance index along the whole trajectory [16, 17]. The redundancy resolution is converted to an optimal control problem with the necessary conditions of optimality given by the Pontryagin’s principle or by the optimal control theory. Both of the aforementioned methods are suboptimal and require the pseudo-inversion of Jacobian continuously over time which is computationally expensive and suffers from local instability problems [18, 19, 20].

Existing approach towards following a time varying task space trajectory optimally is to find the feedforward term using the dynamics inversion concept and the feedback term by solving an HJB equation [21]. However, such solution is only near optimal because of the feedforward term. Therefore by using an augmented system dynamics consisting of feedback tracking error and reference trajectory and using it to solve the HJB equation results in an optimal control law which is a combination of feedforward and feedback control inputs [22].

The remainder of this paper is organised as follows. Problem formulation for optimal kinematic control is presented in Section II. In Section III and IV detailed mathematical derivation for the control scheme along with stability proof is presented. In Section V, various simulation and experimental results for kinematic motion control of a 6-DOF robotic manipulator are evaluated with comparison with the state-of-the-art kinematic control solutions. This paper is finally concluded in Section VI.

II Problem Formulation

The forward kinematics of a manipulator involves a non-linear transformation from Joint space to Cartesian space as described by:

[TABLE]

where, $x(t)\in\mathbb{R}^{n\times 1}$ describes the position and orientation of end effector in workspace at time $t$, $\theta(t)\in\mathbb{R}^{m\times 1}$ each element of which describes the joint angle in the joint space at time $t$, and $f(.)$ is the non linear mapping. Because of non-linearity and redundancy of mapping, it is usually difficult to directly get $\theta(t)$ for desired $x(t)=x_{d}(t)$, where $x_{d}(t)$ is the desired end effector position. By contrast, the mapping from joint space to cartesian space at velocity level is affine mapping. Taking time derivative on both sides of (1) gives:

[TABLE]

where, $J=\frac{\partial f}{\partial\theta}\in\mathbb{R}^{n\times m}$ is the Jacobian matrix of $f(.)$.

In this paper we focus on design of angular speed of the manipulator which serves as an input to the tracking control loop for robot control. Then the robot manipulator kinematics can be rewritten by replacing $\dot{\theta}(t)$ by $u(t)$:

[TABLE]

Thus, we get the dynamics of the system as above. In this paper we propose to solve for an optimal control input ${u}(t)$ for a robot manipulator described by (3), such that the tracking error given by,

[TABLE]

for a given reference trajectory $x_{d}(t)$, reduces to zero with time.

III Optimal Regulation

In this section, we address the optimal kinematic control of a robot manipulator, where the robot end effector has to reach a fixed target position. This is framed as an optimal regulation problem. In the context of kinematic control, optimal regulation means that the robot is made to reach a fixed target position following an optimal trajectory. This trajectory is generated by actuating the desired optimal control policy.

Let $x_{d}(t)\in\mathbb{R}^{n\times 1}$ be the desired end effector position. Differentiating (4) with respect to time, error dynamics of the system is obtained as,

[TABLE]

III-A Formulation of control policy

The infinite horizon HJB cost function for (5) is given by,

[TABLE]

where, $Q\in\mathbb{R}^{n\times n}$ and $R\in\mathbb{R}^{m\times m}$ are positive definite design matrices. The control input needs to be admissible so that the cost function equation (6) is finite. The Hamiltonian for the this cost function with the admissible control input $u(t)$ is,

[TABLE]

Here, $\nabla{V^{*}}$ is the gradient of the cost function $V(e(t))$ with respect to $e(t)$. The optimal control input which minimizes the cost function (6) also minimizes the Hamiltonian (7). Therefore, the optimal control is found by solving the following condition $\frac{\partial H(e,u,\nabla V)}{\partial u}=0$ and obtained to be

[TABLE]

Then, HJB equation may be re-written as follows,

[TABLE]

with the cost function $V^{*}(0)=0$.

III-B Neural Network control design

By using the universal approximation property, $V^{*}(e(t))$ is constructed by a single hidden layer neural network with a non-linear activation function:

[TABLE]

where, $W_{c}\in\mathbb{R}^{l}$, is the constant target NN weight vector, $\sigma_{c}\in\mathbb{R}^{l}$ is the activation function output, $l$ is the number of hidden neurons and $\varepsilon_{c}$ is the function reconstruction error. The target NN vector and reconstruction errors are assumed to be upper bounded according to $\parallel W_{c}\parallel\leq\lambda_{W_{c}}$ and $\parallel\varepsilon_{c}\parallel\leq\lambda_{\varepsilon}$.

[TABLE]

Approximate optimal cost function and its gradient is given by

[TABLE]

where, $\hat{W_{c}}$ is the NN estimate of target weight vector $W_{c}$. Using (8) and (11), the optimal control law can be written as:

[TABLE]

Considering (8) and (13), estimated optimal control law is:

[TABLE]

The modified error dynamics using (15) is,

[TABLE]

Before proposing the critic weight tuning law and stability proofs, following assumptions need to be made:

Assumption 1: For the system represented by its error dynamics (5), with cost function (6), let $J_{s}(e)$ be a continuously differentiable Lyapunov function candidate satisfying,

[TABLE]

Then, there exists a positive definite matrix, $M\in\mathbb{R}^{n\times n}$ ensuring,

[TABLE]

During the implementation, $J_{s}(e)$ can be obtained by selecting a polynomial with respect to the vector $e$, such as $J_{s}(e)=\frac{1}{2}e^{T}e$.

Remark 1: Based on the result of [23], the closed loop dynamics with optimal control law can be bounded by a function of the system state. In such situations, it can be asssumed that $\parallel F(e)+G(e)(u^{*})\parallel\leq\eta\parallel\nabla J_{s}(e)\parallel$ with $\eta>0$, $\parallel\nabla J_{s}(e)\parallel^{T}\parallel F(e)+G(e)(u^{*})\parallel\leq\eta\parallel\nabla J_{s}(e)\parallel^{2}$. Combining (17) with the fact given, $\lambda_{min}\parallel\nabla J_{s}(e)\parallel^{2}\leq\nabla J_{s}(e)^{T}M\nabla J_{s}(e)\leq\lambda_{max}(M)\parallel\nabla J_{s}(e)\parallel^{2}$, it implies that assumption 1 holds.

Moving on, a simple and novel critic weight tuning law is proposed.

[TABLE]

where, $\alpha$ is the learning rate of critic network and $J_{s}(e)=\frac{1}{2}e^{T}e$.By using this weight tuning law, stability and performance is guaranteed theoretically using the Lyapunov approach.

III-C Stability Analysis

In this section, the stability of the system for optimal regulation is investigated.

Assumption 2: The Jacobian matrix is bounded as $\parallel J(\theta)\parallel\leq\lambda_{J}$,where $\lambda_{J}$ is a positive constant. Here, the control coefficient matrix is the Jacobian matrix. Also, $\nabla\sigma_{c},\nabla\varepsilon_{c}$ are bounded as $\parallel\nabla\sigma_{c}\parallel\leq\lambda_{\sigma}$ and $\parallel\nabla\varepsilon_{c}\parallel\leq\lambda_{\nabla\varepsilon}$ where $\lambda_{\sigma}$ and $\lambda_{\nabla\varepsilon}$ are positive constants.

The Lyapunov candidate function is selected as,

[TABLE]

[TABLE]

where, $\widetilde{W}=W_{c}-\hat{W_{c}}$. Using (19),

[TABLE]

The system error dynamics of the system for the optimal control law is $\dot{e}^{*}=J{u}^{*}$. Using the control laws (14) and (15),

[TABLE]

Using (23) in (22) and on simplification,

[TABLE]

Applying Assumption 1 and Assumption 2 here

[TABLE]

From (25), the following inequality may be derived:

[TABLE]

With the above condition to be true, $\dot{L}\leq 0$ and the system is stable in the sense of Lyapunov implying that $\widetilde{W_{c}}$ and $e$ are both bounded. It may be expressed as : $\parallel\widetilde{W_{c}}\parallel\leq\lambda_{\widetilde{W_{c}}}$.

It may be also observed that the estimated optimal controller (15) converges to a neighborhood of the optimal feedback controller (14) with a finite bound $\lambda_{u}$ as,

[TABLE]

Hence, it may be concluded that the instantaneous cost function $V(e,u,t)=e(t)^{T}Qe(t)+u(t)^{T}Ru(t)$ is also bounded.

Taking time derivative of (24),

[TABLE]

Remark 2:

All the terms in $\ddot{L}$ can be shown to be bounded.

It may be observed that $\ddot{L}$ is bounded and Barbalat’s Lemma [24] can be invoked to conclude the asymtotic stability of the system and convergence of the parameter estimation error and the weight estimation errors towards zero. In other words, it ensures that $\widetilde{W_{c}}\rightarrow 0$.

IV Optimal Tracking Control

In this section, the kinematic control of a robot manipulator following a time varying task space trajectory is solved in the framework of optimal tracking.

IV-A Formation of Augmented System

Let the time varying reference trajectory, denoted by $x_{d}(t)\in\mathbb{R}^{n}$ be possessing the dynamics

[TABLE]

with the initial condition $x_{d}(0)={x_{d}}_{0}$, where, $\varphi(x_{d}(t))$ is a Lipschitz continuous function satisfying $\varphi(0)=0$. Let the trajectory tracking error be $e(t)=x(t)-x_{d}(t)$ with the initial condition $e(0)=e_{0}=x_{0}-{x_{d}}_{0}$. Considering (3) and (29), the tracking error dynamics is:

[TABLE]

Next, an augmented system is constructed as in [22],in the form $\xi(t)=[e(t)^{T},x_{d}(t)^{T}]^{T}\in\mathbb{R}^{2n}$ with initial condition $\xi(0)=\xi_{0}=[e_{0}^{T},{x_{d}}_{0}^{T}]^{T}$. The augmented dynamics based on (29) and (30) can be formulated as:

[TABLE]

where, F(.) and G(.) are the new system matrices.

IV-B Formulation of Control Policy

The infinite horizon HJB cost function for the system in (31) is given by,

[TABLE]

where, $\bar{Q}=diag\{Q,0_{n\times n}\}$. $Q\in\mathbb{R}^{n\times n}$ and $R\in\mathbb{R}^{m\times m}$ are positive definite matrices. The control input must be admissible so that cost function given by (32) is finite.

Applying the same methods of solving as in Section III-A, the control policy is obtained as,

[TABLE]

IV-C Neural Network control design

Here, $V^{*}(\xi(t))$ is constructed by a single hidden layer neural network with a non-linear activation function:

[TABLE]

where, $W_{c}\in\mathbb{R}^{p}$, is the ideal weight, $\parallel W_{c}\parallel\leq\lambda_{W_{c}},\phi_{c}\in\mathbb{R}^{p}$ is the activation function output, $p$ is the number of hidden neurons and $\varepsilon_{c}$ is the function approximation error.

Using the same approach as in Section III-B, optimal control law and approximate control law may be obtained as,

[TABLE]

[TABLE]

Applying (36) into the augmented system dynamics (31), $\dot{\xi}(t)=F(\xi)+G(\xi)\hat{u}^{*}(t)$ can be formulated as:

[TABLE]

The weight update law proposed for the critic of optimal tracking control is,

[TABLE]

where, $\alpha$ is the learning rate of critic network and $J_{s}(\xi)=\frac{1}{2}\xi^{T}\xi$.

Remark 3: Assumption 1 and *Remark 1 * holds here. The augmented system states shall replace the system error states as follows ; For the augmented system (31), with cost function (32), let $J_{s}(\xi)$ be a continuously differentiable Lyapunov function candidate satisfying,

[TABLE]

Then,there exists a positive definite matrix, $M\in\mathbb{R}^{2n\times 2n}$ensuring,

[TABLE]

During the implementation,$J_{s}(\xi)=\frac{1}{2}\xi^{T}\xi$.

IV-D Stability Analysis

In this section, the stability of the system is investigated.

Remark 4: Assumption 2 holds here. However, the control coefficient matrix is modified. The assumption $\parallel G(\xi)\parallel\leq\lambda_{g}$, where $\lambda_{g}$ is a positive constant shall be used.

Lyapunov function,

[TABLE]

[TABLE]

Using the similar approach as in Section III-C,

[TABLE]

Applying the Remark 3 and Remark 4 here

[TABLE]

From this, we observe that if the condition below holds, then, $\dot{L}(t)\leq 0$ is negative semidefinite implying that the system is stable in the sense of Lyapunov.

[TABLE]

Hence, $\widetilde{W_{c}}$ and $\xi$ are both bounded. It may be expressed as : $\parallel\widetilde{W_{c}}\parallel\leq\lambda_{\widetilde{W_{c}}}$

Also,

[TABLE]

It may be noted that estimated controller input converges to the neighbourhood of the optimal control input as a finite bound $\lambda_{u}$ exists just as in the case of regulation. It is also observed that the cost function $V(\xi,u,t)=\xi(t)^{T}\bar{Q}\xi(t)+u(t)^{T}u(t)$ is also bounded.

Using the same approach as in Section III-C, it may be shown that $\ddot{L}$ is bounded. Recalling Barbalat’s Lemma [25], it may be concluded that the system is stable and that the parameter estimation error and weight estimation error converge to zero.

V Results and Discussion

In this section, we consider the numerical simulations followed by real time experimental validations on a real 6 DOF UR 10 robot manipulator to demonstrate the effectiveness of the proposed kinematic control.

V-A Experimental Setup

Our experimental setup shown in Fig. 2 consists of a UR10 robot manipulator with its controller box/internal computer and a host PC/external computer. The UR10 robot manipulator is a 6 DOF robot arm designed to safely work alongside and in collaboration with a human. This arm can follow position commands like a traditional industrial robot, as well as take velocity commands to apply a given velocity in/around a specified axis. The low level robot controller is a program running on UR10’s internal computer broadcasting robot arm data , receiving and interpreting the commands and controlling the arm accordingly. There are several options for communicating with the robot low level controller to control the robot including the teach pendent or opening a TCP socket (C++/Python) on a host computer. We used open source C++ based UrDriver wrapper class integrated with ROS on a host PC to implement our proposed velocity based kinematic control scheme. The host PC streams joint velocity commands via URScript to the robot real time interface over Ethernet at $125Hz$. The driver was configured with necessary parameters like IP address of the robot at startup using ROS parameter server.

V-B Reaching a Fixed Position

Simulations are first performed to verify the kinematic control of the robot manipulator to reach a fixed position using the control law expressed in Equation (15). A typical simulation run generated with a random seed pose $(\theta_{0}=[-0.51-1.041.48-1.82-1.45-1.62]rad)$ to a fixed target pose $[-0.658,0.626,0.407]m$ in cartesian space is shown in Figure 3(a).The weights were initialized randomly and $\alpha=\alpha_{initial}(tanh(n-k))+\alpha_{final}$, where, n=50, k=time instance, $\alpha_{initial}=100$ and $\alpha_{final}=150$.The predicted cost function was: $w_{1}e_{1}^{2}+w_{2}e_{1}e_{2}+w_{3}e_{1}e_{3}+w_{4}e_{2}^{2}+w_{5}e_{2}e_{3}+w_{6}e_{3}^{2}$

V-C Following a Time Varying Reference Trajectory

In this section, the kinematic control of the robot manipulator to follow a time varying reference trajectory is validated. Simulations are first performed using the control law derived in Equation (36). A typical simulation generated with a random seed pose $(\theta_{0}=[-0.51-1.041.48-1.82-1.45-1.62]rad)$ and following a time varying reference trajectory moving at an angular speed of $0.075\quad rad/s$ along a circle centered at $[-0.7,0,0.5]m$ with a radius of $0.2m$ is shown in Figure 3(e). Weights were initialized randomly and $\alpha=\alpha_{initial}(tanh(n-k))+\alpha_{final}$, where, n=10, k=time intance, $\alpha_{initial}=20$ and $\alpha_{final}=70$. In this work, we take the predicted cost to be:

$w_{1}\xi_{1}^{2}+w_{2}\xi_{1}\xi_{2}+w_{3}\xi_{1}\xi_{3}+w_{4}\xi_{1}\xi_{4}+w_{5}\xi_{1}\xi_{5}+w_{6}\xi_{1}\xi_{6}+w_{7}\xi_{2}^{2}+w_{8}\xi_{2}\xi_{3}+w_{9}\xi_{2}\xi_{4}+w_{10}\xi_{2}\xi_{5}+w_{11}\xi_{2}\xi_{6}+w_{12}\xi_{3}^{2}+w_{13}\xi_{3}\xi_{4}+w_{14}\xi_{3}\xi_{5}+w_{15}\xi_{3}\xi_{6}+w_{16}\xi_{4}^{2}+w_{17}\xi_{4}\xi_{5}+w_{18}\xi_{4}\xi_{6}+w_{19}\xi_{5}^{2}+w_{20}\xi_{5}\xi_{6}+w_{21}\xi_{6}^{2}.$

V-D Observations

After a short transient time, the error trajectory ${e}(t)$ = ${x}(t)-{x_{d}}(t)$ converges to zero in both cases as shown in Fig. 3(b) and 3(f) respectively. Note that the control input ${u}={\dot{\theta}}$ remain within the maximum joint velocity limits shown in Fig. 3(c) and Fig. 3(g). However unlike in optimal regulation, it does not converge to zero as the velocity compensation is required for perfect tracking control. The time history plot of the associated cost function is shown in shown in Fig. 3(d,h). The experimental validation for the corresponding target pose was performed using a real UR 10 robot manipulator. The results from Fig. 4 shows that the end effector of UR 10 robot manipulator successfully reaches the target pose under the proposed control scheme and successfully follows the reference time varying circular trajectory starting from a random seed pose. The time history plot of the associated cost function is shown in Fig. 4(d,h). Unlike in simulations, the control effort shows some chatter due to the inertia of the real hardware.

V-E Quantitative Test Comparison

In order to quantify the performance of the proposed optimal kinematic control, an automated test process was used where a large, statistically-valid number of random samples were used as inputs. Kinematic models of Universal Robot (UR) 10 was used to demonstrate the tests. The quantitative test methodology for comparing the proposed method for kinematic control against the state-of-the-art kinematic control using an RNN [3] and Singluar Value Filtering (SVF) approach [26] is entailed.

First, a total of $1,000$ random samples and $1000$ different feasible elliptical trajectories were generated in the sample space for regulation and tracking respectively. The sample space is a cuboid volume of task space within robot’s reach and every sample is a pair of seed pose and target pose in the case of regulation. The three kinematic control schemes mentioned above are tested and compared on trajectory cost.

The trajectory cost is defined as normalized total cost: $V/N$ , where $V=\int_{t}^{\infty}{e(t)^{T}{Q}e(t)+u(t)^{T}Ru(t)+\dot{u}(t)^{T}R\dot{u}(t)}dt$ for the same $Q=I_{3\times 3}$ and $R=I_{6\times 6}$ and $N$ is the number of total sample instances. The design parameters were selected such that the maximum control effort is the same for all three approaches.

[TABLE]

\captionof

tableComparisons of different algorithms for Kinematic Control of UR10 Robot Manipulator The cost matrix contain two terms. One for the measure of optimal control action and another for smoothness of the motion. The comparison from Table I shows that the proposed kinematic control has the optimal control action than the state-of-the-art kinematic control.

VI Conclusion

In this work, we have designed an optimal kinematic controller for a robot manipulator using SNAC framework. A simple critic weight update law was proposed which ensured that the closed loop system becomes stable in the sense of Lyapunov while following an optimal trajectory. The robot was expected to reach a target position or follow a time varying trajectory in the task space while optimizing a global cost function. Using the proposed optimal regulation and optimal tracking framework in the context of SNAC, it has been experimentally demonstrated that robot performs desired tasks with optimal cost as evident from Table 1.