On Model Adaptation for Sensorimotor Control of Robots

TL;DR

This paper develops adaptive sensorimotor models for robotic control under uncertainty, enabling robots to handle uncalibrated sensors and unknown object properties through novel computational methods.

Contribution

It introduces new methods for building adaptive sensorimotor models, addressing uncalibrated sensors and uncertain object properties in robotic control.

Findings

Effective shape control of deformable objects demonstrated.

Successful soft manipulation of ultrasonic probes with uncalibrated sensors.

Analysis of computational methods for adaptive sensorimotor modeling.

Abstract

In this article, we address the problem of computing adaptive sensorimotor models that can be used for guiding the motion of robotic systems with uncertain action-to-perception relations. The formulation of the uncalibrated sensor-based control problem is first presented, then, various computational methods for building adaptive sensorimotor models are derived and analysed. The proposed methodology is exemplified with two cases of study: (i) shape control of deformable objects with unknown properties, and (ii) soft manipulation of ultrasonic probes with uncalibrated sensors.

Equations

$${\mathbf{x}}_{t+1}-{\mathbf{x}}_t={\mathbf{u}}_t$$

$$({\mathbf{x}}_{t+1}-{\mathbf{x}}_t)/\mathrm{d}t={\mathbf{u}}_t/\mathrm{d}t\approx {\mathbf{v}}_t$$

$${\mathbf{y}}_t=f({\mathbf{s}}_t)=f(g({\mathbf{x}}_t)):{\mathbb{R}}^n\mapsto {\mathbb{R}}^m$$

$${\mathbf{y}}_{t+1}={\mathbf{y}}_t+{\mathbf{A}}_t{\mathbf{u}}_t$$

$$J={\Vert {\mathbf{A}}_t{\mathbf{u}}_t+\lambda \mathrm{sat}({\mathbf{y}}_t-{\mathbf{y}}^{\ast })\Vert }^2$$

$${\mathbf{A}}_t^{⊺}{\mathbf{A}}_t{\mathbf{u}}_t=-\lambda {\mathbf{A}}_t^{⊺}\mathrm{sat}({\mathbf{y}}_t-{\mathbf{y}}^{\ast })$$

$${\mathbf{u}}_t=-\lambda {\mathbf{A}}_t^{⊺}{\left({\mathbf{A}}_t{\mathbf{A}}_t^{⊺}\right)}^{-1}\mathrm{sat}(\mathbf{y}-{\mathbf{y}}^{\ast })$$

$${\mathbf{u}}_t=-\lambda {\left({\mathbf{A}}_t^{⊺}{\mathbf{A}}_t\right)}^{-1}{\mathbf{A}}_t^{⊺}\mathrm{sat}(\mathbf{y}-{\mathbf{y}}^{\ast })$$

$${\mathbf{y}}_t=f(g({\mathbf{x}}_t))=\mathbf{L}({\mathbf{x}}_t)\mathbf{\pi }$$

$$U=\frac{\gamma }{2}\sum _{k=1}^T{\Vert \mathbf{L}({\mathbf{x}}_k){\hat{\mathbf{\pi }}}_t-{\mathbf{y}}_k\Vert }^2$$

$${\hat{\mathbf{\pi }}}_{t+1}={\hat{\mathbf{\pi }}}_t-\nabla U({\hat{\mathbf{\pi }}}_t)$$

$${\hat{\mathbf{A}}}_t=\frac{\partial }{\partial {\mathbf{x}}_t}\left\{\mathbf{L}({\mathbf{x}}_t){\hat{\mathbf{\pi }}}_t\right\}$$

$${\hat{\mathbf{A}}}_t={\hat{\mathbf{A}}}_{t-1}+\Gamma \frac{{\mathbf{\delta }}_t-{\hat{\mathbf{A}}}_{t-1}{\mathbf{u}}_t}{{\mathbf{u}}_t^{⊺}{\mathbf{u}}_t}{\mathbf{u}}_t^{⊺}$$

$$V=\frac{\gamma }{2}{\Vert {\hat{\mathbf{A}}}_t{\mathbf{u}}_t-{\mathbf{\delta }}_t\Vert }^2$$

$${\hat{a}}_{t+1}^{ij}={\hat{a}}_t^{ij}-\nabla V({\hat{a}}_t^{ij})$$

$$z^l=\left\{\begin{array}{cc}{\overline{\mathbf{x}}}^l& {\hat{\mathbf{A}}}^l\end{array}\right\},\text{for}l=1,\dots ,N$$

$$W^l=\frac{\gamma }{2}\sum _{j\in \mathcal{B}}{h}^{lj}{\Vert {\hat{\mathbf{A}}}^l{\mathbf{u}}_j-{\mathbf{\delta }}_j\Vert }^2$$

$$h^{lj}=\exp \left(-\frac{\Vert {\overline{\mathbf{x}}}^l-{\mathbf{x}}^j{\Vert }^2}{2{\sigma }^2}\right)$$

$${\hat{a}}_{t+1}^{l,ij}={\hat{a}}_t^{l,ij}-\nabla W({\hat{a}}_t^{l,ij})$$

$$l=\underset{j}{\mathrm{argmin}}\left\{\Vert {\overline{\mathbf{x}}}^j-{\mathbf{x}}_t\Vert \right\}$$

$${\mathbf{y}}_t={\left[\begin{array}{cc}\kappa & \theta \end{array}\right]}^{⊺}$$

$${\mathbf{y}}_t={\left[\begin{array}{ccc}\mu & \phi & \mathbf{\omega }\end{array}\right]}^{⊺}$$

Full text

00footnotetext: This work is supported in part by the Research Grants Council (RGC) of Hong Kong under grant number 14203917, and in part by PROCORE-France/Hong Kong Joint Research Scheme sponsored by the RGC and the Consulate General of France in Hong Kong under grant F-PolyU503/18.

1 INTRODUCTION

Sensor-based control encompasses a family of methods that exploit feedback information from (typically external) sensors for controlling the robot’s motion, and in general, its behaviour. On its most fundamental form, it can trace back its origins to the servomechanism problem [1]. Some common examples are visual servoing [2], tactile/force servoing [3, 4], proximity servoing [5], aural servoing [6], deformation/shape servoing [7, 8], to name a few cases.

To effectively execute these types of motion tasks, sensor-based controls invariably require some knowledge (at least approximated) of how the robot’s motor commands transform into sensory signals. This information is captured by the sensorimotor model of the robotic system [9], which besides coordinating action with perception, it can also be used to anticipate the effect that an input motor control will produce on the output sensor measurements [10]. Note, however, that if this information is not known (or is highly uncertain), the robot cannot properly coordinate its motions with sensory feedback.

Existing methods to obtain sensorimotor models require either exact knowledge of its analytical structure [11, 12] (which might not be known) or only compute instantaneous local estimations of it [13] (therefore, they cannot globally describe and control the system). Compared to these computational approaches, the human brain has a remarkable degree of adaptability that allows it to learn new sensorimotor relations from birth through death and under multiple morphological and perceptual conditions (see e.g. the pioneering study [14]). Humans can easily coordinate hand motions through a mirror, position unknown tools attached to the body, and even recover (some) mobility after strokes.

Our aim in this paper is precisely to address the design of computational methods that efficiently provide sensor-guided robots with robust adaptation capabilities. For that, we first formulate the sensorimotor control problem of robots using uncertain perceptual/motor models. Next, we formulate various structure-based and structure-free methods to adaptively compute these unknown relations. Finally, the presented methodology is exemplified with two cases of study and discussions about its implementation are given.

The contribution of this work is that it presents a general and intuitive methodology that can be used as a guideline or even a tutorial for researchers working on adaptive sensor-based control of robots with uncertain models. The proposed control approach can be used to guide the motion of different robotic platforms (e.g. manipulators, omnidirectional robots, active robot heads) with various sensing modalities (e.g. vision, audio, thermal, attitude, proximity).

The rest of this manuscript is organised as follows: Sec. 2 presents the preliminaries of the problem; Sec. 3 derives different adaptive estimation algorithms; Sec. 4 presents the cases of study; Sec. 5 gives final conclusions.

2 PRELIMINARIES

2.1 Notation

Along this note we use very standard notation. Column vectors are denoted with bold small letters $\mathbf{m}$ and matrices with bold capital letters $\mathbf{M}$. Time evolving variables are represented as $\mathbf{m}_{t}$, where the subscript $\ast_{t}$ denotes the discrete time instant, or, the iteration step. Gradients of functions $b=\beta(\mathbf{m}):\mathcal{M}\mapsto\mathcal{B}$ are denoted as $\nabla\beta(\mathbf{m})=(\partial\beta/\partial\mathbf{m}{)}^{\intercal}$.

2.2 Control Architecture

Consider a class of fully-actuated robotic systems whose configuration (e.g. modelling the end-effector pose in a manipulator) is denoted by the vector $\mathbf{x}_{t}\in\mathbb{R}^{n}$. In our formulation of the problem, it will be assumed that the motion of robotic system is commanded via a standard position/velocity controlled interface [15, 16] (which is typically found in the large majority of commercial robotic platforms). With position interfaces, the control commands $\mathbf{u}_{t}\in\mathbb{R}^{n}$ are given in terms of differential displacement motions as follows:

[TABLE]

All methods presented in this note are formulated using the above described position controls, yet, these can be easily transformed into its velocity control equivalent $\mathbf{v}_{t}\in\mathbb{R}^{n}$ by dividing both sides of (1) by the time step $\mathop{}\!\mathrm{d}t$ of the servo-loop as

[TABLE]

2.3 Configuration Dependant Feedback

To perform a sensorimotor task, the robot is equipped with a set of $r$ sensors (not necessarily of the same modality) that continuously measure physical quantities whose values depend on the robot’s configuration. This situation means that relative robot motions produce relative sensory changes. Some examples of configuration dependent signals (measured using either external or internal/on-board sensors) are: geometric visual features, observed end-effector poses, forces applied onto a surface, proximity to an object, intensity of an audio source, temperature from a heat source, ultrasound images from probe, etc.

The feedback signal from the $i$th sensor is denoted by the vector $\mathbf{s}_{t}^{i}=g^{i}(\mathbf{x}_{t})\in\mathbb{R}^{l_{i}}$, where the function $g^{i}:\mathbb{R}^{n}\mapsto\mathbb{R}^{l_{i}}$ represents the analytical sensor model that statically relates the instantaneous configuration of the robot with the feedback signal. All these signals can be conveniently grouped into a single vector $\mathbf{s}_{t}=[{\mathbf{s}_{t}^{1}}^{\intercal},\ldots,{\mathbf{s}_{t}^{r}}^{\intercal}{]}^{\intercal}=g(\mathbf{x}_{t})\in\mathbb{R}^{l}$. Sensorimotor controls often require to construct a vector of meaningful features to quantify and guide the task [2]. To this end, let us introduce the (possibly nonlinear) vectorial functional

[TABLE]

for $m$ as the number of feedback feature coordinates (along this note, we assume that $f(g(\cdot))$ is a smooth functional). There are three cases with this configuration-dependent structure: $n\geq m$ (more controls than features), $n\leq m$ (more features than controls), and $n=m$ (same number of features and controls). These cases have different properties that determine the controllability of $\mathbf{y}_{t}$ by the robot.

2.4 Sensorimotor Control Problem

The differential expression that describes how the motor actions result in changes of feedback features is represented by the first-order difference model:

[TABLE]

for $\mathbf{A}_{t}=[\partial f/\partial\mathbf{s}_{t}][\partial g/\partial\mathbf{x}_{t}]\in\mathbb{R}^{m\times n}$ as the Jacobian matrix of the system (also known as the interaction matrix in the sensor servoing literature [17]), whose elements depend on the instantaneous configuration $\mathbf{x}_{t}$.

The sensorimotor control problem consists in coordinating the motor actions with the feedback signals such that a desired sensory behaviour is achieved. Without loss of generality, such behaviour is characterised as the set-point regulation of $\mathbf{y}_{t}$ towards a constant sensory target $\mathbf{y}^{*}$. The necessary actions $\mathbf{u}_{t}$ to approach the target can be computed by minimising the quadratic cost function111The rational behind the minimisation of (5) is to find a motor command $\mathbf{u}_{t}$ that projects into the sensory space as a vector pointing towards $\mathbf{y}^{*}$.:

[TABLE]

for $\lambda>0$ as feedback gain, and $\operatorname{sat}(\cdot)$ as a vectorial saturation function to ensure that $\mathbf{u}_{t}$ satisfies the differential motion condition in (1). By computing the extremum $\nabla J(\mathbf{u}_{t})=\mathbf{0}$, we obtain the normal equation

[TABLE]

which exposes the different properties of the three cases relating the relative dimensions of $\mathbf{y}_{t}$ and $\mathbf{x}_{t}$.

For $n>m$, the solution to the problem can be obtained from (5) via the right pseudo-inverse of $\mathbf{A}_{t}$ as follows [18]:

[TABLE]

Note that the above motor action will globally minimise (5) (i.e. $\|\mathbf{y}_{t}-\mathbf{y}^{*}\|\rightarrow 0$), as long as the $m$ feature coordinates in $\mathbf{y}_{t}$ are linearly independent with respect to $\mathbf{x}_{t}$. This ensures that the matrix $\mathbf{A}_{t}\mathbf{A}_{t}^{{\intercal}}$ can be inverted.

For $m>n$, the solution is obtained by solving the normal equation (6) for $\mathbf{u}_{t}$, which yields:

[TABLE]

Substituting (8) into (5) shows that the cost function can only be locally minimised (i.e. $\|\mathbf{y}_{t}-\mathbf{y}^{*}\|\rightarrow\eta$, for $\eta>0$). The use of redundant features is useful in practice to cope with intermittent feedback from sensors, such as in the case of camera occlusions or malfunctions.

For the trivial case of $n=m$, the matrix $\mathbf{A}_{t}$ is square, therefore, the solution is simply obtained via standard matrix inversion $\mathbf{u}_{t}=-\lambda\mathbf{A}_{t}^{-1}\operatorname{sat}(\mathbf{y}-\mathbf{y}^{*})$.

3 CONTINUOUS MODEL ADAPTATION

3.1 Uncertain Sensorimotor Models

Computing any of the above motor actions requires some knowledge (at least coarse) of the transformation matrix $\mathbf{A}_{t}$, which in turn, depends on the sensor and the feature models $g(\cdot)$ and $f(\cdot)$. However, if the estimated model is corrupted at some point in time, the robot may no longer properly coordinate action with perception. This situation may happen when the mechanical structure of the robot is altered (e.g. due to bendings or damage of links) or when the configuration of the perceptual system is changed (e.g. due to relocation of external sensors).

The capability to dynamically estimate sensorimotor models is needed to use robots in many growing (and economically important) fields such as domestic/social robotics, field robotics, autonomous systems, etc, where the sensorimotor conditions are highly uncertain. Several methods have been proposed to compute or approximate these models (see [19] for a comprehensive survey on the topic). In this paper, we coarsely classify these methods into the following two approaches: structure-based estimation and structure-free estimation. In the following sections, we present the model adaptation problem and provide various solutions to it.

3.2 Structure-Based Model Adaptation

These types of algorithms represent calibration-like techniques that aim to estimate the parameters $\boldsymbol{\pi}\in\mathbb{R}^{p}$ of the uncertain model. Its implementation requires exact knowledge of the analytical structure of the model $\mathbf{y}_{t}=f(g(\mathbf{x}_{t})$, which for ease of presentation, we assume it can be linearly parametrisable with respect to $\boldsymbol{\pi}$ as follows:222For non-linear model parametrisations, other types of optimisation algorithms (namely non-convex) must be used, whose details are beyond the scope of this expository note.:

[TABLE]

where $\mathbf{L}(\mathbf{x}_{t})\in\mathbb{R}^{m\times p}$ represents a known regression-like matrix that captures the properties of the analytical model, and whose elements depend on the configuration vector $\mathbf{x}_{t}$.

To compute the vector of estimated parameters $\widehat{\boldsymbol{\pi}}_{t}\in\mathbb{R}^{p}$, structure-based methods require to first collect a set of $T$ input-output observation points $(\mathbf{y}_{k};\mathbf{x}_{k})$, for $k=1,\ldots,T$ (see e.g. [12]). Standard methods use this data for computing a quadratic cost function of the following form:

[TABLE]

for $\gamma>0$ as a learning gain. By using (9) and after some algebraic operations, it is easy to show that the above function is convex with respect to the estimation error vector $\widehat{\boldsymbol{\pi}}_{t}-\boldsymbol{\pi}$. Therefore, $U$ can be adaptively minimised with the gradient descent rule333Along this note, the detailed expressions of function gradients are omitted, yet, these can be easily computed after simple analytical calculations.:

[TABLE]

which in the absence of measurement noise and for a sufficient number $T$ of linearly independent observations, it globally minimises the cost $U$ (i.e. $\|\widehat{\boldsymbol{\pi}}_{t}-\boldsymbol{\pi}\|\rightarrow\mathbf{0}$, yet, a small estimation error is typically expected in practice). The adaptive transformation matrix is then simply computed as:

[TABLE]

Structure-based approaches have one major disadvantage: its dependency on fixed analytical models. Note that since the model’s structure (9) is explicitly used within the adaptation algorithm (11), these methods are not robust to unknown changes in the mechanical and perceptual conditions. Furthermore, in many situations, the analytical model might not be available or subject to large uncertainties. In the case of advanced robots whose morphology is constantly evolving, fixed analytical models will clearly fail to capture the system’s properties. These issues limit the applicability of structure-based approaches.

3.3 Structure-Free Model Adaptation

These types of algorithms have the capability to compute the unknown sensorimotor model in the following manner: (i) entirely from scratch (i.e. without requiring any a-priori knowledge of the model’s analytical structure), (ii) on-demand (i.e. they can modify its acquired structure at any time instant so as to identify new relations), and (iii) from data observations only (i.e. by using information from controls and measurements only).

Based on its computation, we coarsely classify these algorithms into the following two general categories: instantaneous estimation, and distributed estimation.

Instantaneous estimation.

As the name suggests, these techniques compute a matrix $\mathbf{A}_{t}$ that is only valid at the current (instantaneous) configuration $\mathbf{x}_{t}$. The Broyden rule [13] is one possible example of such technique. It iteratively computes $\mathbf{A}_{t}$ with the following update rule:

[TABLE]

for $\boldsymbol{\delta}_{t}=\mathbf{y}_{t+1}-\mathbf{y}_{t}$, and $0<\Gamma\leq 1$ as a tuning gain. With “high” gains $\Gamma\approx 1$, by right-multiplying (13) by $\mathbf{u}_{t}$ (namely, projecting the motor action into sensory space), we can see that $\mathbf{y}_{t+1}\approx\mathbf{y}_{t}+\widehat{\mathbf{A}}_{t}\mathbf{u}_{t}$ is satisfied. However, using high gains results in a noisy and rapidly changing matrix $\widehat{\mathbf{A}}_{t}$. For slow robot motions, the Jacobian matrix is expected to change slowly, therefore, using “small” gain values for $\Gamma$ can help to make the computation less responsive, i.e. $\widehat{\mathbf{A}}_{t}\approx\widehat{\mathbf{A}}_{t-1}$ as well as to filter out noisy measurements.

Another example of these techniques can be derived from the following (convex) cost function [20]:

[TABLE]

which provides a metric of the accuracy of $\widehat{\mathbf{A}}_{t}$ at the current configuration $\mathbf{x}_{t}$. The terms of this unknown matrix are continuously adapted with the rule:

[TABLE]

where the scalar $\widehat{a}_{t}^{ij}$ denotes $i$th-row $j$th-column term in $\widehat{\mathbf{A}}_{t}$.

With instantaneous estimation techniques, the matrix $\mathbf{A}_{t}$ must be continuously recalculated with new sensor observations as the robot moves into other configurations. These types of adaptation techniques do not provide a mechanism for preserving knowledge (i.e. remembering models) of previous configurations.

Distributed estimation.

Note that since the feedback feature functional (3) is smooth (i.e. differentiable), its Jacobian matrix is expected to smoothly change along the configuration space $\mathbf{x}_{t}$. This means that a local Jacobian matrix estimated at a particular configuration point is also valid around the neighbourhood surrounding it. This simple, yet powerful, idea forms the working principle of distributed estimation techniques. With these adaptive algorithms, the estimation problem is shared amongst multiple computing units that specialise in a local transformation.

There are many possible implementations of distributed estimation algorithms. Along this note, we briefly describe a variety based on self-organising maps (see [21] for details). Consider a network of $N$ computing units spread around the robot’s configuration space $\mathbf{x}_{t}$. The following data structure $z^{l}$ is associated with each computing unit:

[TABLE]

where $\widehat{\mathbf{A}}^{l}$ stands for a local approximation of $\mathbf{A}_{t}$ estimated at the configuration point $\overline{\mathbf{x}}^{l}$. There are various methods for establishing the distribution of these units around the robot’s workspace, e.g. based on self-organising rules, evenly distributed locations, random point distributions, etc. [22]. For ease of presentation, we assume that the location of these $N$ configuration points $\overline{\mathbf{x}}^{l}$ associated with the units has already been established.

Estimation of local transformation matrices is performed by first collecting a data set of $T$ observation points $(\boldsymbol{\delta}_{k};\mathbf{u}_{k})$, for $k=1,\ldots,T$. Then, the following local cost function for the $l$th unit is computed:

[TABLE]

for $\mathcal{B}$ as a local ball centred at the $l$th unit, and $h^{lj}$ as its Gaussian neighbourhood function computed as

[TABLE]

where $\sigma>0$ determines the ball’s radius. In this method, the idea is to make use of neighbouring data (whose contribution decreases with its distance to the centre unit $l$) for approximating the local transformation matrix. The update rule to adaptively compute the $i$th row $j$th column term of $\widehat{\mathbf{A}}^{l}_{t}$ is as follows:

[TABLE]

Once the $N$ local cost functions (17) have been minimised, the network is trained to perform sensorimotor transformations with each of its units. In order to implement motor commands as the ones derived in Sec. 2.4, a local transformation matrix that best matches the current position $\mathbf{x}_{t}$ must be retrieved from the $l$ units. This can be done by solving the search problem

[TABLE]

Note that this adaptation approach can be combined with the previous instantaneous estimation technique (or possibly others) by defining a cost function $H^{l}=V+W^{l}$ that exploits the current feedback measurements. This allows to also quantify the accuracy of the model based on new sensory data; the cost function can then be minimised with similar gradient descent tools as before.

Compared to the previous estimation approaches, distributed estimation requires more data to approximate the sensorimotor model. However, these methods can effectively preserve local knowledge within its computing units and does not require any prior analytical representation of the robot’s action-to-perception relations.

4 CASES OF STUDY

4.1 Visual Shape Servoing of Elastic Objects

To exemplify our methodology, consider the setup depicted in Fig. 1, where a 3D camera captures point clouds of a beam-like elastic object manipulated by a robot. Let us denote the captured 3D points by $\mathbf{s}^{i}_{t}$, for $i=1,\ldots,r$ (note that the number $r$ is generally in the order of hundreds). The task to be performed is to automatically deform the object into a desired shape. We can use the point cloud to approximate a the object’s backbone (represented as the blue curve in Fig. 1). With this geometric information, we compute the feature vector defined as follows:

[TABLE]

for $\kappa>0$ as the object’s curvature, and $\theta$ as the angle of the object’s bending with respect to it’s frame, see [23] for details. For this task, model adaptation can be performed with distributed estimation algorithms. These approaches provide an efficient solution to the highly nonlinear transformation problem of relating robot poses to object deformations (note that deformation models are hard to compute analytically). For that, several computing units must be first defined at key end-effector poses, e.g. ranging from fully stretched to varying bending configurations; local sensory observations can then be collected for approximating the model. Since $n>m$, the motor command is computed as in (7), with a target shape defined as $\mathbf{y}^{*}=[\kappa^{*},\theta^{*}{]}^{\intercal}$.

4.2 Multi-Modal Scanning with Ultrasound Probes

Consider the setup in Fig. 2, which depicts a robot performing automatic scanning of tissues with an ultrasound probe [24] (we assume the robot has 6-DOF). This system is instrumented with a force/torque sensor and a 3D camera. Let us denote by $\varphi$ the (normal) force applied onto tissues, by $\mu$ the image location of the ultrasound feature of interest, and by $\boldsymbol{\omega}$ the probe’s 3D orientation. The task to be performed is conveniently described with respect to the body’s 3D frame. It consists in positioning $\mu$ at the centre of the ultrasound image, while applying a desired normal force and controlling the probe’s pose over the tissues. Note that this relative orientation can be computed from the 3D point clouds $\boldsymbol{\omega}=q(\mathbf{s}_{t}^{1},\ldots,\mathbf{s}_{t}^{r})$. The task’s feature vector is defined as

[TABLE]

The models for the above feature coordinates are simple to analytically derive, namely using Hooke’s law for $\varphi$, horizontal image displacements for $\mu$, and affine/homogeneous transformations for $\boldsymbol{\omega}$. Therefore, model adaptation can be performed using structure-based algorithms as in (9). With these approaches, we can robustify the sensor-guided task by continuously calculating unknown task parameters such as: stiffness of soft tissues, relative location of ultrasound features, and robot-camera-body transformations. Since $n>m$, the motor command is also computed as in (7), with a set-point feature defined as $\mathbf{y}^{*}=[\mu^{*},\varphi^{*},\boldsymbol{\omega}^{*}{]}^{\intercal}$.

5 CONCLUSIONS

In this expository paper, we addressed the problem of computing adaptive sensorimotor models for robots with uncalibrated sensory feedback and/or uncertain morphology. A general sensor servoing approach was first formulated using an energy minimisation approach. Then, we derived various methods for providing these controllers with continuous model adaptation capabilities. Two cases of study we presented to illustrate the proposed methodology.

The presented sensorimotor controls are formulated based on the assumption that the feedback signals dependent on the robot’s configuration only. Although this condition can be fairly used to represent many sensor-guided applications, it may not be the most accurate model for describing tasks where the measurements also depend additional variables (e.g. manipulating fabrics with infinite dimensional configurations) or even time-varying states (e.g. controlling the effect of cosmetic lasers stimulating skin tissues). The development of more general sensor models is still an open research problem.

The presented model adaptation methods allow robots to perform sensor-guided tasks even when its sensorimotor model is not known or might suddenly change. For example, robots can adapt to unknown sensor configurations and/or morphologies. By understanding the principle of how sensors models can be effectively created from scratch and adapted on-the-fly, we hope to build machines with more resilient properties that allow them to perform long-term operations with minimal supervision. These advanced capabilities are needed to advance towards building truly autonomous robots.