Walking Control Based on Step Timing Adaptation

TL;DR

This paper introduces a simple yet effective walking pattern generator for biped robots that adaptively selects step timing and location at each cycle, enhancing gait robustness without complex multi-step optimization.

Contribution

It proposes a novel, computationally efficient method for real-time step timing and location adaptation that guarantees gait viability, especially useful for robots with limited control authority.

Findings

Successfully stabilizes walking under external pushes

Handles foot slippage effectively

Demonstrates robustness with passive ankles in simulations

Abstract

Step adjustment can improve the gait robustness of biped robots, however the adaptation of step timing is often neglected as it gives rise to non-convex problems when optimized over several footsteps. In this paper, we argue that it is not necessary to optimize walking over several steps to ensure gait viability and show that it is sufficient to merely select the next step timing and location. Using this insight, we propose a novel walking pattern generator that optimally selects step location and timing at every control cycle. Our approach is computationally simple compared to standard approaches in the literature, yet guarantees that any viable state will remain viable in the future. We propose a swing foot adaptation strategy and integrate the pattern generator with an inverse dynamics controller that does not explicitly control the center of mass nor the foot center of pressure. This is particularly useful for biped robots with limited control authority over their foot center of pressure, such as robots with point feet or passive ankles. Extensive simulations on a humanoid robot with passive ankles demonstrate the capabilities of the approach in various walking situations, including external pushes and foot slippage, and emphasize the importance of step timing adaptation to stabilize walking.

Tables

Table 1. TABLE I: Hierarchy of tasks used in the humanoid simulation.

Rank	Nr. of eq/ineq constraints	Constraint/Task
1	6 eq	Centroidal momentum dynamics
	2$\times$4 ineq	Torque limits
2	6 eq	Stance foot constraint
	1 eq	CoM height control
3	3 eq	Swing foot control
4	2$\times$4 eq	Posture control
5	2$\times$6 eq	Force regularization

Table 2. TABLE II: Physical properties of the abstract model.

Value	Description	min	max
$L$	Step length	$-50\mathrm{cm}$	$50\mathrm{cm}$
$W_{right}$	Step width (right)	$-10\mathrm{cm}$	$20\mathrm{cm}$
$W_{left}$	Step width (left)	$-20\mathrm{cm}$	$10\mathrm{cm}$
$T$	Step duration	$0.2s$	$0.6s$

Equations

$$\ddot{x}={\omega }_0^2(x-u_0)$$

$$\dot{x}={\omega }_0(\xi -x)$$

$$\dot{\xi }={\omega }_0(\xi -u_0)$$

$$\xi (t)=({\xi }_0-u_0)e^{{\omega }_{0}t}+u_0$$

$${\xi }_T=({\xi }_0-u_0)e^{{\omega }_{0}T}+u_0$$

$$b={\xi }_T-u_T$$

$$u_T=({\xi }_{cur}-u_0)e^{{\omega }_0(T-t)}+u_0-b,0\le t\le T$$

$$b_x=\frac{L}{e^{{\omega }_{0}T}-1}$$

$$b_y=(-1{)}^n\frac{l_p}{1+e^{{\omega }_{0}T}}-\frac{W}{1-e^{{\omega }_{0}T}}$$

$$d_{\infty }=L_{max}\frac{e^{-{\omega }_0{T}_{min}}}{1-e^{-{\omega }_0{T}_{min}}}$$

$$b_{x,max}=\frac{L_{max}}{e^{{\omega }_0{T}_{min}}-1}$$

$${\xi }_{x,0}-u_{x,0}=\frac{L_{max}}{e^{{\omega }_0{T}_{min}}-1}+ϵ$$

$$b_{x,T}=(\frac{L_{max}}{e^{{\omega }_0{T}_{min}}-1}+ϵ)e^{{\omega }_{0}T}-(u_{x,T}-u_{x,0})$$

$$b_{x,T}=\frac{L_{max}}{e^{{\omega }_0{T}_{min}}-1}+ϵe^{{\omega }_0{T}_{min}}$$

$${\xi }_{x,0}-u_{x,0}\le \frac{L_{max}}{e^{{\omega }_0{T}_{min}}-1}$$

$$b_{x,T}\le (\frac{L_{max}}{e^{{\omega }_0{T}_{min}}-1})e^{{\omega }_{0}T}-(u_{x,T}-u_{x,0})$$

$$b_{x,T}\le \frac{L_{max}}{e^{{\omega }_0{T}_{min}}-1}$$

$$v_x=\frac{L_{nom}}{T_{nom}},v_y=\frac{W_{nom}}{T_{nom}}$$

$$L_{min}\le L_{nom}\le L_{max}$$

$$W_{min}\le W_{nom}\le W_{max}$$

$$T_{min}\le T_{nom}\le T_{max}$$

$$T_{nom}=\frac{B_l+B_u}{2}$$

$$L_{nom}=v_x(\frac{B_l+B_u}{2}),W_{nom}$$

$$B_l=\max \{\frac{L_{min}}{|v_x|},\frac{W_{min}}{|v_y|},T_{min}\},v_x,v_y\ne 0$$

$$B_u=\min \{\frac{L_{max}}{|v_x|},\frac{W_{max}}{|v_y|},T_{max}\},v_x,v_y\ne 0$$

$$\tau =e^{{\omega }_{0}T}\text{or}T=\frac{1}{{\omega }_0}\log (\tau )$$

$$u_T-({\xi }_{mea}-u_0)e^{-{\omega }_{0}t}\tau +b=u_0,0\le t\le T$$

$$\underset{u_T,\tau ,b}{\mathrm{argmin}}$$

$$+{\alpha }_3\Vert b-\left[\begin{array}{c}{b}_{x,nom}\\ b_{y,nom}\end{array}\right]{\Vert }^2$$

$$\left[\begin{array}{c}{L}_{min}\\ W_{min}\end{array}\right]\le u_T-u_0\le \left[\begin{array}{c}{L}_{max}\\ W_{max}\end{array}\right]$$

$$e^{{\omega }_0{T}_{min}}\le \tau \le e^{{\omega }_0{T}_{max}}$$

$$u_T+b=({\xi }_{mea}-u_0)e^{-{\omega }_{0}t}\tau +u_0$$

$$\left[\begin{array}{c}{b}_{x,min}\\ b_{y,max,out}\end{array}\right]\le b\le \left[\begin{array}{c}{b}_{x,max}\\ b_{y,max,in}\end{array}\right]$$

$$\underset{c_i}{\mathrm{argmin}}$$

$$0\le z(t)\le z_{max}$$

$$z_{f,d}(0)=0z_{f,d}(t_{k-1})=z_{k-1}{z}_{f,d}(T)=0$$

$${\dot{z}}_{f,d}(0)=0{\dot{z}}_{f,d}(t_{k-1})={\dot{z}}_{k-1}{\dot{z}}_{f,d}(T)=0$$

$${\ddot{z}}_{f,d}(0)=0{\ddot{z}}_{f,d}(t_{k-1})={\ddot{z}}_{k-1}{\ddot{z}}_{f,d}(T)=0$$

Full text

Step adjustment can improve the gait robustness of biped robots, however the adaptation of step timing is often neglected as it gives rise to non-convex problems when optimized over several footsteps. In this paper, we argue that it is not necessary to optimize walking over several steps to ensure gait viability and show that it is sufficient to merely select the next step timing and location. Using this insight, we propose a novel walking pattern generator that optimally selects step location and timing at every control cycle. Our approach is computationally simple compared to standard approaches in the literature, yet guarantees that any viable state will remain viable in the future. We propose a swing foot adaptation strategy and integrate the pattern generator with an inverse dynamics controller that does not explicitly control the center of mass nor the foot center of pressure. This is particularly useful for biped robots with limited control authority over their foot center of pressure, such as robots with point feet or passive ankles. Extensive simulations on a humanoid robot with passive ankles demonstrate the capabilities of the approach in various walking situations, including external pushes and foot slippage, and emphasize the importance of step timing adaptation to stabilize walking.

I Introduction

I-A Motivation

Due to the unilateral nature of feet-ground interaction, legged robots can easily fall down. The most important aspect of walking control is therefore preventing falls even in face of strong perturbations. In this regard, a controller can act on three different aspects of the gait:

1. where to step, 2) when to step, and 3) how to manipulate ground reaction forces. State of the art walking pattern generators mostly focus on the third aspect, i.e, the optimal modulation of the center of pressure (CoP), sometimes in conjunction with step placement or timing adaption. While such strategies afford great flexibility in walking pattern generation, they implicitly assume high control authority on the end-effectors’ CoP. Furthermore, since adapting step timing often gives rise to non-convex optimization problems, most available walking pattern generators keep footstep timing fixed.

In this paper, we study the problem of optimal footstep location and timing adaptation without explicit control of the feet CoP or the robot center of mass (CoM). This enables the relaxation of control constraints on the CoP associated to more traditional receding horizon algorithms, which is particularly relevant for biped robots with no or limited control authority over their CoP: robots with passive ankles, small or point feet.

I-B Realtime walking pattern generators

To date, the most successful approaches for real-time walking control mostly consider linear models of the CoM dynamics and especially the linear inverted pendulum model (LIPM) [27]. Indeed, more complex models often lead to high dimensional, non-convex and computationally complex algorithms, therefore limiting their applicability for real-time planning and control [37, 32, 7, 23]. Recent work for quadruped locomotion also consider more descriptive dynamic models than the LIPM while retaining the capability of running algorithms in real-time. For instance, assuming the legs are massless and the base rotation is small, [10] uses a linear dynamic model between the base states and contact forces for real-time motion planning. [17] proposes an efficient motion planner and controller based on differential dynamic programming (DDP) that uses an approximation of the robot base dynamics. Although these proposed model predictive control (MPC) approaches are real-time capable, they need another reactive planner level to generate contact sequences. There are also formulations based on more descriptive models with the capability to reason about contact sequence and timing, for example using a relaxed contact model [39] or adding implicitly complementary constraints [55]. However, the resulting optimization problems are not convex and can easily get stuck in undesired local minima. In bipedal locomotion, the LIPM is still often preferred because legs constitute a considerable amount of the robot’s total mass and there is little need to vary so much the CoM height.

Leveraging analytical solutions of the LIPM, several approaches generate CoM trajectories consistent with a predefined ZMP trajectory [19, 38, 4]. In these approaches, both the position and velocity of the CoM are restricted which constrains both divergent and convergent parts of the LIPM dynamics. In contrast, [49] constrained only the divergent part of the CoM dynamics to generate a trajectory for the divergent component of motion (DCM) based on predefined footprints (ZMP trajectory). Prior to [49], the divergent part of the LIPM dynamics was also used to explain human walking characteristics under the name of extrapolated center of mass (XCoM) [24]. This concept is equivalent to the original capture point (CP) idea [44], i.e. the point on which the robot should step to come to a stop. In these methods, there is no feedback from the current state of the robot to adapt the CoM motion in the presence of disturbances. To circumvent this, [12] proposed a feedback law tracking a DCM trajectory. Although this controller can quickly react to disturbances, perfect DCM tracking assumes unconstrained CoP manipulation.

All of these walking pattern generators can be seen as variants of the same MPC scheme [54]. One of the pioneering work relating walking pattern generation to optimal control [26] proposed a preview control method to generate CoM trajectories based on predefined ZMP trajectories. Feedback from the current robot state was used to recompute and adapt the motion online. [52] improved the performance of this approach in the presence of relatively severe pushes, by constraining the motion of the ZMP inside the support polygon rather than predefining a desired ZMP trajectory. The resulting algorithm computes both ZMP and CoM trajectories respecting feasibility constraints at each control cycle. In all these approaches, the receding horizon is several foot steps long with fixed step locations and timing.

I-C Step adjustment and timing adaptation

Explicit manipulation of the CoP to control the DCM or CoM imposes several restrictions on the leg design. Indeed, robots with point contact feet [25] or robots with passive ankles [30] have very limited control authority, if any, on the foot CoP. The modulation of the CoP is also very limited for robots with actuated ankles, due to the rather small foot support area and the limited amount of available ankle torque. This is in contrast to step adjustment, which allows to select the next step location in a relatively large area compared to the support polygon. It constitutes therefore a more significant tool for stabilizing biped walking. Step adaptation algorithms makes walking pattern generators more robust against disturbances. Successful approaches can either employ heuristics to adapt predefined step sequence based on DCM tracking error [12, 30], consider step location as a decision variable in an MPC framework [11, 20, 21], use constrained optimization to adapt step location [15, 28] or modify a preview control approach [50]. However, in all these methods step timing is never adapted as it would render the problem non-convex when considered over several footsteps.

One of the earliest work on step timing adaptation [38] used an analytical approach to generate CoM/CoP trajectories, and a step period adaptation algorithm to alleviate ZMP fluctuations in case of an immediate change of the step location. [40] proposed a three stage algorithm to adapt ZMP trajectory, step location and timing. First, preview control is used to adapt the ZMP and compensate for CoM tracking error. If the computed ZMP is outside the support polygon, a second stage extends or shortens the step period. Finally the third stage adapts the next step location. [14] showed that using a combination of step location and timing adaptation increases significantly the basin of attraction for bipedal locomotion. [36] proposed an analytical method for computing nominal gait variables for a desired walking velocity, and an algorithm for adapting both step location and timing based on heuristics. [8] proposed an analytical method for step timing and foot placement adaptation based on the CoM state feedback, with a priority given to the sagittal gait. [18] modified the analytical approach of [12] by adjusting both step location and timing using heuristics to compensate for the DCM tracking error. In addition to step timing, single support duration can also be adapted [41, 31] to deal with soon or late landing of the swing foot using contact detection.

Step duration was also used as an optimization variables in [1, 34]. However, these approaches result in non-convex optimization problems which are computationally expensive and do not guarantee convergence to a global minimum. [35] proposed an extension of the gait planning approach in [20] to adjust step duration. They related the problem through a mixed-integer quadratic program (MIQP) which has combinatorial complexity. In [3], a robust approach is proposed to deal with the nonlinearity introduced by adapting step timing in a receding horizon controller. Furthermore, [5, 6] used timing adaptation to limit the acceleration of the swing foot, during walking on uneven terrains.

I-D Viability and capturability constraints

Viability theory [2] is an appealing framework to discuss the stability of walking. The viability kernel includes all the states from which it is possible to avoid falling [51]. Computing this kernel is generally not possible but it has been argued [53] that it is sufficient to limit an integral of the states of the system over several previewed steps to guarantee long-term walking stability [54]. In general, viability can be guaranteed by setting a terminal condition on the states when considering several steps [49, 12], or by minimizing any derivative of the CoM over a sufficiently large horizon [26, 52, 20]. As a result, the majority of walking pattern generators today consider several steps of preview to optimize walking.

Capturability is the ability to come to a stop after a certain number of steps and was extensively analyzed for the LIMP and its extension in [33]. Based on an N-step capturability analysis, [9] proposed a heuristic approach that tries to find the gait parameters that bring the robot back to a desired gait cycle by taking the least number of steps. In [33], in particular, a bound for the infinite-step capturability set is computed, which is almost equivalent to computing the viablity set of the LIPM. It is a very valuable result, as this can guarantee viability without the need to consider several walking steps and without over-constraining the system, therefore significantly simplifying the walking pattern generator problem. To the best of our knowledge, this bound has never been used to synthesize a walking pattern generator.

I-E Contributions of the paper

In this paper, we use results characterizing the viability kernel of the LIPM [33] to argue that it is sufficient to only consider the next step location and timing to ensure walking stability from any viable state222Throughout the paper, we use the term walking stability in the sense of viability (i.e. weak invariance) [51]. We also define robustness as the ability to recover from external impulsive disturbances.. This means that adding a longer preview of steps, under the LIPM assumption, does not improve the ability to reject disturbances as long as both step timing and location are adapted for the next footstep. To the best of our knowledge, this insight has not been exploited in control synthesis, as state of the art receding horizon approaches always include sufficient conditions for viability over a preview of several steps despite it being unnecessary and computationally more complex.

Consequently, we design a model predictive controller that optimizes the next step location and, most importantly, timing at every control cycle and guarantees walking stability from any currently viable state. The algorithm consists in solving a quadratic program which contains at least an order of magnitude less decision variables than standard MPC walking algorithms [21]. To the best of our knowledge, it is the first algorithm able to adapt both step timing and location while keeping a convex problem. As our approach does not rely on CoP modulations, it is applicable to any biped robots, including robots with passive ankles and point feet.

We also propose a strategy to adapt online Cartesian swing foot trajectories to follow adapted step timing and location. Extensive simulations on a full humanoid robot with passive ankles using hierarchical inverse dynamics [22] demonstrate the capabilities of the approach for robots with limited control authority over their CoP. In particular, our experiments emphasize the importance of timing adaptation during walking. Interestingly, the full body controller does not require an explicit control of the CoP nor of the CoM horizontal motion. As a result, our approach allows the robot to handle a significant amount of disturbances including external pushes and foot slippage.

This paper extends our preliminary work [29] in three main directions. First, we present a complete viability analysis of the LIPM under footstep constraints and show that we can find the viability kernel by taking into account only the next footstep. This enables the formulation of a MPC walking controller with viability guarantees. Second, we present several push recovery simulations on a full humanoid robot with passive ankle and show that we can stabilize walking in the presence of strong pushes by merely adjusting step location and timing without any explicit control of the CoP or CoM. Finally, we present several slippage recovery simulations demonstrating that our approach is robust to stance foot slippage. To the best of our knowledge, slippage recovery for humanoid walking is seldom studied in the literature.

II Problem formulation and viability

II-A Fundamentals

The LIPM constrains the motion of the CoM on a plane by using a telescopic massless link connecting the CoP to the CoM [27]. Its dynamics is written as

[TABLE]

where $x\in\mathbb{R}^{2}$ is the CoM horizontal position (CoM height has a fixed value $z_{0}$), and $u_{0}\in\mathbb{R}^{2}$ is the CoP position on the floor. For point contact feet, $u_{0}$ is identical to the contact point. $\omega_{0}$ is the natural frequency of the pendulum ($\omega_{0}=\sqrt[]{g/z_{0}}$, where $g$ is the gravity constant). Using the DCM, $\xi=x+\dot{x}/\omega_{0}$, the LIPM dynamics can also be written as [12]

[TABLE]

Equation (2) explicitly reveals the stable and unstable parts of the LIPM dynamics, where the CoM converges to the DCM (2a) and the DCM is pushed away by the CoP (2b). Solving (2b) as an initial value problem gives

[TABLE]

and the DCM at the end of a step of duration $T$ is

[TABLE]

This representation is very convenient, as it is sufficient to constrain the DCM motion without considering the stable part to ensure stable walking, for example by constraining the DCM location at the end of a specified time $\xi(T)$ as in [12]. In this case, a terminal condition (captured state) is set at the end of a predefined number of steps, and the desired DCM at the end of the current step is recursively computed.

This is not the only way to keep the DCM from diverging. In fact, a legged robot can instantaneously change its CoP location $u_{0}$ by taking a step [33] in order to limit the DCM motion. This is for example the approach taken in [44]. In [33], it was shown that the $\infty$-step capturability region is only a function of the maximum step length and minimum step time. Therefore, any viable state remains viable as long as the next step timing and location are decided such that the distance between the DCM and next step location does not exceed a certain bound. It is the approach we exploit in this paper.

II-B The DCM offset

We now introduce the DCM offset, which is a convenient change of variable to synthesize controllers that enforce a desired CoM average velocity

[TABLE]

where $u_{T}$ is the next step location and $\xi_{T}$ the DCM at the end of the step (Fig. 1). The LIPM solution can be written in terms of the next footprint location, the step duration and the DCM offset by solving (2b) as a final value problem

[TABLE]

in which $\xi_{cur}$ is the current DCM of the robot. Assume a desired CoM average velocity given by a desired step length $L$, width $W$ and duration $T$, then the resulting desired DCM offset in the sagittal and lateral directions is simply333cf. Appendix A for the derivation details

[TABLE]

where $l_{p}$ is the default step width and $n=1$ when the right foot is stance, and $n=2$ when the left foot is stance. $W$ is the deviation of the step width with respect to the pelvis width. As this value shows how much the robot moves laterally, we will call it the step width (Fig. 1).

II-C Viability bound on the DCM offset

We now express the viability region of the LIPM [2, 51, 53] in terms of the DCM offset. Computing the viability kernel is generally intractable but fortunately, it is possible to characterize these bounds for the LIPM as it was first shown in [33], where it was characterized in terms of the $\infty$-step capturability region (i. e. the set of DCM/ICP states from which the system can come to a captured state by taking an infinite (or less) number of steps). The $\infty$-step capturability bound $d_{\infty}$ is written as [33]

[TABLE]

with maximum step length $L_{max}$ and minimum step duration $T_{min}$. (8) enables the analysis of the viability of the current state of the system but is not necessarily convenient to relate viability to the step location and duration. We now write the viability bounds in terms of the DCM offset. We limit our analysis to the sagittal plane dynamics for forward walking as the analysis of backward walking is similar and the lateral direction analysis is given in Appendix B. The maximum DCM offset $b_{x,max}$ is related to the maximum step length and minimum step duration as

[TABLE]

Interestingly, this maximum offset separates viable and non-viable states: I) if the DCM offset is larger than $b_{x,max}$, all possible choices of step timing and location will lead to its divergence and II) if the DCM offset is smaller than (or equal to) $b_{x,max}$, there exists at least one combination of step timing and location that keeps the DCM from diverging.

Case I

If $b_{x,0}>b_{x,max}$ at the start of a step then

[TABLE]

where $\epsilon>0$. Using (4), (5) and (10), the DCM offset at the end of the step is

[TABLE]

Substituting $u_{x,T}-u_{x,0}=L_{max}$ and $T=T_{min}$, the minimum realizable offset $b_{x,T}$ is therefore

[TABLE]

Comparing (10) and (12), we see that the minimum realizable DCM offset at the end of the step increases by $e^{\omega_{0}T_{min}}$. A sequence of steps will therefore result in a diverging geometric series with common ratio $e^{\omega_{0}T_{min}}$ and all possible choices of step location and timing lead to divergence (i.e. a fall).

Case II

If $b_{x,0}\leq b_{x,max}$ at the start of a step then

[TABLE]

Using (4), (5) and (13), we find

[TABLE]

Selecting the next step position and timing as $u_{x,T}-u_{x,0}=L_{max}$ and $T=T_{min}$, we find

[TABLE]

which shows that for any state satisfying (13), there exists at least one choice of step position and timing that keeps the next DCM offset bounded by $b_{x,max}$. The state is viable.

• Remark

The DCM offset bound (9) is equal to the $\infty$-step capturability bound (8) derived in [33]. However, it affords a different interpretation: the viability kernel defined by the DCM offset bound is the closure of the $\infty$-step capturability set. It contains states that never lead to a fall but that are not $\infty$-step capturable because they cannot lead to a stop even in an infinite number of steps.

III Stepping controller

The stepping controller (Fig. 2) has three main stages:

1. we compute nominal step location and duration, and DCM offset for a desired walking velocity,
2. using these nominal values, we compute optimal desired step duration and location at each control cycle and
3. we adapt the swing foot trajectories used in a whole body controller to generate walking (Sec. IV).

III-A First stage: nominal values for stepping

We find desired set of step length, width and duration that ensure a desired average walking velocity while satisfying the robot and environment constraints. The problem can be formulated in terms of nominal step length $L_{nom}$, width $W_{nom}$, and duration $T_{nom}$ as

[TABLE]

where $v_{x}$ and $v_{y}$ are the desired average walking velocities in sagittal and lateral directions. Step location bounds, $L_{min}$, $L_{max}$, $W_{min}$ and $W_{max}$, are set according to the robot (limited step length and width) and environment (limited area for stepping) limitations. Minimum step timing limits swing foot acceleration and maximum step timing limits slow stepping.

We compute these nominal values to keep them as far as possible from their boundaries

[TABLE]

where we chose

[TABLE]

and we remove the term including $v_{x}$ or $v_{y}$ from these equations ,when they are equal to zero. The resulting desired DCM offsets are then computed using (7).

III-B Second stage: online foot location and timing adaptation

This stage adapts foot step timing and location based on the DCM measurement at each control cycle (typically 1 KHz on a torque controlled robot) to enforce viability constraints. Introducing the following change of variable for step timing

[TABLE]

Equation (6) becomes linear in the decision variables $\tau$, $b$, $u_{T}$

[TABLE]

where $t$ is the time elapsed since the beginning of the actual swing phase. The step adaptation problem is then the solution to the quadratic program

[TABLE]

where $b_{x,nom}$ and $b_{y,nom}$ are the nominal DCM offsets in sagital and lateral direction computed in Sec. III-A. The last inequality in (III-B) is the viability condition implemented as a hard constraint (cf. Appendix B for $b_{y,max,in}$ and $b_{y,max,out}$ definitions). It guarantees that the system remains viable provided that the current state is viable. In practice, we implement this constraint as a soft constraint with a very high weight with respect to the other cost terms to guarantee that the program will always return a solution even if viability cannot be maintained. This ensures that measurement noise or differences between the LIPM and the real robot do not lead to a controller failure and enables to subsequently activate a fall management strategy is necessary. Lexicographic optimization [13] could also be employed in this case but this makes little numerical difference.

III-C Third stage: Swing foot trajectory adaptation

We adapt foot trajectories to follow changing step location and timing. Foot trajectories horizontal to the ground are simply represented with fifth order polynomials so trajectories are continuous in acceleration, which is important for inverse dynamics control. In the vertical direction, the swing foot height increases until the middle of the step and then decreases to land on the ground. Simply using two fifth order polynomials for each part of this trajectory would lead to two important problems:

1. a change in step timing close to mid-time can cause a jump from the first spline to the second one and 2) a change of step timing in the second part generates unavoidable fluctuations in the vertical direction which may cause ground-foot collisions. We use instead a 9th order polynomial for the whole step where we keep the swing foot height at the mid-time of the step as close as possible to the desired step height and enforce a stricly positive foot height that is lower than a maximum height. This results in the following QP

[TABLE]

where $z_{f,d}$ is the vertical component of the swing foot, $k-1$ is the previous sample time and $T$ is the adapted step timing. The coefficients $c_{i}$ of the polynomial are computed at each control cycle and we evaluate the polynomial at the current time to obtain the current desired foot position.

IV Whole body control

We use hierarchical inverse dynamics to control the desired swing foot motion, while enforcing robot constraints, non-slipping contacts and constant CoM height. We specify a hierarchy of desired task space behaviors and constraints expressed as linear equalities and inequalities and the controller computes the resulting optimal joint accelerations, contact forces and actuation torques. We describe in the following our control tasks and constraints, all details of the controller can be found in [22].

IV-A Foot trajectory tracking

The swing foot tracking task ensures tracking of the desired motions computed in Section III-C. The task is written as

[TABLE]

where $\dot{q}$ is the configuration of the robot, $X_{f}$ and $X_{f,d}$ are the actual and reference swing foot positions, and $J_{sw}$ is the swing foot Jacobian. $K_{p}$ and $K_{d}$ are diagonal gain matrices. We do not control swing foot orientation because we will apply approach to a robot with passive ankles without enough DOFs to control the orientation. For the stance foot, the task is to remain on the ground

[TABLE]

where $J_{st}$ is the stance foot Jacobian. This task keeps the stance foot in a stationary contact with the ground surface.

IV-B Center of Mass height tracking

While it is not necessary to control the horizontal motion of the center of mass, we want to keep a desired CoM height with the following task

[TABLE]

where $J_{CoM}$ is the CoM height Jacobian, $z$ is the CoM height and $z_{d}$ is set to the CoM height used in the LIPM ($z_{d}=z_{0}$).

IV-C Posture control

We ensure a straight robot posture with the task

[TABLE]

where $K_{p}$ and $K_{d}$ are non zero diagonal gain matrices for the actuated joints.

IV-D Force regularization

During the double support phase, we exploit contact force redundancy to find forces that distribute the contact forces among the end-effectors in contact by defining the task objective $\lambda=F_{des}$, where $F_{des}$ is the desired contact force. We put this task in the lowest priority (see Table I) such that acts only as a regularizer when relevant.

IV-E Task hierarchy

The hierarchy used for all our experiments is shown in Table I. We give the highest priority to physical consistency and actuation limits. The second rank enforces stance foot contact constraints and CoM height. The swing foot control, which essentially generates the walking motions is in the third rank. The last two ranks are used for the posture control and force regularization.

V LIPM simulation

In this section, we present simulation results using the LIPM simulation, where (1) is integrated and $u_{0}$ is reset by $u_{T}$ at time $T$ computed from (III-B). In our implementation of (III-B), we use a large $\alpha_{3}$ compared to $\alpha_{1}$ and $\alpha_{2}$ to favor solutions close to the desired walking speed. For all reported experiments, we set $\alpha_{1}=1$, $\alpha_{2}=5$, $\alpha_{3}=1000$. Since we optimize the time of the current step at each control cycle, we need to prevent instantaneous stepping (and prevent step timing that are shorter than the current step). Therefore, we apply the adapted gait variables if $T\geq t+T_{gap}$, where $t$ is the current time and $T_{gap}$ is a user defined time gap. Otherwise we use the previous step values. Additionally, towards the end of a step, when the time gap is smaller than the remaining step time, we stop foot location or timing adaptation until the end of the step. We noticed in our experiments that this was sufficient to avoid time adaptation jitter.

The main goal in this section is to show the performance of our gait controller independent of the whole body controller. We study push recovery capabilities and show the importance of step timing adaptation. We then compare our controller with a standard MPC-based walking controller [20].

V-A Simulation results using the LIPM

We simulate the LIPM with our controller using a set desired velocity. Footsteps and swing foot trajectories are computed as described in Section III. During each step, the stance foot is set as the point of contact of the LIPM. At the end of a step where the point of contact of the LIPM is changed, the foot index $n$ is changed. We apply pushes on the robot and compare the recovery capabilities of the controller with and without step timing adjustment. We set the mass to 60 Kg and the CoM height to 80 cm, Table II defines the other variables (the step location limitations are specified with respect to the stance foot) which correspond to the Sarcos humanoid robot used in the next section.

We set a forward desired velocity ($v_{x}=$1\text{,}\mathrm{m}\mathrm{s}^{-1}) and compute the nominal step length and step duration using ([17](#S3.E17)), as well as the nominal DCM offset using ([7](#S2.E7)). After four steps, the robot is pushed at $t=$1.4\text{\,}\mathrm{s} to the right direction with a force $F=$325\text{,}\mathrm{N}, during $\Delta t=$0.1\text{\,}\mathrm{s}. We conduct two simulations to compare the results of fixed and optimized step duration. In one case, we use (6) for the step adjustment, using the current DCM measurement. In a second case, we solve the optimization problem in (III-B) at each control cycle to generate the desired time and location of the next step.

Figure 3 illustrates the resulting trajectories for each case. We observe that without timing adjustment, the robot steps on the borders of the feasible area to recover from the push but because step timing is fixed, the DCM ends up diverging. In the case where timing adjustment is enabled, the algorithm constantly adapts the next footstep and landing time. As a result, the robot steps on the borders of the feasible area very fast to recover from the push. We notice that our approach results in smooth swing foot trajectories (without any discontinuity in position or velocity), despite the constant step location and timing adjustment.

Figure 4 shows the adapted step time at each control cycle. The nominal stepping time for this scenario is $T_{nom}=$0.35\text{,}\mathrm{s}. When the push is exerted at $t=$1.4\text{\,}\mathrm{s}, the optimizer quickly decreases step timing. Interestingly, the optimizer increases step timing after push rejection above the nominal step time ($T=$0.446\text{,}\mathrm{s}). Indeed, at $t=$2.019\text{\,}\mathrm{s} when the push is rejected, the DCM is very close to the stance foot location (Fig. 3, top-right) and the DCM offset far from its nominal value. Hence, the optimizer increases stepping time to bring the robot back to its nominal speed.

V-B *Comparison with a standard MPC walking controller *

We compare the robustness of our controller to the approach proposed in [20], which can be considered, together with variations of this technique [26, 52, 11] as a standard, state of the art, walking pattern generator. This controller automatically generates step locations and CoM trajectory for a desired walking velocity. Step timing, however, remains fixed. In the paper, a horizon of $N=16$ time intervals of length $T=$0.1\text{,}\mathrm{s}$$ is used. We note here that solving the quadratic program for this problem with a 1.6 second horizon is considerably more expensive than our optimization approach. Indeed, considering CoM jerk and foot location as decision variables in 16 time step horizon yields $4\times 16=64$ decision variables at each control cycle without timing adaptation, while our approach only has 6 decision variables.

We used the same parameters for both approaches using a LIPM with point contact and computed the maximum push that each approach can recover from in various directions (Fig. 5). For each simulation, a force during $\Delta t=$0.1\text{,}\mathrm{s} is applied at the start of a step in which the left foot is stance. We use $T_{nom}=$0.5\text{\,}\mathrm{s} (computed from the first stage of our algorithm) for both approaches.

Figure 5 shows that our walking controller, which only adapts the next step location and time, can recover from much more severe pushes compared to the approach with a preview of several steps but without step timing adaptation. For side pushes, the direction of the push affects the maximum value of the push that the robot is able to withstand. Indeed, for certain pushes (outward direction, left in this case) the feasible stepping area is more limited than the other direction (inward direction) due to leg self-collisions. The maximum push in the direction $\theta=$75\text{,}\mathrm{d}\mathrm{e}\mathrm{g}$$ is larger than the other directions because the feasible area for stepping in this direction is larger than the other directions.

We also tested the controller [20] with minimum step timing $T_{min}$ as nominal behavior and, as expected, it leads to the exact same robustness results as our approach. In the case of a very severe push, both algorithms yield stepping on the boundaries of the feasible area. This confirms our claim that it is sufficient to solely consider the next foot step timing and location. Moreover, in our approach step timing is kept as close as possible to the nominal step time and the shortest step time is only selected when necessary, which should lead to more desirable behaviors in general. Indeed, constantly stepping with minimum step time puts an extra strain on the system. Actuators will work at their maximum capacity all the time, augmenting the risk of failure and energy consumption. Furthermore, increasing the frequency of stepping can make the system more vulnerable to complex terrains with surface unevenness. In contrast, our algorithm steps at maximum speed only when absolutely necessary.

VI Humanoid with passive ankles simulation

VI-A Description of the simulations

In this section, we use a simulation of a Sarcos humanoid with passive ankles in the SL simulation environment [47]. Contacts are simulated with a penalty method with linear springs and dampers. We used 18 contact points for each foot. The controller has only access to the resultant wrench, simulating a 6-axis force sensor. All experiments are performed on a 2.7 GHz intel i7 processor with 16Gb of RAM. The quadratic program (III-B) is solved using a slightly adapted version of the QuadProg++ software, which implements an active set method described in [16]. Each leg of the robot has 4 active degrees of freedom with passive ankle joints and prosthetic feet. We simulate the passive ankle joints with stiff springs and dampers. Actuation torques are computed using the controller described in Section IV.

We conduct different simulations with various external disturbances. Since the robot has finite size feet (i.e. the CoP can move inside the foot), we set the current contact point $u_{0}$ in our controller as the current CoP measurement. The constraints and physical properties are the same as for the LIPM experiments, except that the minimum step duration is now set to $T_{min}=$0.3\text{,}\mathrm{s}$$ to account for acceleration limits of the robot. We test two scenarios: push recovery and slippage recovery. In the first scenario, the pelvis is pushed during stepping in different directions. In the second scenario, the stance foot is pushed such that slippage occurs.

VI-B Push recovery

We compare our controller with fixed stepping duration and then with timing adaptation to better understand the influence of step timing adaptation on push recovery capabilities.

Fig. 6 shows a typical side-step adaptation behavior. To recover from the push, the robot automatically starts stepping to the right direction. Once the push is rejected, the robot resumes its forward walking.

Fig. 7 and Fig. 8 show push recovery capabilities without time adaptation. The maximum lateral (inward) disturbance the robot can withstand is $200\text{\,}\mathrm{N}$ for $\Delta t=$0.1\text{,}\mathrm{s}$$. We notice that feet trajectories are adapted to reach the desired landing locations. After the push, the controller sacrifices lateral velocity tracking and adjusts foot positions to recover balance. The adapted feet trajectories remain smooth resulting in smooth control input generated by the whole body controller.

Fig. 9 and Fig. 10 show the push recovery results when step timing adaptation is enabled. In this case, the maximum lateral (inward) disturbance that the robot was able to withstand was a push of $390\text{\,}\mathrm{N}$ for $\Delta t=$0.3\text{,}\mathrm{s}$$, which is nearly six times the impulse that the robot could withstand when timing was not adapted. This is consistent with the results found for the LIP model (Fig. 5 with $\theta$ = $0\text{\,}\mathrm{d}\mathrm{e}\mathrm{g}$). This result further illustrates the importance of timing adaptation for stable walking. Indeed, inspecting (3) we see that the DCM diverges as an exponential of time, therefore taking fast steps (decreasing the step timing) enables an exponential improvement in DCM regulation. This effect is magnified when the robot takes several steps to recover from a disturbance.

We note a slight difference between these two simulation scenarios. Indeed, the vertical lines in the figures show that for the scenario with timing adaptation the push is exerted in the middle of a step ($t=$3.9\text{,}\mathrm{s} in Fig. [9](#S6.F9)), while in the scenario without timing adaptation the disturbance is exerted exactly at the start of a step ($t=$3.7\text{\,}\mathrm{s} in Fig 7). Since the DCM diverges exponentially fast, the later the push during a specified step, the more difficult the recovery (e. g. the DCM diverges more when time evolves from 0.2 s to 0.4 s than 0 s to 0.2 s). As a result, although the experimental condition is more difficult in the case of timing adaptation, the robot recovers from a push nearly six times larger than the case without timing adaptation.

Fig. 9 shows that both step location and timing are adjusted. To recover from the push, the robot takes five steps with minimum step time to the left on the boundaries of the feasible area. The feet trajectories (Fig. 10) are again adapted very smoothly in the case when step timing is adjusted. Furthermore, the step height is also adapted when step duration is changed, consistently with the constraints specified in (21). Trajectory tracking in the vertical direction degrades compared to the case without timing adjustment (Fig. 8), which is due to an increase in the desired swing foot acceleration. However, such trajectory tracking performance in the vertical direction is sufficient for feasible stepping.

Fig. 9 shows that after rejecting the exerted push by stepping to the right, the robot starts (around $t=$6\text{,}\mathrm{s}) to step back in left direction. This behavior can be explained by looking at the angular momentum (see Fig. [11](#S6.F11)). The stance foot constraints are in a high rank and the whole body controller sacrifices joint posture and generates angular momentum to keep the stance foot steady on the ground. After exerting the push, angular momentum is accumulated until the push is fully rejected (around $t=$6\text{\,}\mathrm{s}). After this, the whole body controller reduces the angular momentum to zero and brings back the joints toward the desired upright upper-body posture. This acts as a disturbance to the walking controller and the robot starts stepping in the opposite direction until the angular momentum is zero and the upper-body is upright.

VI-C Slippage recovery

We now investigate the slippage recovery capabilities of the controller. Walking controllers based on CoP modulation tend to be very sensitive to foot slippage. Indeed, when the stance foot slips, the assumption of stationary flat foot on the ground is not valid anymore. Since our controller does not rely on CoP control it can recover from large slippage by adjusting the swing foot landing location and time. During walking, we apply strong forces on the stance foot such that it slips.

We compare performance with and without timing adaptation. For the case without timing adaptation, Fig. 12 shows a typical forward walking simulation. Slippage of the left foot can be observed in the actual foot trajectory in the lateral direction (Fig. 13) which causes a faster DCM divergence in the right direction. Hence, the right foot steps further in the right direction to recover from the disturbance. Figure 14 shows the lateral trajectories when step timing adjustment is enabled. Again, we can see a degradation in the foot trajectory tracking in lateral direction (Fig. 15). The controller exploits a combination of step timing and location adjustment to recover from this strong disturbance. Some high frequency oscillations can be seen in the DCM trajectory and the estimated next footprint (Fig. 14 ), which is due to the large disturbance on the stance foot. This disturbance causes high frequency oscillations of the passive elements in the ankles. Nevertheless, the robot is able to withstand an impulse on the foot which is more than three times larger than the impulse withstood when step timing is not adjusted. This again illustrates the importance of timing adjustment during walking.

VII Discussion

VII-A Features of the approach

• •

Generality Our approach can generate robust gaits without modulating the foot CoP. This suggests that our approach can be used for biped robots with active ankles, passive ankle, or point contact foot. We successfully used the controller to control an underactuated biped robot with passive ankle joints and prosthetic feet.

• •

Computational efficiency The size of our optimization problem is drastically smaller than typical MPC approaches using several preview steps [26, 52, 11, 20], at least an order of magnitude smaller in the number of decision variables. Viability is ensured by optimizing the DCM offset without the need to integrate the motion forward over several time steps [54].

• •

Controller receding horizon length Our paper provides an answer to an open question for MPC-based controllers: how far ahead in time shall our controller optimize motion for?[56]. It is only sufficient to look at the next step to recover walking from any viable state. Interestingly, several reported experiments on human walking seem to support this argument, even in the presence of obstacles [43, 42]. However, if the terrain is very difficult or constrained (e.g. walking on stepping stones), humans seem to look two steps ahead [42].

• •

Step timing vs step location adjustment To highlight the difference between step location and timing adjustment, let’s assume that the swing foot can move in all directions without any kinematic constraints and that the only constraint is a limited foot acceleration. Then, the maximum distance the swing foot can travel in a given duration is achieved using a bang-bang controller (maximum acceleration and deceleration) with equal acceleration and deceleration periods if kinematic constraints are not considered. Specifying step period $T$ and maximum acceleration/deceleration value $a_{max}$, the swing foot travels $u_{T}=a_{max}\;\frac{T^{2}}{4}$ during this period. Assuming $u_{0}=0$ for the current step in the DCM dynamics, at time $T$ the DCM is $\xi_{T}=\xi_{0}e^{\omega_{0}T}$. The DCM diverges as an exponential of time while the next step location can only be adapted with a second order time dependency.

• •

Robustness against various disturbances The measured CoP and estimated DCM are employed to adapt the landing location and time of the swing foot but the controller does not try to control the CoP. This feature enables robust walking under strong pushes and stance foot slippage. This feature might also enable the robot to walk over rough terrains, but this is left as a future work. Furthermore, our experiments showed that timing adjustment significantly increased walking robustness.

• •

Comparison to existing approaches Step timing adaptation can serve different purposes, i. e. decreasing the ZMP fluctuation in case of immediate change of step location [38], walking on uneven terrain [40, 41, 31], disturbance recovery [1, 34, 29, 18], respecting acceleration constraint of the swing foot moving on a specified path [6], or collision avoidance for walking in crowds [3]. Despite a variety of goals, the employed approaches are either based on analytical methods and heuristics [41, 38, 40, 36, 8, 18, 31] or optimization [1, 34, 6, 35, 3]. Optimization provides a principled way of taking into account all constraints while optimizing a desired performance cost. However, non-convexity of the system dynamics with respect to step timing can significantly increase computation time and is prone to getting stuck in undesired local minima. Our approach with at least one order of magnitude less decision variables than existing approaches [1, 34, 3, 6, 35] and a convex formulation circumvents both concerns and provides a reliable tool for realtime applications.

• •

Viability In [48], an approach similar to ours [29] is used for guaranteeing viability of the gait by enforcing a desired offset between the capture point and landing location of the swing foot. However, the approach to realize this offset is based on modulating the CoP of the current step, while the next step location and timing are fixed. [46] used a combination of ZMP modulation and step location adjustment to keep the capture point inside the support polygon at the end of the current step (1-step capturability). As we showed in this paper, endowing step location adaptation with timing adjustment and using exact viability constraint in the controller structure can ensure that any initially viable state remains viable.

VII-B Limitations

• •

LIPM assumption: the viability guarantees of our controller are valid under the assumption of a fixed CoM height and constant angular momentum at the CoM. While these assumptions restrict walking on even ground, since we do not rely on CoP modulation we anticipate that the controller will also be robust to mild rough terrain provided that the low level controller manage the uncertainty in landing. An interesting direction of research would be to use the concept of 3D-DCM and virtual repellent point (VRP) [12] instead to enable our approach to handle 3D maneuvers.

• •

Timing constraint Constraints on step locations are explicit and depend on the kinematic and environment limitations. However, constraints on step timing are a function of maximum acceleration of the swing foot as well as the distance between the current state of the swing foot and the landing location. As a result, considering a constraint on step timing for the worst case decreases the capabilities of the controller. To circumvent this, we could use more complex, yet linear, models which take into account the swing foot dynamics [49]. Then, we could directly constrain swing foot acceleration instead of step timing. A simpler alternative is to use an empirical linear inequality constraint on the next step location and minimum time as a function of maximum foot speed and remaining stepping time [20].

• •

Walking on stepping stones or in scattered environments While we do not need to consider a multi-step horizon to guarantee viability in unconstrained environments, when the feasible foot locations are very limited, such as waking over stepping stones or in the presence of obstacles, we would need to consider multiple steps in advance to steer the DCM motion in the direction of safe contact points. Our approach can be readily extended to consider this issue, for example one could adapt step timing only in the first step within a desired horizon and optimize for the subsequent step locations with a fixed nominal step timing, therefore retaining a QP.

• •

Real robot experiments Previous works [14, 45] have shown that our modeling assumptions can be successfully used on a real robot. While this suggests that our approach would work as well, real robot experiments, especially for robots with passive ankles or point feet would be most valuable. At the time when the research was conducted, we did not have access to such robots.

VIII Conclusion

We proposed a walking controller that adapts both step location and timing in real time to generate robust gaits and we showed that for the LIPM, optimizing the next footstep timing and location was sufficient to ensure that viable states remain viable. Comparison with a standard MPC-based walking controller emphasized the importance of step timing adaptation and demonstrated that our approach can be significantly more robust (the algorithm could in our simulation tolerate up to about 5 times larger impulsive pushes) than methods that do not adjust timing, even when those controllers optimize over several preview steps. Simulations in push and slippage recovery scenarios on a humanoid with passive ankles showed the robustness of the controller even without control authority in the ankles and the foot CoP.

Acknowledgment

This research was supported by the European Unions Horizon 2020 research and innovation program (grant agreement 780684 and European Research Councils grant 637935) and in part by the MPI-ETH center for learning systems. We would like to thank the reviewers for their thoughtful comments on a previous version of the manuscript.

Appendix A: Derivation of (7)

Using (4) and (5), the LIPM equation in terms of the DCM offset and the next step location is

[TABLE]

For a constant walking velocity, the desired DCM offset in the sagittal direction, $b_{x}$, is considered to be constant over several steps, therefore we have

[TABLE]

since $L=u_{T,x}-u_{0,x}$ over two consecutive steps, we have

[TABLE]

Note that this DCM offset causes a limit cycle for the robot nonlinear (switching) dynamics, in the sense that keeping $L$ and $T$ constant over several steps leads to a steady state forward walking velocity. For a desired sideward walking, the distance between the feet is equal to $l_{p}+W$ or $l_{p}-W$ (Fig. 1). Using (26), the equations for the right and left foot DCM offset are

[TABLE]

Solving (Appendix A: Derivation of (7)) for $b_{y,r}$ and $b_{y,l}$ yields

[TABLE]

We use the index $n$ to distinguish the right ($n=1$) and left ($n=2)$ stance foot and write the DCM offset in the latteral direction as

[TABLE]

Appendix B: Viability in the lateral direction

In the lateral direction, the feasible area for stepping is not symmetric with respect to the stance foot due to self collision. The maximum allowable DCM offsets for outward and inward directions are then different. Here we call outward direction the one where the legs can experience a self collision, and inward for the opposite direction. The maximum allowable DCM offsets in these directions are

[TABLE]

To show that these values exactly split the state space into the viable and non-viable parts, we again consider the situations where the DCM offsets are less or more than these values for both outward and inward directions. We conduct this analysis for the case where the right foot is stance (the same results directly follow for the other cases). Considering a DCM offset more (less) than the value in (32a) for the outward direction at the start of a step

[TABLE]

the minimum feasible DCM offset at the end of this step ($b_{y,T}$) is obtained taking a step with the minimum step width (for avoiding self collision) at the minimum step time using (26)

[TABLE]

Using this value as the DCM offset at the start of next step, and taking a step with the maximum step width at the minimum step time yield the minimum feasible DCM offset of the next step

[TABLE]

which simplifies into

[TABLE]

Comparing (33) with (36) reveals that starting from a DCM offset more than (32a) (positive $\epsilon$ in (33)) and after taking two steps on the boundaries of the feasible area at the minimum time, the DCM offset increases by a factor $e^{2\omega_{0}T_{min}}$ times $\epsilon$. As a result, any DCM offset more than (32a) for causes divergence. However, starting from a DCM offset less than the value in (32a) (negative $\epsilon$ in (33)), there exists one evolution (stepping on the boundaries of the feasible area at the minimum time) that keeps the DCM from diverging. Hence, (32a) for the outward direction exactly splits the state space into viable and non-viable parts.