Static Stability of Robotic Fabric Strip Folding

TL;DR

This paper investigates the static stability of fabric strip folding by robots, identifying instabilities that affect manipulation accuracy, and evaluates different folding methods considering these stability issues.

Contribution

It introduces the concept of static instability in robotic fabric folding, verified through simulation and experiments, and assesses existing folding methods under these stability constraints.

Findings

Static instability occurs during fabric folding.

One folding path is suitable for fabrics with internal friction.

A dynamic folding path exists for ideal elastic materials.

Abstract

Planning accurate manipulation for deformable objects requires prediction of their state. The prediction is often complicated by a loss of stability that may result in collapse of the deformable object. In this work, stability of a fabric strip folding performed by a robot is studied. We show that there is a static instability in the folding process. This instability is detected in a physics-based simulation and the position of the instability is verified experimentally by real robotic manipulation. Three state-of-the-art methods for folding are assessed in the presence of static instability. It is shown that one of the existing folding paths is suitable for folding of materials with internal friction such as fabrics. Another folding path that utilizes dynamic motion exists for ideal elastic materials without internal friction. Our results show that instability needs to be considered in planning to obtain accurate manipulation of deformable objects.

Equations

$$\mathbf{E}=\frac{1}{2}\left({\mathbf{F}}^{\top }\mathbf{F}-\mathbf{I}\right),\mathbf{F}=\mathbf{I}+\nabla \mathbf{u},$$

$$\mathbf{S}=\det \mathbf{F}{\mathbf{F}}^{-1}\left(\mathbf{C}:\mathbf{E}\right){\mathbf{F}}^{-\top },$$

$$\mathbf{\sigma }$$

$$\mathbf{D}$$

$${\eta }_m=\left(1-{\nu }^2\right)\frac{\rho }{hE},{\eta }_b=12\left(1-{\nu }^2\right)\frac{\rho }{h^{3}E}\text{.}$$

$$\nu =0,{\eta }_m=10^{-3}\text{ m−2s2}\text{.}$$

$$\rho \frac{{\partial }^2\mathbf{u}}{\partial t^2}=\nabla \cdot {\left(\mathbf{F}\mathbf{S}\right)}^{\top }+\rho \mathbf{g},$$

$$0=\nabla \cdot {\left(\mathbf{F}\mathbf{S}\right)}^{\top }+\rho \mathbf{g}.$$

$$\mathbf{r}(\mathbf{u},\lambda )=0,$$

$$\mathbf{K}(\mathbf{u},\lambda )\Delta \mathbf{u}=-\mathbf{r}(\mathbf{u},\lambda ),$$

Full text

I INTRODUCTION

Planning robotic manipulation of deformable objects remains a challenging research area. The examples of deformable objects include flexible cables, ropes, elastic or paper sheets, or fabrics. These objects have a large or infinite number of degrees of freedom [1]. This fact complicates the prediction of the object state, for which a complex dynamic model is needed. Compared to rigid bodies, deformable objects are more difficult to immobilize by conventional grippers. Due to the large number of degrees of freedom, some of them often remain free, and the system may become autonomous which results in an uncontrollable motion or collapse. Special treatment needs to be considered for planning in the state space where the system becomes autonomous.

One example of deformable object manipulation task is garment folding. In a folding process, a robot picks the garment at one side and, by following the folding path, brings the garment from its flat initial configuration to the expected folded shape (Fig. 1). In recent years, researchers have focused on the planning of a folding path which provides an accurately folded garment [2, 3]. Good folding accuracy has been achieved by using a simulation of the garment and by analysing its shape when performing the folding. However, the autonomous behavior of the garment in the folding process and its effect on the folding accuracy have not been analyzed before.

This paper provides an analysis of such an autonomous behavior in the single layer fabric strip folding. It is shown that there is a static instability in the folding process. This instability results in a dynamic motion which is not controllable by a robot and corresponds to the autonomous behavior of the model. The static instability is firstly analyzed for a simple motion in a physics-based simulation. The states of the strip where instability occurs in the simulation are compared to real fabric strip measurements. Then folding paths proposed in literature are analyzed in simulation and implications of static instability for the folding planning are discussed.

The contributions of the paper are:

• •

We show that there is a static instability in the fabric strip folding.

• •

We demonstrate how this instability can be detected using a physics-based simulation and compare it to physical measurements.

• •

We analyze how the existing methods deal with the presence of the instability and discuss implications of our observations on the folding planning.

• •

We show that internal friction is required to fold an elastic strip statically.

II STATE OF THE ART

The study of the garment behaviour is necessary for predicting the garment state accurately. The analysis of the behaviour was performed for various tasks including garment drying [4], flattening and ironing [5, 6, 7, 8], or folding [9]. A garment manipulation is analysed in our study.

A garment manipulation pipeline can be considered to consist of four tasks: (a) isolating, (b) unfolding and classifying, (c) folding, and (d) stacking of the folded piece of garment [10]. The goal of the isolating task is to pick a single piece of garment from a heap of garments. The main parts of the task are the detection of a point to grasp and the grasping itself. Several strategies for grasping were compared in [11]. Furthermore, specialized grippers have been proposed for handling garments in [10, 12, 13]. Unfolding and classification task [14, 15, 16] ensures that the garment lies flattened on a folding surface prior to folding. The stacking task cleans the folding surface after the folding is done. All tasks were successfully performed for a towel in [17]. Performing the pipeline for a wider range of garment types was proposed in [18, 19].

In the folding task, the garment lies flattened on a surface and the garment’s type is known. The folding can be decomposed to several independent folds based on the garment type as suggested in [9]. In [9], the robot performed folding in an open-loop manner. A more robust version was proposed in [20] where the garment pose was detected after each fold. Both works [9, 20] parametrized an individual fold by grasping points and a folding line. The folding line divides the garment into two parts such that at the end of the folding one part lies on the other part (Fig. 1).

In each fold, a robot is following a folding path. Several methods for designing the folding path have been proposed. The methods could be divided into several categories according to the complexity of the path computation: (i) the geometrical methods (Sec. II-A) design a path which depends on the garment dimensions only; (ii) the simulation based methods perform an underlying static (Sec. II-B) or dynamic (Sec. II-C) simulation to design a path; and (iii) the learning based methods (Sec. II-D) learn a feedback control policy from observations.

In this work, we analyse how the geometrical and static simulation based paths deal with the instability in the folding process. Other state-of-the-art methods require specialised hardware to fold the garment in a closed-loop manner or their underlying model is not comparable to our finite element simulation. However, we list these methods to make the overview of the folding path generation complete.

II-A Geometrical Paths

Two folding paths which depend on the garment dimensions only were proposed in literature: the triangular [21] and circular [22]. The triangular path was designed based on assumptions that the garment is infinitely flexible and the friction between the garment and the desk is infinite. The circular path assumed a rigid material with flexibility in folding line only. Both paths are visualised in Fig. 2. In this paper we analyse how both paths deal with the static instability in a fabric folding.

II-B Static Simulation Based Path

A physics-based simulation was used in [23] to simulate a fabric strip according to Euler-Bernoulli beam theory. A fabric material was parametrized by a single parameter weight-to-stiffness-ratio. The folding path was derived based on the static equilibrium of forces. The method was extended to single arm rectangular fabric sheets folding in [3]. A Kirchoff-Love shell theory was used for model and the material was parametrized by three parameters weight-to-membrane-stiffness-ratio, weight-to-bending-stiffness-ratio, and Poisson’s ratio. In [24] it was shown that weight-to-bending-stiffness-ratio is sufficient to describe the fabric and the method for estimating this parameter was proposed. Path designed according to this method is visualised in Fig. 2. As its shape resembles ‘R’ letter it is called R-path, hereinafter. In this work, we model a fabric strip as a 3D elastic solid instead of using beam [23] or shell theory [3]. The parameters of the state-of-the-art method [3] are related to our model. It allows us to analyze this folding path in the presence of static instability.

II-C Dynamic Simulation Based Path

A specialised high-speed robot hands were used in a series of works [25, 26, 27] to perform a dynamic-based folding of a towel. The hands swipe the towel in air according to the motion planned on a simulated model. The finger of the hands catch the swiping end of the towel based on the camera feedback. A specialised hardware is required to perform this dynamic folding.

Another simulation based method was proposed in [2]. The authors used an animation software Autodesk Maya to evaluate a quality of the fold for a given garment and a folding path. The folding path was then perturbed until the quality was sufficient. The model used in [2] consists of $n$-particles connected by springs. Spring network structure and the stiffness of individual springs are used to simulates various garment properties. Such models are computationally effective but tuning the parameters to get real garment behaviour is difficult. In [2] author tuned one parameter but the resulting accuracy of the robotic folding was not analysed.

II-D Learning Based Approaches

The more recent approaches tried to learn a policy for the folding. In work [28] the authors proposed a feedback control policy based on the features extracted from an RGBD data. They proposed a policy which iteratively minimise distance between the current and goal state in a future space. One of the tested task was a towel folding in which the authors tested a single piece of towel. Another work [29] learns a convolutional neural network feedback based policy based on RGB image in simulation. The policy learned in a simulation was then directly evaluated on a single piece of rectangular garment folding in reality. The learning from demonstration was used in [30]. The work was tested on a single rectangular garment. Deep learning was used in [31] and tested on a single piece of rectangular garment too. A t-shirt was folded in work [32]. Authors combined learning from demonstration with dimensionality reduction to learn a folding task parametrized by dynamic movement primitives. A total number of 540 real robot executions were used for training. All presented learning based approaches solve a task which is more general than a simple folding path design, but they do not consider the accuracy. As long as the folding motion imitated the expected folding motion it is considered as success regardless of accuracy of the final garment state.

III FABRIC MODEL

This section describes our model of a fabric strip and introduces the concept of continuation [33] which is common in structural mechanics analysis. We use continuation to analyse the static stability of the strip folding in the subsequent sections.

III-A Model Geometry

The strip geometry in reference configuration is represented by a box with dimensions $l\times h\times b$. The whole edge with dimension $b$ is grasped and manipulated during the folding and the fabric material is assumed to be homogeneous and isotropic. Consequently, the model can be represented by 2D geometry as shown in Fig. 3. We refer our model to a Cartesian system placed in the middle of the strip (Fig. 3). When visualising the model state, only the mid-surface (i.e. a cut $z=0$ in a reference configuration) is plotted.

III-B Boundary Conditions

During the folding, three boundary conditions are considered [3]: (i) frictionless contact between the supporting surface and the fabric, (ii) a holding point condition which prevents the strip from slipping, and (iii) a grasping point condition which simulates the gripper grasping. All these boundary conditions are visualised in Fig. 3.

III-C Constitutive Relation

The fabric strip model is represented by a 3D elastic solid. A stress-strain relation needs to be specified to describe an elastic solid. This constitutive relation characterizes the behaviour of a material.

As a strain measure, we use the Green-Lagrange strain [33] defined as:

[TABLE]

where $\bm{F}$ denotes a deformation gradient, $\bm{I}$ stands for an identity tensor and $\bm{u}$ represents the displacement of the solid from its reference configuration. The Second Piola-Kirchhoff stress tensor $\bm{S}$ [33] is usually used in the presence of large deformations. It is computed as:

[TABLE]

where $\bm{C}$ is the elastic tensor and the symbol ‘$:$’ stands for the double-dot tensor product.

An isotropic homogeneous material is assumed. In this case, the elastic tensor depends on Young’s modulus $E$ and Poisson’s ratio $\nu$ only. Because of the tensor symmetry we can represent the double-dot product $\bm{C}:\bm{E}$ completely in a Voigt notation [33]:

[TABLE]

where $\bm{\varepsilon}$ is the Voigt form of the strain tensor $\bm{E}$ and $\bm{\sigma}$ is the result of the double-dot product in the Voigt form.

III-D Relation to weight-to-stiffness-ratios

A Poisson’s ratio $\nu$, weight-to-membrane-stiffness-ratio $\eta_{m}$, and weight-to-bending-stiffness-ratio $\eta_{b}$ were used in [24] to describe the strip model. These parameters model the elastic membrane stiffness independently on the bending stiffness. The weight-to-stiffness-ratios are computed as:

[TABLE]

We use these equations to map the weight-to-stiffness-ratios into the model described in Sec. III-C. Note that by using this mapping the parameter $h$ no longer represents the physical thickness of the strip. Furthermore, $\nu$ and $\eta_{m}$ do not have significant impact on the folding path because the fabric is usually not stretchable significantly [24]. In our experiments, we fixed these parameters to values:

[TABLE]

III-E Equilibrium Equation

The equilibrium equation is given by Newton’s second law:

[TABLE]

where $\rho$ is material density, $t$ represents time, and $\bm{g}$ is the gravitational acceleration vector. In case of a static equilibrium problem, the inertial term given by the left side of Eq. (7) can be neglected:

[TABLE]

Eq. (7) or (8) together with boundary conditions forms a strong form [33] of the problem. In finite element modeling, the strong form is transformed into the weak form [33] which is then expressed by the nonlinear residual equation

[TABLE]

where $\bm{r}$ is a residual force vector and $\lambda$ is a single control parameter, e.g. a pseudo time or a gripper force holding the strip. In a static case, the residual force represents the balance of internal and external forces while in a dynamic case there is an additional contribution from inertial terms. For a fixed control parameter $\lambda$, Eq. (9) can be solved iteratively using Newton-Raphson method [33]

[TABLE]

where $\bm{K}(\bm{u},\lambda)$ is called stiffness matrix for a static problem. Hereinafter, the static problem is assumed unless explicitly stated otherwise.

III-F Continuation

Changing parameter $\lambda$ incrementally and seeking for a state in equilibrium for each value of $\lambda$ is called continuation. The state in equilibrium satisfies Eq. (9) and the sequence of equilibrium states for a varying value of control parameter is called equilibrium path.

Two phenomena are observed during continuation in structural mechanics: snap-through (limit point) and buckling (bifurcation). In a snap-through case, the increasing value of control parameter at some point $\lambda_{c}$ causes the system to change its configuration significantly, i.e. there is a ‘jump’ in a state vector $\bm{u}$. The buckling means that there is a point $\lambda_{c}$ at which two equilibrium paths intersect. The real system chooses which path is followed. The simulated ideal system is not able to decide the path solely from Eq. (10). Point $\lambda_{c}$ is called a critical point in both cases.

At the critical point the stiffness matrix becomes singular. The common indicator used to detect the singularity is the value of the smallest eigenvalue of the stiffness matrix. At the critical point the smallest eigenvalue is zero. The structural mechanics interpretation of the critical point is that there exists a non zero displacement $\Delta\bm{u}$ which requires no additional force. The direction of this displacement is given by the eigenvector associated with the zero eigenvalue. Consequently, the system is not statically stable.

IV CONTINUATION IN X-AXIS

This section studies the fabric strip motion induced by varying gripper position in x-axis. Therefore, in this case the parameter $\lambda$ represents the gripper position in x-axis. The static stability is analysed according to Section III-F. The start state of the strip and the studied motion are visualised in Fig. 4(a). The value of the control parameter is increased starting from value $\lambda_{0}$. At the critical point $\lambda_{c}$, the solution of Eq. (10) can not be found due to the singular stiffness matrix. From this point onwards it is not possible to find consequent strip states by static analysis because the strip starts to move dynamically. However, the strip states can be simulated using a dynamic solver. After the strip touches the ground, the problem becomes statically stable and we can switch back to the static continuation. The states found by this approach are visualised in Fig. 5. Another visualisation of the phenomena plots the control parameter/deflection curve as shown in Fig. 6.

We observed the static stability loss phenomena while manipulating real fabric strip as well. To compare modelled and real observations we evaluate a value of $\lambda_{c}$ for several gripper heights as shown in Fig. 7. The modelled and real values of $\lambda_{c}$ are shown in Fig. 8.

The real measurement was performed for several fabric strips with varying materials. It can be seen in Fig. 8 that the modelled values of critical points underestimate the real values. Moreover, the difference between these values varies for different types of the fabric strip. This observation can be attributed to properties which were omitted in the modelling process such as internal friction between individual fabric threads and nonlinear stiffness. The simplification makes computation of the model state tractable but the modelled and real states differ slightly as the model reaches the singularity.

On the other hand, the fact that the model always underestimates the value of critical point can be used for planning. If the strip following a given path is stable in the model then it will be stable also in reality regardless of the value of omitted parameters. This observation will be used in the assessment of state-of-the-art folding paths in the next section.

V ASSESSMENT OF FOLDING PATHS

We next study folding paths proposed in literature using the static analysis. In this section the control parameter $\lambda$ represents a pseudo time. The gripper position is given by a folding path and depends on the control parameter.

V-A R-path

Consider a continuation of the control parameter for a folding path called R-path [3]. The continuation is stopped when a critical point is reached as shown in Fig. 9(a). At this point, the dynamical solver can be used to trace the state evolution. We observed that there are two different directions which the strip can dynamically follow as shown in Fig. 10(a) by red and blue color. In simulation we used a disturbance to select one of the directions.

However, in Section IV we observed that in reality the strip remains stable longer than predicted by the model. This can be attributed to internal friction. We approximate it by a force which pulls the current state of the model towards the previous state. The pulling force was a few orders of magnitude smaller than the gripper force. With this model we are able to follow the whole R-path as shown in Fig. 10(b). The control parameter/deflection curve is shown in Fig. 11.

The added internal friction stabilizes the model and the folding path could be followed. With the internal friction, the uncontrolled dynamical motion is avoided and the states of the strip correspond to the planned states. Consequently, the position of point where the strip touches itself for the first time equals to its expected position. This was experimentally confirmed in [23, 34, 3].

V-B Triangular and Circular Path

Triangular path was designed for an infinitely flexible material (i.e. $\eta_{b}=\infty$). If the path is used for the material with finite flexibility, it reaches the critical point as shown in Fig. 9(b). Then it falls down dynamically similarly to Fig. 5. However, in an ideal model the position at which the strip touches itself does not correspond to the planned position because the flexibility is different. The $x$-coordinate of the first touch position is smaller than planned. For real fabric material, the position at which the strip falls down is hard to predict according to our measurements shown in Section IV. For one particular material, the position at which the strip touches itself would be correct.

For a circular path, the situation is similar to the triangular path as shown in Fig. 9(c). The strip will fall down dynamically. However, the $x$-coordinate of the first touch position is larger than planned. For real fabric material the $x$-coordinate of this position is even larger according to our observations from Section IV.

V-C Discussion

For triangular and circular paths, snap-through behaviour occurs after the critical point is reached. The critical point is underestimated in our model, i.e. the snap-through behaviour occurs later than predicted. Thus, outcome of both paths is hardly predictable by a model and requires dynamical modeling.

On the other hand, the R-path avoids the snap-thorugh behaviour if there is internal friction in the system. This friction corresponds to the observation that the model underestimates the value of critical point for fabric material. The following of the R-path is statically stable for materials with internal friction.

The R-path has an advantage compared to the other folding paths: At the beginning, it rise up the strip until the part of the strip which lies on the ground is same as in the folded state as shown in Fig. 9. The rest of the path does not change the amount of the strip which lies on the ground. The path does not rely on the snap-through outcome and its only assumption is that the strip remains stable.

The existence of the force which prevents the uncontrolled motion of the strip is not limited to fabric material only. The R-path was successfully used to accurately fold rubber strips in [23]. The origin of the force in rubber material is different than thread friction and may correspond e.g. to memory effect of the material. However, there are materials without this force, for example thin metal. Using R-path for folding this material would not be feasible because any disturbance in a gripper position would cause the dynamic motion. For this material it may be better to plan a path which utilizes the snap-through behaviour. Predicting the outcome of the dynamic motion would be possible for material without the internal friction.

VI CONCLUSION

The paper presented a static instability analysis for a single layer strip folding. It was shown that the instability exists in the folding process and that it results in the dynamic motion of the strip. The presented comparison of the real fabric strip and the simulated model suggested that there is an internal friction in the real system. This observation was used to analyse the state-of-the-art folding paths in a presence of the static instability. It was shown that R-path folds a strip accurately for materials with internal friction and that another path may be more accurate for other materials.

This paper studied only strip folding. However, the same method can be used to predict static instability for folding garments of more complicated shapes. For example, it can be used to predict a collapse of a single arm rectangular garment folding which was shown in [3]. The detected instability can be then considered in planning to avoid the region of instability or to immobilize the garment with another gripper. The eigenanalysis of the stiffness matrix can likely be used to determine the position of the additional gripper.

The concept of static instability might also be used in areas not related to folding. For example in grasping of a deformable object the instability would suggest that the object is not held properly. Again the eigenanalysis can reveal the part of object which is not stable.