Implicit Contact-Rich Manipulation Planning for a Manipulator with Insufficient Payload

TL;DR

This paper introduces a probabilistic planning method enabling a mobile manipulator to manipulate objects beyond its payload capacity by leveraging environmental support without explicit contact mode analysis.

Contribution

It presents a novel single-shot probabilistic roadmap approach that automatically plans contact-rich manipulation motions considering environment support and force constraints.

Findings

Successfully manipulates heavy objects using environmental support.

Avoids explicit contact mode analysis and transition graphs.

Validated through simulations and real-world experiments.

Abstract

This paper uses a mobile manipulator with a collaborative robotic arm to manipulate objects beyond the robot's maximum payload. It proposes a single-shot probabilistic roadmap-based method to plan and optimize manipulation motion with environment support. The method uses an expanded object mesh model to examine contact and randomly explores object motion while keeping contact and securing affordable grasping force. It generates robotic motion trajectories after obtaining object motion using an optimization-based algorithm. With the proposed method's help, we can plan contact-rich manipulation without particularly analyzing an object's contact modes and their transitions. The planner and optimizer determine them automatically. We conducted experiments and analyses using simulations and real-world executions to examine the method's performance. It can successfully find manipulation motion that met contact, force, and kinematic constraints, thus allowing a mobile manipulator to move heavy objects while leveraging supporting forces from environmental obstacles. The mehtod does not need to explicitly analyze contact states and build contact transition graphs, thus providing a new view for robotic grasp-less manipulation, non-prehensile manipulation, manipulation with contact, etc.

Tables

Table 1. TABLE I: Performance of object trajectory planning with different thickness of M e subscript M e \textnormal{M}_{\textnormal{e}}

	Task 1				Task 2
Thickness (cm)	0.5	1.5	3	2.60$\sim$3.35 (average 3.00)	0.5	1.5	3	1.91$\sim$2.38 (average 2.09)
Planning time (s)	-	6.34	5.82	6.29	8.39	5.81	2.40	2.92
Success rate (%)	0	80	90	90	20	40	60	50

Equations

$$\delta \theta =l_{arm}{\tan }^{-1}(\delta d).$$

$$\underset{{\mathbf{F}}_{\mathbf{h}}}{\min }$$

$$\sum {\mathbf{F}}_{\mathbf{s}}+\mathbf{G}=0$$

$$\sum {\mathbf{r}}_{\mathbf{s}}\times {\mathbf{F}}_{\mathbf{s}}+{\mathbf{r}}_{\mathbf{h}}\times {\mathbf{F}}_{\mathbf{h}}+{\mathbf{r}}_{\mathbf{g}}\times \mathbf{G}=0$$

$${\mathbf{F}}_{\mathbf{s}}\in \mathbf{F}{\mathbf{C}}_{\mathbf{s}},{{\mathbf{F}}_{\mathbf{h}}}^T{\mathbf{F}}_{\mathbf{h}}⩽F_{max}$$

$$\underset{\mathbf{q}(t)}{\min }$$

$$\{\begin{array}{l}\mathbf{q}(t+1)=[q(i+1{)}^0,q(i+1{)}^1,...,q(i+1{)}^d{]}^T\\ \mathbf{q}(t+1{)}^k\in [{\theta }_k^{-},{\theta }_k^{+}],k=JointID\end{array}$$

$$\{\begin{array}{l}{\mathbf{p}}_{\mathbf{r}}(t+1),{\mathbf{R}}_{\mathbf{r}}(t+1)=FK(\mathbf{q}(t+1))\\ ||{\mathbf{p}}_{\mathbf{r}}(t+1)-{\mathbf{p}}_{\mathbf{h}}(t+1)|{|}_2⩽{\epsilon }_p\\ \angle ({\mathbf{R}}_{\mathbf{r}}(t+1),{\mathbf{R}}_{\mathbf{g}}(t+1))⩽{\epsilon }_R\end{array}$$

$$\{\begin{array}{l}\mathbf{\tau }(t+1)=\mathbf{J}(\mathbf{q}(t+1){)}^T\mathbf{F}(t+1)\\ {\tau }^k(t+1)\in [{\tau }_k^{-},{\tau }_k^{+}],k=JointID\end{array}$$

$$\{\begin{array}{l}||q^0(t+1)-q^0(t)||⩽d_x\\ ||q^1(t+1)-q^1(t)||=0\\ ||q^2(t+1)-q^2(t)||=0\end{array}$$

Full text

Design/methodology/approach: The paper proposes a single-short probabilistic roadmap-based method to plan and optimize manipulation motion with environment support. The method uses an expanded object mesh model to examine contact and randomly explores object motion while keeping contact and securing affordable grasping force. It generates robotic motion trajectories after obtaining object motion using an optimization-based algorithm. With the proposed method’s help, we can plan contact-rich manipulation without particularly analyzing an object’s contact modes and their transitions. The planner and optimizer determine them automatically.

Findings: We conducted experiments and analyses using simulations and real-world executions to examine the method’s performance. The method successfully found manipulation motion that met contact, force, and kinematic constraints. It allowed a mobile manipulator to move heavy objects while leveraging supporting forces from environmental obstacles.

What is original/value of paper: The paper presents an automatic approach for solving contact-rich heavy object manipulation problems. Unlike previous methods, the new approach does not need to explicitly analyze contact states and build contact transition graphs, thus providing a new view for robotic grasp-less manipulation, non-prehensile manipulation, manipulation with contact, etc.

I Introduction

This paper studies using a mobile manipulator to manipulate objects beyond the robot’s maximum payload. Fig. 1 exemplifies how a human manipulates a large, heavy object beyond the person’s maximally affordable weight. The human took advantage of environmental support to share partial object weight and thus unloaded the object from the table onto the ground. This paper develops a planning and optimization approach for a mobile manipulator to handle objects like the human in the figure. The approach uses an expanded object mesh model to examine contact and randomly explore object motion while keeping contact and securing affordable grasping force. It generates robotic motion trajectories after obtaining object motion using an optimization-based algorithm. With the proposed methods’ help, we can plan contact-rich manipulation without particularly analyzing an object’s contact modes and their transitions, and thus allow a mobile manipulator to move heavy objects while leveraging supporting forces from environmental obstacles.

Handling large and heavy objects are challenging for robotic manipulators, as high-quality motor and reducer modules and careful mechanical design are needed to increase a robotic manipulator’s payload. The module and design costs make heavy-duty robots unaffordable for small and medium manufacturers. On the other hand, collaborative manipulators are safe and can be quickly and flexibly deployed to manufacturing sites. They have a limited payload and cannot lift and move heavy objects that exceed the limitation. Previously, researchers have been developing methods for collaborative manipulators to handle large and heavy objects. The methods took advantage of environmental support to share partial object weight. The robot manipulator performs non-prehensile actions to push, flip, or pivot large and heavy objects [1][2][3][4]. Using collaborative manipulators while considering environmental support allowed the manipulation of large and heavy objects with affordable hardware costs.

However, the previous methods used uniform sampling and search graphs to model the object-environment and object-robot contact states. The methods inherently have completeness problems. Whether a contact and grasping state will be considered for exploration and optimization depends on if the state was encoded in the graph. The methods cannot exploit an unmodeled state to achieve given goals.

Unlike previous methods, we propose a single-short probabilistic contact-rich planning method to explore contact states that meet force constraints automatically. The method is based on the Rapidly-exploring Random Tree (RRT) routine. It randomly explores object motion while maintaining object-environment contact. The contact is not defined as a limited number of points, edges, or faces. It is instead maintained using an expanded contact mesh and may change freely as long as the “crust” formed by the expanded object mesh is in touch with the environment. The expanded mesh is determined functionally by considering the mechanical compliance of a manipulator. Robotic motion are generated using an optimization-based algorithm based on the probabilistically planned object motion. With the proposed method’s help, we can find contact-rich manipulation without explicitly analyzing an object’s contact states and their transitions. The planner and optimizer determine them automatically.

We conducted experiments using a mobile manipulator to analyze the proposed method’s performance. The method could help find contact-rich manipulation trajectories allowing the mobile manipulator to move large, heavy objects with environmental support. The optimizer ensured the manipulator’s joints bore affordable forces not to damage themselves. The proposed method is expected to be an important contribution to contact-rich manipulation and starts a new view for solving related problems.

The organization of this paper is as follows. Section II reviews related work. Section III presents an overview of the proposed method using a schematic diagram. Section IV$\sim$V show algorithmic details like the expanded collision models, RRT-based planning, and robotic trajectory optimization. Section V presents experiments and analysis. Conclusions and future work are drawn and discussed in Section VI.

II Related Work

II-A Manipulating Heavy Objects

Using robots to manipulate heavy objects is a challenging topic [5]. In contemporary studies, people developed special-purpose mechatronic systems, manipulators with a large payload [6], robot-human collaboration [7][8], robot-machine collaboration [9][10][11][12], non-prehensile manipulation policies [13][14][15], etc., to solve the problems.

Particularly, O’Neill et al. [16] developed an autonomous method for safely tumbling a heavy block sitting on a surface using a two-cable crane. Kayhani et al. [17] developed an automated lift path planning method for heavy crawler cranes in no-walk scenarios employing a robotics approach. Du et al. [6] developed a high-duty manipulator to replace heavy disc cutters installed on the head of a tunnel boring machine. Kim et al. [7] proposed a novel human–robot collaboration (HRC) control approach to alert and reduce a human partner’s static joint torque overloading while executing shared tasks with a robot.

In this work, we employ environmental contact as essential support [18] to manipulate heavy objects. In order to realize the manipulation of objects that exceed the payload, we optimized the robot motion by considering contacted object trajectories, supporting forces, manipulator poses, and the load on each joint. One advantage of using the environment is that no additional equipment or facilities are required. While additional equipment can be very effective in accomplishing specific tasks, issues such as loss of versatility still need to be addressed. For example, using a cart to load and move objects makes it very difficult to maneuver in an environment with steps [19]. Also, when robots are integrated with additional equipment, it is necessary to solve a highly constrained closed-chain problem that occurs when robots work together, which results in a considerable restriction on the IK range and requires planning for re-grasping. In contrast, taking advantage of support from environmental obstacles provides more freedom for maintaining contact. With the proposed object trajectory planning and robot motion optimization methods in this work, a robot can find a contact-rich manipulation motion without analyzing an object’s contact modes and transitions and thus move heavy objects while leveraging supporting forces from environmental obstacles.

II-B Manipulation Considering Changing Contact Modes

Contact modes have been an old topic in robotic manipulation [20][21][22][23][24]. Modern studies used contact modes and mode transition graphs to guide probabilistic planning or optimization. For example, Raessa et al. [25] developed a hierarchical motion planner for planning the manipulation motion to repose long and heavy objects considering external support surfaces. Cheng et al. [26] proposed Contact Mode Guided Manipulation Planning (CMGMP) for 3D quasistatic and quasi-dynamic rigid body motion planning in dexterous manipulation. Murooka et al. [27] proposed a planning method of whole-body manipulation for various manipulation strategies. Hou et al. [28] proposed a method to select the best-shared grasp and robot actions for a desired object motion. Some others do not perform detailed planning and leave contact mode selection to optimization. For instance, Sleiman et al. [29] presented a reformulation of a Contact-Implicit Optimization (CIO) approach that computes optimal trajectories for rigid-body systems in contact-rich settings. Aceituno-Cabezas et al. [30] proposed a global optimization model (CTO, Contact-Trajectory Optimization) for this problem with alternated-sticking contact.

In this work, we develop a planner that implicitly takes advantage of environment contact and switches among contact modes to manipulate heavy objects. To enable this contact mode switching, we introduce an extended contact model. This allows objects to change freely, with contact not being fixed to a limited number of edges and faces, and enables planning of contact-rich path panning without detailed analysis of object contact modes and their transitions.

III Schematic View of the Proposed Method

Fig. 2 shows the schematic diagram of the proposed method. The red box represents the user-defined input. The green box is the output. The proposed method accepts “Object Mesh Model”, “Initial and Goal Object Poses”, “Environmental Mesh Models”, “Robot Parameters”, and “Object Weight” as the input. It first computes a set of pre-annotated grasp poses and creates an expanded mesh using the “Object Mesh Model” input data. The pre-annotated grasp poses will be used for incremental planning and optimization. The method incrementally selects a grasp pose, determines if it is accessible at the initial and goal object poses, and plans the object motion and optimizes the robot trajectory. When planning the object motion, the method uses the “crust” formed by the expanded object mesh model and the original one to ensure continuous contact. The planner also examines the force born by the selected grasp pose at each randomly sampled roadmap node to make sure the robot can hold and move the object.

The planning part is not necessarily successful due to strict constraints. When failed, the workflow will return to the “Select a Grasp Pose” stage to try different grasp poses until a feasible object motion is obtained. Once a feasible object motion is obtained, the workflow will switch to the “Optimize the Robot Trajectory” stage to generate robotic motion trajectories. The final robotic motion trajectories may not involve the IK solutions obtained in the “Plan the Object Motion” stage. It is formulated as an optimization problem to minimize the robotic movement while considering the selected grasp pose and its motion in association with the object. Like the planning stage, the optimization is not necessarily successful. It will return to the “Select a Grasp Pose” stage in case of failure. If all stages were performed successfully, the method would produce an optimized robotic motion and the predicted object trajectory under the robotic motion as the output.

IV Planning Object Motion

IV-A Expanding Models Considering Mechanical Compliance

The concept behind expanding object mesh models is to build a “crust” and judge if the object is in contact with the environment by examining if the “crust” overlaps. Fig. 3 illustrates the concept. Here, $\textnormal{M}_{\textnormal{o}}$ indicates the original object mesh model. $\textnormal{M}_{\textnormal{e}}$ indicates the “crust”. We judge the relationship between the object and the environment by recognizing the overlap between $\textnormal{M}_{\textnormal{o}}$ and the environment and $\textnormal{M}_{\textnormal{e}}$ and the environment. The object is considered to be in contact with the environment when $\textnormal{M}_{\textnormal{e}}$ overlaps with it, while $\textnormal{M}_{\textnormal{o}}$ does not. If both $\textnormal{M}_{\textnormal{o}}$ and $\textnormal{M}_{\textnormal{e}}$ overlap with the environment, we consider the object is in a collision.

The thickness of the “crust” is crucial for the assumption to be valid. We define it by considering the mechanical compliance of a mobile manipulator. Since a mobile manipulator is not fixed and the mobile base is not entirely stiff due to springs installed at the wheels, there is slight mechanical compliance at the Tool Center Point (TCP) of the arm component. Fig. 4(a) illustrates the compliance. We compute the compliant values by assuming two orthogonal virtual rotation joints at the base of the mobile manipulator. The two virtual rotational joints will induce absolute compliance, and the “crust” thickness can be computed by considering the rotation ranges of the virtual joints. Fig. 4(b) and (c) illustrates the two orthogonal virtual rotation joints.

We set the mobile manipulator in a vertical pose, applied forces to its TCP, and measured the TCP’s displacements to obtain the rotation ranges. Particularly, we computed the rotation ranges of the virtual joints using the trigonometric relations between the arm length and the displacements. Fig. 4(d) shows the trigonometric relation about the local $x$ direction. Here, $l_{arm}$ is the distance between the assumed joint position and the TCP. $\delta d$ is the measured hand displacement. The rotation range around the local $x$ direction can be computed using

[TABLE]

After obtaining the rotation ranges, we can compute the compliance of the robot hand at an arbitrary arm pose by using an $l_{arm}$ updated using forward kinematics, as shown by Fig. 4(e). The robot will have more minor compliance when folded and larger compliance when expanded. The thickness of the “crust” will be determined dynamically considering the changing $\delta\theta$.

IV-B RRT-Based Object Motion Planning

The object motion is planned using RRT as the backbone. When sampling new nodes using the RRT backbone, we will examine the following constraints to secure continuous support forces from the environment as well as affordable ensuing robot manipulation.

• •

Contact Constraint: The object at a newly sampled pose is in contact with environmental obstacles. The contact is assured by the collision and non-collision relationship of $\textnormal{M}_{\textnormal{o}}$, $\textnormal{M}_{\textnormal{e}}$, and the environmental obstacles’ mesh models, as discussed in the previous subsection.

• •

Force Constraint: The minimum force required for the robot hand must be smaller than the mobile manipulator’s maximum payload.

Especially for the force constraint, the thickness of $\textnormal{M}_{\textnormal{e}}$ is determined dynamically at each object pose. When a new pose is sampled during RRT planning, we will update $\textnormal{M}_{\textnormal{e}}$ following the robot’s TCP position at the pose and determine if the object meets the contact constraint using the updated values.

For a better understanding, the force constraint is illustrated in Fig. 5(a). During a contact-rich manipulation, an object is affected by gravitational force, environmental supporting forces, and robotic holding force. These forces are labeled as $\mathbf{G}$, $\mathbf{F_{s}}$, and $\mathbf{F_{h}}$ in the figure. We assume the environment is stationary and strong enough to provide enough supporting force for the target objects. There is thus no limitation on $\mathbf{F_{s}}$. Under this assumption, the $\mathbf{F_{h}}$ can be obtained by solving the following optimization problem.

[TABLE]

The optimization finds the minimum force the robotic hand needs to balance the object while considering supporting forces from the contact. Equation (2a) is the optimization goal. Equation (2b) and (2c) are constraints for balanced forces and torques. Equation (2d) represents the constraints about friction cones and the maximum robot payload. A grasping pose is judged invalid if this optimization problem has no solution. The system will switch to a different candidate grasping pose and restart the object motion planning, considering the force constraints at the new grasping pose. The candidate grasping poses are planned using a grasp planner developed by the same authors [31], as shown in Fig. 5(b.i). The manipulators’ kinematic constraints are considered at the starting and goal object poses, as shown by Fig. 5(b.ii) and (b.iii). Here, the red hands indicate collided grasping poses. The yellow hands indicate the unreachable grasping poses. The hands with original colors are the feasible poses. The candidate grasping poses are the intersection of the feasible ones at the starting and goal object poses.

The result of RRT-based object motion planning is an object motion path and the associated grasp poses for each element on the path. We represent them using $\{(\mathbf{p_{o}}(t),\mathbf{R_{o}}(t))\}$ and $\{(\mathbf{p_{h}}(t),\mathbf{R_{h}}(t))\}$, where subscripts $\mathbf{o}$ and $\mathbf{h}$ represent object and hand respectively. Notation $t={1,2,...}$ in the parenthesis is a time sequence identifier. The result will be used to optimize the motion of the mobile manipulator later, as presented in the next section.

V Optimizing Robot Motion

We use optimization to produce the joint-level robot motion for moving the object following the planned object motion and hand motion. The optimization problem is formulated as follows.

[TABLE]

This optimization aims to find a robot configuration such that the load born by each joint of the robot does not exceed the upper limit of each joint torque. The optimization is expected to be carried out iteratively between every two sequentially adjacent object poses, and their associated grasp poses. Equation (3a) is the optimization goal. Equation (3b) is the constraint on joint ranges. Equation (3c) is the constraint on the grasping pose, where $FK$ is the forward kinematics. Equation (3d) is the constraint on joint torques. The torque born by each joint is computed using the robot Jacobian $J$ and the force $F_{h}$ found by solving the object motion planning problem (equation (2)). Equation (3e) is the constraint on the moving mobile base. We allowed the robot to translate along its local $x$ direction while keeping the other freedoms fixed. There was no particular reason for this constraint. We included it to accelerate optimization. Like object motion planning, the optimization problem may not have a solution. If a feasible robot motion is not found, the grasp pose is determined to be invalid. The system will switch to a different candidate grasping pose and return to the motion planning part to start a new optimization routine.

VI Experiments and Analysis

We carried out experiments and analyses to examine the performance of the method. Notably, we used xArm7 (UFACTORY) for a cooperative manipulator, used Water2 (YUNJI TECHNOLOGY) for a mobile base, and mounted a xArmGripper (UFACTORY) to the xArm7 manipulator as the robot hand. The payload of the xArm7 manipulator is 3.5 kg without any end-effector and 1.87 kg after mounting the hand. The environmental obstacles and objects used in the experiments included a work table (length$\times$width$\times$height$=180$ cm$\times 74$ cm$\times 75.5$ cm, weight$=12.2$ kg), a Japanese tatami block (length$\times$width$\times$height$=$164 cm$\times$82 cm$\times$3 cm, weight$=2.5$ kg), and a plywood cabinet (length$\times$width$\times$height$=$29 cm$\times$41 cm$\times$29 cm, weight$=5.5$ kg). Fig. 6 illustrates the robots, environmental obstacles, and objects.

VI-A Mechanical Compliance

We measured the mechanical compliance of the mobile manipulator following the method presented in Section IV.A. Especially, we set the manipulator part of the robot to a vertical pose, attached optical markers to its TCP, and measured the TCP displacements by recording the changes in marker positions. The setup is shown in Fig. 7(a) and (b). As illustrated in the figure, we moved the TCP of the vertically posed manipulator around the mobile base’s local $x$ and $y$ axes to obtain the ranges of the two assumed orthogonal joints.

Fig. 7(c) shows the recorded data. Following the data, we concluded that the $\delta d$ in the two directions were 15 mm and 25 mm, respectively. The rotation ranges of the two virtual joints are thus $0.5^{\circ}$ and $0.859^{\circ}$ following equation (1).

VI-B Performance of Object Motion Planning

After obtaining the rotation ranges, we used them to determine the thickness of $\textnormal{M}_{\textnormal{e}}$ dynamically and thus carried out RRT-based object motion planning. First, we carried out the experiments in simulation using the Japanese Tatami Block and planned the object’s motion for the two tasks shown in Fig. 8. In the first task, the robot planned its motion to flip the tatami on the table. In the second task, the robot planned its motion to unload the tatami block from the table to the ground. Since the tatami is heavier than the robot’s maximally affordable payload, the robot must make sure the environment partially supports it during manipulation. The RRT parameters were configured as follows during the experiments: Maximum iterations$=$10000, Smoothing iterations$=$350, and Maximum planning time $=$100 s.

The left part of Table I shows the results of the first task. To understand the dynamic method’s performance, we also plan using fixed $\textnormal{M}_{\textnormal{e}}$ thickness and compared them with our dynamic method. Columns 1, 2, and 3 of the table are the results with fixed thicknesses. The fixed values were 0.5 cm, 1.5 cm, and 3 cm, respectively. The last column is the results using the dynamically changing thickness computed following the changes of the robot’s TCP. The planning time and success rate data are the averages of ten times of attempts. If the distance between the object and the robot exceeded the length of the arm, the thickness was defined as 0. This constrained the trajectory planning to be within the manipulator’s operating range. The results show that different fixed thicknesses led to different performances. Thicker “crust”s will lead to a higher success rate and smaller time costs. When the thickness was 0.5 cm, the success rate was 0.5 cm. Maintaining contact during probabilistic planning under such a thin thickness was very difficult. In contrast, the success rate reaches 90 when the thickness is changed to 3 cm. The success rate of a dynamically changing thickness had a performance competitive to the results of 3 cm thickness. The range of the dynamically changing thickness is from 2.6 cm to 3.35 cm. In the best case, it grew to a value larger than 3 cm. This happened when the robot was most expanded. In the worst case, it reduced to 2.6 cm when the robot was most folded. We could expect the dynamically changing thickness to be more flexible in the best case and safer when the robot is folded.

The right part of Table I shows the results of the second task. The planner successfully solved the problem even though the thickness was fixed to 0.5 cm. However, the success rate was low. Only two of the ten attempts had positive values. Like the first task, the planning time decreased, and the success rates grew as we increased the thickness of $\textnormal{M}_{\textnormal{e}}$. The proposed dynamic thickness had competitive performance to the 3rd column of the table. The dynamic range differs from the previous task as the robot’s TCP positions shifted significantly.

VI-C Real-World Experiments and Torque analysis

Then, we optimized the robot motion for the two tasks based on the object trajectories obtained using the dynamic thickness. The simulation and real-world execution results are shown in Fig. 9. The results indicate that the robot maintained contact between the tatami and the environment during the manipulation to let the environment partially share the workload. For task 1, the robot maintained contact between a long edge of the tatami and the table surface. A partial weight of the tatami was supported by the work table, as seen in (a.ii), (a.iii), and (a.iv). For task 2, the robot maintained contact between the tatami surface and the short edge of the table. The table partially supported the tatami until it touched the ground, as seen in (b.iii), (b.iv), and (b.v).

We measured the force born by the robot gripper in the two tasks using the force sensor installed at the end flange of the robot (see Fig. 6(a) for the position of the force sensor). The force curves are shown in Fig. 10(a.i) and (b.i). The maximum forces born by the robot hand in the two tasks were smaller than 35 N. The robot was bearing a payload smaller than its 3.5 kg payload when performing the tasks. The executions were successful, and the proposed methods are thus considered effective. Fig. 6(a.ii) and (b.ii) show the torque at each joint of the robot. The torque curves are separated into three groups and plotted in three different diagrams since joints 1 and 2, joints 3, 4, and 5, and joints 6 and 7 had the same maximum torques, respectively. The dashed lines in the diagrams show the maximum torque that can be born by the joints. The torque curves were coherent with force at hand. All of them were smaller than the maximum torques of the joints.

For comparison, we simulated the torques born by the joints after removing the supporting table. During the simulation, the robot followed the same motion to manipulate the tatami block. Fig. 6(a.iii) and (b.iii) show the results. We can observe from the results that without supporting forces from the table, the 5th and 6th joints of the robot in the first task and the 3rd and 7th of the robot in the second task need to provide torque larger than their maximally bearable values. The observation confirmed that environmental support was helpful for the robot to manipulate the tatami block.

We also compared the necessity of robot motion optimization. Fig. 10(a.iv) and (b.iv) show the torque curves without solving the optimization problem presented in equation (3). The robot motion was generated by directly solving the inverse kinematics of the robot following the trajectory of the object and the holding hand. The curves were slightly different from the ones measured with motion optimization. For task 1, the robot could still successfully finish the execution, although the torques born by the robot joints became more significant than the optimized results. For the second task, the robot failed to carry out the execution. The curves in (b.iv) were simulation results. We could observe from them that the 6th joint of the robot needed to provide a much larger torque to perform the motion successfully. Optimizing the robot motion helped find a proper robot pose sequence and thus avoid excessive joint load. For our readers’ convenience, we visualized two optimized and non-optimized robot motion results in Fig. 11. Interested readers are recommended to see the figures for detailed examination.

In addition to the tatami block, we also conducted experiments using the cabinet shown in Fig. 6. Like the tatami block, we defined two tasks shown in Fig. 12(a) and (b), and asked the system to find a manipulation sequence for the mobile manipulator to move the cabinet from the initial poses to the goal poses. Since the cabinet is much shorter than the tatami block, we included a smaller and lower table beside the original one and let the robot unload the cabinet from the original table to the lower one. With the help of the proposed method, the robot successfully solved both problems. Readers may also find execution details in the video supplementary published with this manuscript.

VII Conclusions and Future Work

We proposed an expanded contact model and used it to plan the trajectory of an object while maintaining contact. The trajectory of the robot based on the trajectory was obtained by solving an optimization problem that incorporates joint torque constraints and other factors. The experimental results show that the method can be applied to various objects and different target states. It helped secure the manipulation of heavy objects by sharing the load between the robot and environmental obstacles. The robot can avoid overloaded joint torque by following the generated manipulation motion. In the future, we plan to verify whether the simulation behavior can be correctly realized in a real environment while improving the problems mentioned in the discussion section.