Effects of Different Hand-Grounding Locations on Haptic Performance With a Wearable Kinesthetic Haptic Device

TL;DR

This study investigates how different hand-grounding locations affect haptic perception and user experience with a wearable kinesthetic device, providing insights for improved design in VR and teleoperation.

Contribution

It introduces a novel wearable device and systematically compares three grounding locations, revealing their impact on perception sensitivity, realism, and comfort.

Findings

Proximal phalanx grounding yields higher perception sensitivity.

Middle phalanx grounding enhances perceived realism for certain feedback directions.

Back of the hand grounding offers greater comfort for specific feedback modes.

Abstract

Grounding of kinesthetic feedback against a user's hand can increase the portability and wearability of a haptic device. However, the effects of different hand-grounding locations on haptic perception of a user are unknown. In this letter, we investigate the effects of three different hand-grounding locations-back of the hand, proximal phalanx of the index finger, and middle phalanx of the index finger-on haptic perception using a newly designed wearable haptic device. The novel device can provide kinesthetic feedback to the user's index finger in two directions: along the finger-axis and in the finger's flexion-extension movement direction. We measure users' haptic perception for each grounding location through a psychophysical experiment for each of the two feedback directions. Results show that among the studied locations, grounding at proximal phalanx has a smaller average just noticeable difference for both feedback directions, indicating a more sensitive haptic perception. The realism of the haptic feedback, based on user ratings, was the highest with grounding at the middle phalanx for feedback along the finger axis, and at the proximal phalanx for feedback in the flexion-extension direction. Users identified the haptic feedback as most comfortable with grounding at the back of the hand for feedback along the finger axis and at the proximal phalanx for feedback in the flexion-extension direction. These findings show that the choice of grounding location has a significant impact on the user's haptic perception and qualitative experience. The results provide insights for designing next-generation wearable hand-grounded kinesthetic devices to achieve better haptic performance and user experience in virtual reality and teleoperated robotic applications.

Tables

Table 1. TABLE I : Results of the two psychophysical experiments for stiffness discrimination. In Study A, the hand-grounded haptic device provided feedback along the axis of the finger with three different grounding locations. In Study B, the feedback direction was flexion extension movement of the index finger.

Grounding Location		Back of the Hand		Proximal Phalanx		Mid Phalanx
	Subject No.	PSE (N/m)	JND (N/m)	PSE (N/m)	JND (N/m)	PSE (N/m)	JND (N/m)
Study A	1	154.42	57.15	150.74	47.35	120.34	41.32
	2	111.17	9.86	107.58	17.44	103.37	6.2
	3	139.75	48.5	129.95	34.27	81.3	7.32
	4	87.56	6.07	92.95	5.72	102.27	20.24
	5	98.62	32.37	85.2	6.79	114.85	19.04
	6	113.23	13.08	104.68	25.28	100.74	20.1
	7	99.6	13.21	108.99	7.97	94.68	9.48
	8	118.75	46.88	128	31.13	109.11	38.95
	9	115.5	20.23	103.4	10.16	105.51	19.72
	10	114.64	12.93	103.46	12.15	116.09	19
	11	85.87	7.32	108.22	12.65	117.25	17.41
	12	92.94	6.66	114.14	22.17	91.54	15.65
	Mean	111.004	22.855	111.442	19.423	104.754	19.536
	Std. Dev.	19.581	17.677	16.843	12.404	11.178	10.411
Study B	1	96.06	0.17	101.2	12.26	103.8	13.75
	2	92.71	13.51	104.59	13.02	92.79	8.64
	3	125.72	40.06	94.9	28.36	121.88	45.98
	4	104.87	6.41	98.32	14.36	87.34	6.75
	5	115.52	41.74	114.25	40.84	102.2	20
	6	122.16	33.52	90.2	16.44	119.91	16.88
	7	121.54	20.98	116.83	36.64	125.8	64.77
	8	105.76	10.69	108.89	11.67	104.6	26.4
	9	98.85	25.9	100	16.83	117.3	25.28
	10	99.45	41.48	95.07	22.3	104.98	24.18
	11	138.96	41.62	127	30.4	122.64	24.03
	12	113.34	38.78	95.35	45.78	120.79	60.43
	Mean	111.245	26.238	103.883	24.075	110.336	28.091
	Std. Dev.	13.426	14.761	10.435	11.520	12.181	18.222

Equations

$$p(x,z)={\left[r(1-\cos \theta ),r\sin \theta \right]}^T.$$

$$T_j$$

$$p_{xj}$$

$$p_{zj}$$

$$s_a=(r+r_a)({\theta }_o-{\theta }_t),$$

$$s_b=(r-r_b)({\theta }_o-{\theta }_t).$$

$$e(t)$$

$$U$$

$$JND=\frac{(PSE-J_{25})+(J_{75}-PSE)}{2}.$$

Full text

I INTRODUCTION

Preprints of the IEEE Robotics and Automation Letters (RAL) paper presented at the

2019 International Conference on Robotics and Automation (ICRA), Palais des congres de Montreal, Montreal, Canada,

May 20-24, 2019. The final version of the article can be accessed at DOI: 10.1109/LRA.2018.2890198

The majority of the existing haptic devices providing kinesthetic feedback are world grounded [1]. They offer numerous advantages like high forces and torques, many degrees of freedom (DoF), and a wide dynamic range. These features allow such devices to provide more realistic haptic renderings compared to tactile haptic devices that only stimulate the skin. However, the world-grounded kinesthetic haptic devices generally have a large footprint as well as limited portability and wearability, which limits their application and effectiveness for many virtual and real-world applications. World-grounded haptic devices also offer a limited range of motion to the user due to the scaling of weight and friction with increased size [2].

On the other hand, wearable haptic devices must be portable and typically offer a large range of motion. But, majority of the existing wearable haptic devices are tactile in nature and provide feedback in the form of vibration or skin deformation. They are commonly grounded against the user’s fingertip or the nearby region [1]. Though tactile feedback is capable of providing directional cues and aiding users in completing various tasks, it may not be sufficient to perform certain tasks, such as the suture knot-tying in robot-assisted surgery [3], and manipulating objects in virtual reality [4]. As demonstrated by Suchoski et al. [2] in their study, kinesthetic feedback is capable to give more sensitive haptic information to carry out a grasp-and-lift task than the skin deformation feedback (a form of tactile feedback). Similarly, the role of kinesthetic (force) feedback in surgical training and skill development looks very promising [3].

Kinesthetic haptic devices, that are not world grounded but instead impart feedback by grounding forces against the user’s hand (hand-grounded haptic devices), provide a solution to challenges of portability, wearability and limited workspace in kinesthetic haptic devices. As noted by Pacchierotti et al. [1], the primary advantage of wearable kinesthetic devices is their small form factor as compared to the world-grounded devices. Similarly, body-grounded kinesthetic devices, i.e. Exoskeletons, could be another potential solution, but they generally encumber the user movement and are difficult to don and doff. However, designing these hand or body-grounded devices is challenging due to the need for increased forces/torques and number of degrees-of-freedom (DoF) in comparison to the fingertip tactile devices. Additionally, the effects of hand-grounded kinesthetic feedback on users’ perception and haptic experience are still unknown.

There exist numerous examples of hand-grounded kinesthetic haptic devices, including [5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]. These devices are either grounded against the back of the hand [5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19], act like a glove  [20, 21, 22], are grounded against the user’s palm [23, 24], or are grounded against the user’s fingers [25, 26]. To the best of our knowledge, there exists no device that can be grounded against different locations on the user’s hand or a study that explains the effect of different grounding locations on the user’s haptic perception and qualitative experience with kinesthetic (force) feedback.

We aim to study the effects of different hand-grounding locations on a user’s haptic perception by providing kinesthetic feedback on the user’s index finger tip. For this purpose, a wearable 2-DoF haptic device is designed that can provide kinesthetic feedback grounded at three different regions of the user’s hand (Fig. 1): (i) back of the hand, (ii) proximal phalanx of the index finger, and (iii) middle phalanx of the index finger. The light-weight and modular design provides kinesthetic feedback in two directions: (A) along the index finger axis, and (B) in flexion-extension.

We aim to understand how different hand-grounding locations affect the user’s haptic performance and overall experience. To identify the significance and impact of different hand-grounding locations, two psychophysical experiments are carried out using the method of constant stimuli [27] — one for each feedback direction. The participants were asked, in separate trials, to discriminate the stiffness of two virtual surfaces based on the kinesthetic feedback provided by the hand-grounded device. The Point of Subjective Equality (PSE) and Just Noticeable Difference (JND) were computed to measure the effective sensitivity and precision of the participants’ perception of stiffness for each hand-grounding location, in both feedback directions. The PSE gives insight about the accuracy of the applied/perceived feedback, as it represents the point where the comparison stimulus (stiffness) is perceived by the user as identical to the standard stimulus. JND indicates the resolving power of a user and is defined as the minimum change in the stimulus value required to cause a perceptible increase in the sensation [27].

The results show that the choice of grounding location has profound impact on the user’s haptic perception (measured through the metrics described above) and experience (based on user ratings). These findings provide important insights for the design of next-generation kinesthetic feedback devices, particularly in terms of grounding of forces, to achieve compelling and natural kinesthetic haptic interaction in real-world haptic and robotic applications. For example, using these findings we can now design hand-grounded wearable kinesthetic devices with appropriate grounding to offer superior haptic performance and user experience. As hand-grounded devices offer comparatively larger operating range and smaller from factor than their world-grounded counterparts, the knowledge related to the choice of hand-grounding location may help to increase the use of wearable kinesthetic devices in the fields of haptics and robot teleoperation. The contribution of this work is the design of a novel wearable kinesthetic device and study results for understanding of the role played by different hand-grounding locations on user stiffness perception.

II DEVICE DESIGN & CONTROL

II-A Design

The device has a base (Fig. 3) that can be tied to the back of the user’s hand using a hook-and-loop fastener. It has two rings (A and B) which are fitted to the proximal and middle phalanxes of the index finger. The fingertip cap is connected to actuators A and B through two cables, which route through the passage holes on rings A and B, as shown in Fig. 3. When both actuators A and B move in the same direction (clockwise or anti-clockwise), a flexion or extension movement at the finger is produced. When both actuators move in opposite directions, a pull force is generated along the finger axis.

To provide hand-grounded kinesthetic feedback at the fingertip, a number of grounding locations can be used. Fig. 1 shows the three grounding locations considered in this case: the back of the hand, proximal phalanx of the index finger, and middle phalanx region of the index finger. Another potential location, the palm region, was rejected because such an arrangement may affect the user’s ability to open/close the hand and fingers.

To achieve different groundings, the device has three different modes. In mode A (Fig. 2(a)), the back of the hand acts as the grounding location. In mode B (Fig. 2(b)), the base is physically connected to the ring A at the proximal phalanx, providing grounding at this region. In mode C (Fig. 2(c)), the base is rigidly connected with both rings to provide grounding at the middle phalanx region. Different device modes enable execution of different joints of the index finger in the flexion-extension direction. For example, in mode A, the torque is applied at all three joints (MP1, PIP, and DIP). In mode B, only PIP and DIP joints are executed. In mode C, the torque applies only at the DIP joint.

Based on its kinematic design and actuator specifications, the device can apply, in different modes, a maximum force of 28.9 N along the finger axis and a torque in the range of 80 to 300 N-mm at the fingertip. It is driven by two Faulhaber 0615 4,5S DC-micromotors with 256:1 gearboxes, and 50-counts-per-revolution optical encoders are used for position sensing. The device prototypes with different grounding modes (Fig. 5) weigh 31, 43, and 49 grams, respectively.

II-B Kinematics

The device renders forces on the user’s index finger by controlling the tendon lengths. To calculate the position and configuration of the finger, we use a robot-independent kinematic mapping between the actuator space and the task space. The obtained homogeneous transformation remains identical for all three grounding modes of our device. It is assumed that the device’s tendons, when fit to the user index finger, exhibit a continuum-curve shape. The geometry of this curve allows determination of the tip position and configuration of the finger. Fig. 4 shows a simplified representation of the haptic device in such a scheme.

As the haptic device aims to provide kinesthetic feedback in two directions (along the finger axis, and in the finger’s flexion-extension direction), the kinematic mapping between the inertial frame ($O$) and the fingertip ($p(x,z)$) is described in a 2-D ($x-z$) plane. Tendon lengths ($l_{a},l_{b}$), their respective distance from tip center-point ($r_{a},r_{b}$), and the arc parameters, namely length ($l$) and radius ($r$), are used to determine the tip position and figure configuration in $x-z$ plane. The position of fingertip can be expressed as,

[TABLE]

The homogeneous transformation for tendons $a$ and $b$, from $O$ to $p_{a}(x,z)$ and $p_{b}(x,z)$ respectively, is

[TABLE]

[TABLE]

The displacements of tendons $a$ and $b$ can be expressed in terms of arc radius and angles as

[TABLE]

where $\theta_{o}$ is the initial angle angle of tendon $a$ and $\theta_{t}$ represents the tendon angle at time $t$.

II-C Control System

Using the tendon displacements (5) and (6), a separate control is implemented for each of the actuators to apply force and control the user’s finger configuration. Fig. 6 shows the block diagram of the control in the virtual reality setup. The control of the 2-DoF kinesthetic device was achieved by using a Nucleo-F446ZE board by STMicroelectronics™connected to a Desktop computer via USB. The microcontroller reads the encoders of the motors and receives the desired force from the virtual environment sent using a PC’s serial port. Using this information, it calculates the desired torque output of the motors. The control loop runs at a frequency of approximately 1 kHz. The CHAI3D framework was used to render the 3-D virtual reality environment [28] using the god-object algorithm [29] to calculate desired interaction force. The user can move the cursor (red sphere in Fig. 7(a) & (b)) in 3-D space. Because the wearable device has only 2 DoFs, the third dimension does not give any force feedback to the user. Given the nature of the tasks in the user studies, the third dimension ($y$-axis) is not required to display the force feedback.

The user’s hand position ($\vec{x_{u}}$) is tracked using a Phantom Omni haptic device (set up to provide no haptic feedback, just position tracking) from SensAble Technologies, Inc. and sent to the virtual environment as $\vec{x_{d}}$. The resulting interaction force command from the virtual environment ($\vec{F_{d}}$) is calculated in the computer and then fed to the hand-grounded haptic device. The device then uses a mapping between the force magnitude and the device tip position (force-position translator) to output the desired tip position to the PD controller which, using the encoders mounted on each motor shaft, can estimate the current tip position and configuration and outputs the appropriate tendon displacements ($\vec{s_{c}}(a,b)$) to the device’s motors. As the tendons shorten, the user’s finger tip is moved to the right position, allowing him/her to feel a force.

The PD controller error and the control law are

[TABLE]

where, $K_{P}$ represents the proportional gain and $K_{D}$ is the derivative gain. $e(t)$ is the position error, $y(t)$ represents the motor shaft position, and $r(t)$ is the reference position calculated from the desired tendon displacements ($\vec{s_{d}}(a,b)$ in Fig. 6).

III USER STUDY

To evaluate the effects of the three different hand-grounding locations on the user’s haptic perception and experience, we conducted two separate user studies (Study A & Study B); one for each haptic feedback DoF provided by the hand-grounded device. In Study A, the kinesthetic feedback is provided along the axis of the user’s index finger. In Study B, the feedback is provided along the flexion-extension movement of the finger. The purpose of evaluating each feedback DoF separately is to develop a clear understanding of the relation between the hand-grounding location and the corresponding feedback direction.

III-A Study A: Feedback Along the Finger Axis

III-A1 Experimental Setup

13 subjects (9 males and 4 females) participated in this study, which was approved by the Stanford University Institutional Review Board. The metrics were PSE and JND of stiffness perception while the hand-grounded device was set up for each of the three grounding locations (back of the hand, proximal, and middle phalanx of the index finger). All subjects participated in the experiment after giving informed consent, under a protocol approved by the Stanford University Institutional Review Board. The participants used the hand-grounded haptic device on their right hand and performed tasks in a virtual environment, while holding the stylus of the Phantom Omni device in the same hand (Fig. 7(a)). A pilot study was conducted to determine a convenient posture to hold the Phantom Omni stylus while the kinesthetic device is donned to the index finger. In the user studies, the participants were instructed to hold the Phantom Omni device in that predefined way to make sure that its stylus does not come into contact with the wearable kinesthetic device.

III-A2 Experimental Procedure

Each participant used the haptic device configured for each of the three hand-grounding locations in a predetermined order to minimize the effect of selection bias. As mentioned earlier, a Phantom Omni device was used to track the user hand position during the experiments as shown in Fig. 7(a). The Phantom Omni only determined the user hand position, while the kinesthetic feedback was rendered by the hand-grounded haptic device. Participants were wore ear protection to suppress the motor noise in order to avoid sound cues. After the experiments were completed, the participants rated the realism of haptic feedback and comfort/ease-of-use for all three devices with different hand-grounding locations on a scale of 1-7: 1 meaning ‘not real’ and 7 meaning ‘real’, or 1 for ‘not comfortable’ and 7 for ‘comfortable.’ The realism was with respect to the users’ feeling as if they would be pressing against a very smooth real surface using their right-hand’s index finger.

III-A3 Method

We conducted a two-alternative forced-choice experiment following the method of constant stimuli [27]. Subjects were asked to freely explore and press against the two virtual surfaces shown on the virtual environment display and state which surface felt stiffer. In each trial, one surface presented a reference stiffness value while the other presented a comparison stiffness value. The reference stiffness value was selected to be 100.0 N/m. The reference value was included as one comparison value, and the other comparison values were then chosen to be equally spaced: 10, 28, 46, 64, 82, 100, 118, 136, 154, 172, and 190 N/m.

Each of the eleven comparison values was presented ten times in random order for each of the three hand-grounded haptic devices over the course of one study. Each participant completed a total of 110 trials for each grounding mode (330 trials for the entire study). The participants used the kinesthetic feedback from the hand-grounded device to explore the virtual surfaces until a decision was made; they recorded their responses by pressing designated keyboard keys, corresponding to which virtual surface they thought felt stiffer. Subject responses and force/torque data were recorded after every trial. There was no time limit for each trial, and participants were asked to make their best guess if the decision seemed too difficult. Subjects were given an optional two-minute break after every fifty-five trials, and a ten-minute break after the completion of each grounding mode.

III-B Study B: Feedback in the Flexion-Extension Direction

In study B, the kinesthetic feedback was rendered along the flexion extension movement direction of the index finger. A total of 14 subjects (9 males and 5 females) participated, and the study was approved by the Stanford University Institutional Review Board. The procedure was the same as in Study A. However, in Study B the virtual surfaces were presented lying in the horizontal plane (Fig. 7(b)) to make the haptic feedback intuitive for the user.

IV RESULTS & DISCUSSION

For both user studies, we determined the number of times each participant responded that the comparison value of stiffness was greater than the reference stiffness value. A psychometric function was then fit for each participant’s response data to plot a psychometric curve, using the python-psignifit 4 library (https://github.com/wichmann-lab/python-psignifit). Data from twenty-four out of the twenty-seven subjects fit sufficiently to psychometric functions and the mean JNDs and PSEs for both experiments were determined. Example plots for a representative subject are shown in Fig. 8. Three relevant values: the PSE, the stimulus value corresponding to a proportion of 0.25 ($J_{25}$), and the stimulus value corresponding to a proportion of 0.75 ($J_{75}$) were determined. The JND is defined as the mean of the differences between the PSE and the two J values $J_{25}$ and $J_{75}$:

[TABLE]

The PSE and JND results of the psychophysical experiments for both studies are summarized in Table 1. Because these studies use a single reference force, the Weber Fractions (WFs) are simply the JNDs scaled by the reference value. Therefore, we do not report WF separately.

In Study A, the best average PSE (closer to the reference value) for stiffness perception among all three grounding locations is found for the middle phalanx location of the index finger (104.75 N/m), shown in Fig. 9. This indicates that the grounding location closer to the fingertip helps users to perceive the stiffness more accurately. This is also supported by the user ratings for the realism of kinesthetic feedback, as shown in Fig. 11. The smallest average JND was found for grounding at proximal phalanx (19.42 N/m), which is closely followed by average JND values for grounding location at the proximal phalanx (19.54 N/m). Like the PSE, the average JND showed largest value for back of the hand grounding location (see Fig. 10). This indicates that the proximal and middle phalanx are preferable locations, in the given order, to have a more realistic and accurate feedback perception. However, the user ratings for the comfort and ease-of-use indicate that the back of the hand is a more desirable grounding location.

In Study B, the best average PSE (closer to the reference value) for stiffness perception among all three grounding locations is found in the grounding at the proximal phalanx of the index finger (103.88 N/m), shown in Fig. 9. This grounding location also results in the smallest average JND value (24.07 N/m) among all three grounding locations. The user ratings for kinesthetic feedback realism and the comfort/ease of use, as show in Fig. 11, also rate this location as the best to impart most realistic and comfortable haptic experience. The second best location in terms of average JND value is the back of the hand. This holds for the feedback realism ratings as well. The grounding location with least realistic feedback ratings and largest average JND (28.09 N/m) was the proximal phalanx location.

If we compare the average JND values across both studies, the values for feedback along the finger-axis (Study A) are significantly smaller than that of the feedback along flexion-extension direction (study B). This indicates that the haptic device was able to provide better haptic feedback in case of Study A, i.e. along the axis of the index-finger. The reason for this probably relates to the simpler nature of this feedback direction where the finger configuration remains unchanged during all modes. However, the realism and comfort ratings show a distinct pattern; realism is higher for Study B (kinesthetic feedback along flexion-extension) when the grounding locations are the back of the hand and proximal phalanx. The realism in case of Study A is higher than that of the B when grounding location is middle phalanx. This again depends on the different nature of the second feedback DoF, where the finger configuration has to change in order to render a torque at the finer joints. The use of the Phantom Omni for tracking may introduce some passive forces that introduce variance in the study. Despite this, we observed significant performance differences among the studied grounding locations.

On the other hand, the comfort/ease-of-use ratings are higher for Study A than B, when the grounding locations are the back of the hand and the middle phalanx, respectively. The feedback in flexion-extension movement (Study B) has shown higher comfort ratings than for the finger-axis direction (Study A) when grounding is set as the proximal phalanx region. The highest comfort rating among all grounding locations across both studies is given to the proximal phalanx, and that is for the feedback along the flexion-extension movement. Similarly, the highest comfort rating is given to the same grounding location, i.e., proximal phalanx, across both studies, and that too is for the feedback along flexion-extension direction.

V CONCLUSION

A novel hand-grounded kinesthetic feedback device was created for studying the effect of different grounding locations on the user’s haptic experience. The device can provide kinesthetic feedback along the user’s index finger, and in its flexion-extension movement direction. Two psychophysical experiments – one for each feedback DoF – were conducted to evaluate the user’s haptic performance and experience. It is shown that the choice of grounding-location in wearable haptic devices has significant impact over the user haptic perception of stiffness. The realism of the haptic feedback increases, while the comfort level decrease, as the grounding location moves closer to the fingertip. The relationship between the grounding-location and user haptic perception is similar in both feedback directions. If the design objective is to achieve maximum comfort, feedback realism, and best haptic perception in both DoFs simultaneously, it is recommended to have grounding at the proximal phalanx region of the finger.

These findings about the choice and impact of different hand-grounding locations give important insights for designing next-generation wearable kinesthetic devices, and to have better performance in a wide range of applications, such as virtual reality and robot teleoperation. In the future, we plan to conduct further experiments to explore the effects of these hand-grounding locations when the kinesthetic feedback is applied to both DoFs simultaneously.