Deadeye Visualization Revisited: Investigation of Preattentiveness and Applicability in Virtual Environments

TL;DR

This paper investigates the Deadeye highlighting technique in 3D virtual environments, evaluating its effectiveness and robustness in VR, and exploring its application in volume rendering with user feedback.

Contribution

It extends Deadeye's application from 2D to 3D visualizations, providing quantitative evaluation and practical insights for VR scenarios.

Findings

Deadeye maintains high accuracy (>90%) in 3D VR with distractors.

The technique effectively suppresses binocular disparities for target highlighting.

User survey indicates benefits and limitations in VR volume rendering.

Abstract

Visualizations rely on highlighting to attract and guide our attention. To make an object of interest stand out independently from a number of distractors, the underlying visual cue, e.g., color, has to be preattentive. In our prior work, we introduced Deadeye as an instantly recognizable highlighting technique that works by rendering the target object for one eye only. In contrast to prior approaches, Deadeye excels by not modifying any visual properties of the target. However, in the case of 2D visualizations, the method requires an additional setup to allow dichoptic presentation, which is a considerable drawback. As a follow-up to requests from the community, this paper explores Deadeye as a highlighting technique for 3D visualizations, because such stereoscopic scenarios support dichoptic presentation out of the box. Deadeye suppresses binocular disparities for the target object, so we cannot assume the applicability of our technique as a given fact. With this motivation, the paper presents quantitative evaluations of Deadeye in VR, including configurations with multiple heterogeneous distractors as an important robustness challenge. After confirming the preserved preattentiveness (all average accuracies above 90 %) under such real-world conditions, we explore VR volume rendering as an example application scenario for Deadeye. We depict a possible workflow for integrating our technique, conduct an exploratory survey to demonstrate benefits and limitations, and finally provide related design implications.

Full text

\teaser

Left: Our experiments in VR with homogeneous and heterogeneous distractors, as we investigate the preattentiveness and robustness of Deadeye in such scenarios. Right: We demonstrate and evaluate volume rendering in VR as a possible real-world application scenario for our technique.

\vgtcinsertpkg

Introduction

Highlighting objects of interest in scientific visualizations allows our attention to be attracted and guided. Researchers have uncovered a number of effective ways to make an object pop out, be it color, motion, or flickering. Such visual cues can be recognized by our visual system preattentively, i.e., within a split second, no matter how complex the overall visualization. Hence, advances in the exploration of preattentive features can provide substantial benefits to visualization researchers and practitioners.

Our prior work introduced Deadeye [39], a preattentive visualization technique based on dichoptic presentation. Deadeye attracts and guides attention by rendering the target object for one eye only, which works preattentively due to the induced binocular rivalry. One particular benefit to visualizations is that Deadeye does not modify any visual properties of the target object in contrast to other preattentive cues that have to alter the target by recoloring, reshaping, or displacing (3D depth cue). Clearly, such alterations are undesirable, as they might lead to data misinterpretation. Also, by using Deadeye, we do not have to reserve visual dimensions, such as color for highlighting, and can use these variables for, e.g., encoding more data dimensions. Our studies also confirmed that the display of inconsistent stimuli for each eye does not lead to headache or any other physical strain, which is an important consideration for real-world applicability.

To render the target for one eye only, Deadeye relies on a stereoscopic setup such as a 3D TV with stereo glasses. The only difference between the left and right image is the presence or absence of the target. Hence, one might consider such a setup solely for highlighting purposes as an overkill, and several members of our community proposed applying Deadeye in a truly stereoscopic environment. In such a case, our technique would not require any additional hardware and would perform out of the box. Furthermore, due to the availability nowadays of consumer VR equipment, visualization research increasingly employs virtual environments as a target scenario [37, 65, 44, 24, 63, 51], and having more complex visualizations in VR triggers the need for suitable highlighting approaches.

Given that community-driven motivation, we revisit Deadeye under stereoscopic conditions. In contrast to more established preattentive cues such as color or flickering, it is rather difficult to predict the behavior of Deadeye in such a setup due to its monocular nature. By removing the object for one eye, we lose the binocular disparity information for the target, which is a crucial part of depth estimation. However, the disparity-based mechanism is not the single point of failure, as our vision system relies on numerous mechanics, such as occlusion geometry [75], to estimate the depth of an object, which suggests a successful application of Deadeye in VR scenarios.

0.1 Motivation for Extending Deadeye to VR

There are manifold reasons for visualization in VR, such as a better understanding of spatial relationships [64], or the increased presence and an improved cognitive map due to natural locomotion (i.e., walking) [62, 38, 61]. Nevertheless, these advantages cannot prevent us from possibly getting lost in the amount of visualized data, and, thus, we continue to need robust and intuitive highlighting techniques. And while certain preattentive cues, such as color, are not affected by the transition to VR, temporal approaches, such as flickering, often interfere with aliasing caused by constant micromovements in VR. Hence, we postulate that establishing Deadeye as a validated highlighting approach in VR without occupying any additional visual dimension offers significant benefits for the visualization community, as outlined in Figure 1.

One possible application scenario of Deadeye is the educational context. Instructors in virtual reality classes need a subtle, yet efficient way to draw the students’ attention to a certain element or an area, such as a particle shower caused by a near-light-speed collision in the Belle II experiment [14]. Similarly, we can utilize Deadeye to highlight particles or pathlines in flow field visualizations [56], as our method is not limited to a single target. The only requirement is that the highlighted property, target, or region need to be binary in their nature, as Deadeye either shows or removes it for one eye.

Such highlighting is also beneficial for visual storytelling, be it for experts or the general audience. One example is the biological exploration of cells in VR to better understand the cellular processes [33]. Again, Deadeye can be used to emphasize various points of interest, such as the nucleus or endosomes, and help the audience to follow their movement during the visualization.

Another candidate for our method would be visualizations that involve cluttered, complex graphs. One important reason for such application scenarios is that VR usually allows to perceive and analyze larger graphs compared to non-stereo setups [78]. Naturally, aspects like layout and interaction [41, 43, 42] are the dominant factors that impact the success of such visualizations. However, as color is often over-represented in these scenarios, we postulate that Deadeye is a valuable alternative to, e.g., highlight the vertices or edges being currently selected by the user. To summarize, we assume that the unique selling point of Deadeye, i.e., the fact that it does not require a dedicated visual dimension, such as color or motion, is of enough value for a number of VR visualizations, which justifies our follow-on research.

0.2 Key Advances of the Follow-On Research

In this section, we briefly outline the key novelties to facilitate comprehension. Especially readers that are familiar with the basic idea of Deadeye [39] will find four key advances in this paper:

Preattentiveness in VR. As a first step, we explore whether Deadeye is still preattentive in a VR setup, despite being considered a subtle cue [88]. Apart from the listed benefits for VR visualizations, having Deadeye in VR would also remove its prior disadvantage of requiring extra stereo hardware for highlighting.

Heterogeneous Distractors. The original studies on Deadeye were executed with one type of objects, i.e., colored circles. And while this is a valid and most commonly used approach for detecting preattentive cues, it is not guaranteed that these cues perform equally well in a heterogeneous setup with varying distractors. As visualizations are rarely limited to homogeneous objects, we need to assure that the method maintains its robustness under such circumstances. Therefore, we extend the initial research scope by studying the performance of Deadeye under different combinations of heterogeneous distractors: color, shape, and 3D depth.

Deadeye and depth perception. As Deadeye removes the object for one eye, our visual system cannot rely on binocular disparity for depth estimation of the target object anymore. We examine whether our visual system is still able to extrapolate the depth based on additional cues, such as occlusion geometry. This is an important detail, as otherwise, Deadeye might limit our understanding of spatial relationships in VR.

Evaluation of real-world applicability. We demonstrate how to integrate Deadeye into complex scientific visualizations in VR using the example of volume rendering (cf. Figure Deadeye Visualization Revisited: Investigation of Preattentiveness and Applicability in Virtual Environments). In particular, we present a workflow for an intuitive application of such a highlighting feature and conduct an exploratory survey with visualization practitioners. Based on the survey outcomes, we discuss the benefits and drawbacks of Deadeye in such scenarios and generate a set of design implications for future research and applications.

1 Preattentive Cues

Before exploring the behavior of Deadeye [39] in a virtual, stereoscopic environment, we briefly introduce the essence of preattentive features and provide a motivation for such research. We will not go into detail regarding the basics of our visual system [55, 32, 83] in order to maintain focus on the visualization-related aspects. We recommend the state-of-the-art summary by Healey and Enns [28] for a broad overview of preattentive research done in past decades.

In short, our visual system is able to detect certain outstanding features such as color in a glance, i.e., before our eyes initiate a saccadic movement. Consider the example in Figure 2: looking at such an image for a split second would suffice to tell whether or not there was a red circle among blue ones. Since a saccade usually needs about 200-250 ms [28] to initiate, researchers utilize that threshold to determine if a cue is preattentive.

Apart from color, prior work has already discovered a large set of other preattentive features, including size, density, lighting direction, flicker, or orientation. For a more detailed overview, we refer to the work by Wolfe and Horowitz [82] and Healey and Enns [28]. At this point, we also emphasize that preattentive cues differ in their underlying nature, and, thus, the performance or detectability usually depends on various factors, such as the type and heterogeneity of distractors, viewing angle, or lighting condition.

A straightforward application for preattentive cues in visualization is to draw and guide attention [77, 23, 7] due to the nearly instant recognition time. Here, an even more important advantage is that preattentive features perform equally well, no matter how many other objects—also called distractors—are present. In other words, we can confirm a red circle from the previous example in less than 250 ms, even if hundreds of blue elements are on the screen.

Our community has made extensive use of preattentive cues for different approaches, such as dynamic narrative visualizations in the case of Attractive Flicker [76] or document representations as done with Popout Prism [69]. Other examples include the shading-effects-based Stylized Focus [11] and the utilization of 3D depth in graph visualizations [3]. Readers seeking further application examples and higher level design guidelines for such visual features might also be interested in the work by Huber and Healey [30]. In addition, studies by Gutwin et al. [22] provide insights into performance differences of preattentive cues in the peripheral sight area. Note that Deadeye also suffers from decreasing performance in areas far from the focus point [39], and, if this aspect is of high importance (e.g., multimonitor setups), other cues such as motion or flickering should be favored instead.

Although we can spot a single cue instantly, searching for a combination of multiple features often results in a serial and no longer parallel process [74, 72, 73, 80, 71, 50]. A detailed discussion of that so-called conjunction search is outside the scope of this paper, yet it is important to know about this limitation of preattentive cues when designing complex visualizations. This rule has a few exceptions, i.e., conjunction search setups where parallel processing can be achieved. Prominent examples are the works by Müller and Muhlenen [52] (form and motion) and Nakayama Silverman [54] (3D depth and color/motion). That latter report was the main motivation for studying conjunction search abilities of Deadeye, since 3D depth and Deadeye both rely on binocular disparities. However, the experiments did not support the hypothesis that Deadeye is suitable for parallel processing when combined with color. Hence, we are not further examining conjunction search in our VR setup, as we do not see any evidence for a change in behavior in this regard.

To fully understand the underlying mechanisms behind conjunction search and preattentive processing in general, we recommend starting with the Integration Theory by Treisman [74]. Other fundamental research regarding preattentive models includes the Texton Theory by Julesz et al. [36] and the Boolean map theory by Huang and Pashler [29]. As we will not go into detailed explanations of these models, the state-of-the-art paper by Healey and Enns [28] might be considered as a starting point for obtaining an overview.

2 Underlying Mechanics of Deadeye

The concept behind the original Deadeye visualization technique is to highlight an object by removing it for one eye only. We refer to such a principle when each eye is exposed to a different stimulus as dichoptic presentation. In general, that difference in stimuli leads to binocular rivalry [47, 6, 19, 2, 59], i.e., our vision system enters a context-switching mode that allows us to perceive both monocular images alternately instead of experiencing a superimposition.

Whether or not binocular rivalry can be perceived in a preattentive manner has been discussed in a number of prior works. Wolfe and Franzel [81] assumed that such a cue is not preattentive in general, with the exception of the so-called luster effect where the target object is dimmer than the background for one eye and brighter for the other eye. Such luminance variations and luminance disparities in general were subjects of further extensive research [13, 70, 4, 18]. Zou et al. [88] reported that although dichoptic presentation can be preattentive, it is usually too weak and overridden by more pronounced features such as orientation. Consequently, dichoptic presentation has only rarely been employed in visualization or for highlighting purposes in general [85, 84].

On the other hand, research by Paffen et al. [58] and especially the work by Zhaoping [86] has provided further evidence that binocular rivalry should be reconsidered as a preattentive cue. In particular, Zhaoping confirmed that ocular discontinuities can be used for drawing attention by comparing ocular singletons to orientation singletons and evaluating the role of the primary visual cortex during the construction of related bottom-up saliency maps. The original studies on Deadeye also align with these findings and confirm the achievable preattentiveness.

A well-known case of dichoptic presentation is the binocular disparity generated by the horizontal offset of our eyes. That information is processed by our visual system to gather depth information and forms the basis for stereo vision [34, 35, 9, 49, 48]. On a side note, a number of related stereo cues such as lighting direction and three-dimensionality have also been proven to be preattentive [15, 16, 57].

In contrast to the disparity-based depth perception,  da Vinci stereopsis [53] provides depth information based on monocular vision. This phenomenon is common in daily life, i.e., when an object is partially occluded by another, our vision system perceives monocularly occluded regions as depicted in Figure 2. Although receiving little attention by the general public, monocular occlusion plays an important role in our depth perception process [21, 46, 66, 67, 25]. Consequently, prior research [48, 8, 27, 87, 79] established several models that show how such unpaired image points contribute to depth estimation.

Regarding Deadeye, one important aspect of da Vinci stereopsis is the classification of monocular areas in valid and invalid combinations as shown in Figure 2. Clearly, only valid cases appear in nature. As shown by Shimojo and Nakayama [66], such valid regions are not subject to binocular rivalry and usually appear as part of more distant surfaces, whereas invalid regions appear more ambiguous in depth estimation. In the case of Deadeye, the exposed image pair falls into the invalid category. Hence, at first glance, depth estimation for the highlighted object might be error prone. However, more recent research [31, 5, 20] has revealed that da Vinci stereopsis works in a rather stimulus-dependent way, i.e., the quantitative depth computation approach depends on the given occlusion configuration. Tsirlin et al. [75] concluded that occlusion geometry is most likely the main source for depth extraction in such cases. These recent findings allow us to assume that Deadeye-enhanced objects would also perform well regarding depth estimation in VR setups, especially when we consider that such a stereoscopic environment allows us to observe the object in question from multiple perspectives to achieve a more sophisticated depth impression [68].

3 Preattentiveness of Deadeye in VR

We conducted an evaluation to understand whether and how the preattentiveness of Deadeye behaves in a virtual environment. In a first step, we recreated an experiment similar to the original Deadeyes tudy [39] to generate a set of comparable data. The design of our experiment follows the traditional approach for preattentive cues: series of images—or scenes in our case—are displayed for a short amount of time (100-250 ms), and participants have to decide for each image whether a highlighted object is present or not. If a cue is preattentive, the high success rate (typically $>80$%) is maintained independently from the number of overall objects on the screen. Hence, preattentive experiments are performed with a varying number of distractors to verify the performance stability.

Accordingly, we formulated a first hypothesis H1: Stereoscopic environments do not impact the accuracy of Deadeye in the case of homogeneous distractors. Note that although we utilized stereoscopic equipment in the original experiments to hide the target object for one eye, the image pairs were otherwise identical, i.e., no disparity-based depth cues were present. In contrast, the experiments to be presented utilize a 3D scene that allows spatial vision and where all objects are three-dimensional, as shown in Figure 3.

In a second step, we significantly extended the original setup by utilizing objects that vary in protruding properties, such as color or shape, to explore the robustness of our technique. Note that preattentive cues perform differently under heterogeneous conditions, and understanding such behavior is crucial because visualizations seldom consist of only one type of objects. Since prior research [88] has demonstrated that dichoptic presentation is a rather weak cue that could be easily overridden by stronger features, we regard an evaluation under heterogeneous conditions as an important milestone for establishing Deadeye as a “working” cue for real-world visualizations. Hence, our second hypothesis is that Deadeye accuracy remains robust in scenarios where distractors and the targets have heterogeneous visual properties (H2).

3.1 Procedure and Applied Measures

Our experiments took place in a virtual reality lab. In our recruitment call, we requested normal or corrected to normal visual acuity and no defects of vision as a study prerequisite. Upon entrance, we informed the participants about the overall procedure and administered a questionnaire to obtain general demographic data.

We then provided the participants an Oculus Go [17] HMD with a per eye resolution of 1280 x 1440 pixel and briefly explained its usage. The remainder of the study took place in the VR environment, i.e., all trials, briefings, pauses, and questionnaires were done via the HMD to guarantee equal conditions (e.g., between-run delays) for all participants.

On-screen text briefed the participants that they would see a series of static 3D scenes consisting of floating objects, and one of the objects might pop out. Each scene was shown for a split second, and, subsequently, participants would vote via yes and no buttons whether they thought that a highlighted object was present. Before each scene, participants saw a crosshair in the middle of the screen and were asked to maintain focus on it. This detail is important to prevent saccadic eye movements such that the eyes would remain in the focus stage when a scene becomes visible. The briefing also mentioned that there would be a training stage before each round where participants would receive audio feedback that indicated the correctness of the given answer. During the real run, the audio would be replaced by a neutral sound to prevent distractions, i.e., reflecting upon wrong answers and lack of concentration as a result.

The entire study consisted of two parts: Exp-1 with homogeneous distractors and Exp-2 focusing on heterogeneous distractors. Exp-1 consisted of three sets that differed only in the number of displayed objects: 4, 16, and 30. We utilized randomly generated cubes on a 5 x 6 grid with jittering/offset functions as depicted in Figure 3. All cubes were aligned on the same depth plane, resulting in a horizontal viewing angle of $14.93^{\circ}$ from the focus point (vertical: $12.23^{\circ}$). Each cube had a size of approximately $1.8^{\circ}$. Each of the three sets included 48 scenes, half of them with a target object at a random position in a randomized order. We exposed each scene for 250 ms to the participants; the previously displayed crosshair lasted for 2500 ms. Furthermore, each set began with a training round with 20 scenes. To summarize Exp-1, we replicated our setup from the previous study as precisely as possible to generate statistically comparable data. The only altered condition was the shift from a 2D image with flat objects to a scene with three-dimensional objects with binocular disparity.

Exp-2 included the three sets depth2, depth2-color-shape, and depth3-color in random order. All three sets consisted of 30 objects and varied only in the type and/or depth distribution of utilized elements. All other conditions were similar to Exp-1 to allow direct comparisons. For depth2, we chose the same objects as in Exp-1, but distributed them on two distinct depth planes as shown in Figure 3 and also included a minor depth jittering function. The distance between the depth planes was twice the cube-side length. The maximum partial occlusion of a far-plane object by a closer one was 10 % of its screen space, i.e., at least 90% of the far object remained visible. For depth2-color-shape, we included different forms of objects and also randomly assigned different colors to see how Deadeye performs in the presence of two rather dominant visual cues. In addition, we kept the depth distribution over two planes. The scenes of depth3-color further increased the depth variance by adding a third plane and increasing the maximum possible occlusion of the furthermost object to 20 %. The objects were all cubes with randomly assigned color.

Between all sets of our study, we displayed a stereoscopic 3D landscape photo for 30 seconds to provide short pauses and reduce task fatigue, similar to the original Deadeye experiments. We also re-implemented the original questionnaire in VR, as shown in Figure 5, and administered it after Exp-1 and after Exp-2. The questionnaire is based on the NASA-TLX survey [26], which includes six subscales: mental demand (low/high), physical demand (low/high), temporal demand (low/high), performance (good/poor), effort (low/high), and frustration level (low/high). Each scale ranges from 0 to 100 in increments of 5. Our questionnaire also contains two additional items on a seven-point Likert scale ranging from 0 to 6, with larger numbers indicating a more positive outcome: Clearness: how well have you perceived the highlighted object? and Decision-making: how sure were you that you made the right decisions?

3.2 Results

The quantitative study included twenty-four persons (fifteen females, nine males), aged 18 to 43 ($M=26.71,\,SD=6.82$). All reported normal or corrected to normal visual acuity and no defects of vision. The presented results are based on the automated logging of our VR application. All variables were normally distributed according to Kolmogorov-Smirnov tests.

The average accuracies, as depicted in Figure 4, for Exp-1 (4 objects: $M=0.93,\,SD=0.08$; 16 objects: $M=0.91,\,SD=0.09$; 30 objects: $M=0.91,\,SD=0.07$) and Exp-2 (depth2: $M=0.93,\,SD=0.08$; depth2-color-shape: $M=0.91,\,SD=0.06$; depth3-color: $M=0.90,\,SD=0.06$) are similar to the values of other preattentive cues. We applied a repeated measures ANOVA with the set type as a within-subject variable to investigate whether the number and kind of distractors influence the accuracy of Deadeye in VR. The result, $F(5,115)=1.04,\,p=.397$, shows no significant difference. In other words, the performance of the participants was affected by neither increasing the number of cubes nor adding depth layers and modifying the shape and color of displayed objects. Regarding the false negatives to false positives ratio in case of wrong answers as depicted in Figure 4, it is more likely to miss a target than to report a false alarm, which is also a common behavior of other preattentive cues.

We also performed independent samples t-tests to explore whether the transition to VR had an impact on performance. To enable such comparisons, our sets in Exp-1 replicated the prior 2D setup [39] as close as possible. The results for 4 ($t(43)=-1.97,p=.055$), 16 ($t(42.18)=-0.02,p=.988$), and 30 objects ($t(43)=-0.85,p=.402$) show no significant differences in accuracy between 2D and 3D experiments.

To evaluate the subjective perception of Deadeye, we conducted a paired-samples t-test to compare the outcomes of the NASA-TLX questionnaire for Exp-1 and Exp-2, as shown in Figure 6. Physical demand ($t(23)=-2.22,p=.037$) and temporal demand ($t(23)=2.72,p=.012$) are significantly different. Physical demand was reported to be significantly higher ($M=39.58,\,SD=18.65$ vs $M=26.88,\,SD=23.35$) in Exp-2, whereas temporal demand was significantly higher in Exp-1 ($M=48.33,\,SD=20.62$ vs $M=38.13,\,SD=24.44$). Furthermore, we applied independent samples t-tests to compare the NASA-TLX results of Exp-1 with the prior 2D experiment and found no significant differences between any of the subscales.

In a similar way, we compared our custom questions regarding clearness (Exp-1: $M=3.83,\,SD=1.90$; Exp-2: $M=3.54,\,SD=1.35$) and decision-making (Exp-1: $M=3.58,\,SD=1.44$; Exp-2: $M=3.00,\,SD=1.29$) as can be seen in Figure 7. We found no significant differences between Exp-1 and Exp-2, or between Exp-1 and our prior 2D results.

3.3 Discussion

Our results support our assumption that a virtual environment does not limit the preattentive nature of Deadeye, because the feature is still recognized with an average accuracy of $\sim$ 90%. In particular, the outcomes of our independent samples t-tests for the sets of Exp-1 and the respective sets of the prior 2D experiment support our hypothesis H1, i.e., the transition to fully stereoscopic environments does not impact the behavior of Deadeye in the case of homogeneous distractors.

The behavior regarding wrong answers also remains unchanged: about $\sim$ 65% of the errors were false negatives, which is quite common for preattentive cues. In other words, it is easier to overlook a highlighted object rather than it is to mistakenly call out a false alarm in the absence of a highlight.

In our opinion, the most important finding of our experiment is that the performance of Deadeye was not impacted by heterogeneous distractors, because our evaluation did not reveal any significant differences in performance and fully supported our second hypothesis H2. We intentionally picked strong visual attributes for our distractors, such as shape and color, because previous work suggested that the preattentive character of binocular rivalry might be easily overridden [88]. However, Deadeye performed surprisingly robustly, which allows us to suggest the technique as a suitable highlighting method for real-world applications. Clearly, an unstable performance under heterogeneous conditions would have significantly limited the potential of Deadeye, because visualizations, in most cases, involve a complex interplay among multiple visual attributes.

Furthermore, we made two rather unexpected observations during Exp-2. First, the performance of the depth3-color set surpassed our expectations. We suspected that target objects on the most distant depth plane would perform significantly worse due to the small screen size and the partial occlusion. However, as shown in Figure 8, a detailed analysis of missed target locations did not reveal any error trend toward a specific depth plane. Second, we assumed that thinner objects, i.e., cylinders in the depth2-color-shape set, might perform worse due to the influence of aliasing (“shimmering”) caused by slight HMD movement. Yet again, analyzing the missed target locations did not reveal any relation between shape and error rate. The only notable connection between the target position and its likeliness to be overlooked is the angular distance from the focus point, i.e., Deadeye performance decreases for objects in the outer columns. At this point, we will not go into detail regarding this finding, because it was already described and analyzed in the original Deadeye paper [39]. Furthermore, as the visual quality of current HMDs is very limited in peripheral areas, this aspect is of low importance compared to non-VR, multi-monitor setups.

Interestingly, a closer inspection of the NASA-TLX questionnaire outcomes does not give us a clear picture regarding the actual exertion induced by Deadeye. Although the average values are positive compared to other visual cues [22], we point our readers at the rather large standard deviations. For each subscale, the answers provided by the participants always contained both extreme values, i.e., 0 and 100. We suppose that such a spread is due to the subtle nature of Deadeye and our inability to put the perceived effect into words. This presumption is also underpinned by our custom questions that reflect an average confidence of participants regarding their made decisions.

As a side note, we attribute the significant differences for physical and mental demand between Exp-1 and Exp-2 to possible sequence effects of our study. Temporal demand decreased over time, because participants got used to the fast pace of the trials. In contrast, physical demand increased with the duration of the experiment, because the HMD is still an uncomfortable device when worn over a longer period of time.

4 Integration into VR Volume Rendering

In addition to rather fundamental studies on preattentiveness, we now consider volume rendering in VR as a real-world application scenario of Deadeye. We present an integration example and an exploratory study to demonstrate the benefits and limitations of our approach.

4.1 Implementation

Given a volume rendering system with VR support (cf. [37, 65]), the technical implementation of Deadeye can be done in the following way. For simplicity, let us assume that Deadeye should be achieved by removing the target for the right eye. To remove the footprint of the target volume regions for the right eye, we store a per-voxel mask volume on the GPU along with the volume data. During GPU-based ray-casting (cf. [40]), we compare this mask value against a global constant in the shader, which whether the rendering is done for the left or right eye. A voxel is skipped if the mask is set and rendering is done for the right eye.

The generation of such a mask volume depends on the application domain. For instance, in the case of already segmented [45] data sets, the per-voxel segment ID of the region of interest can be utilized. Another option is to interactively create and modify the mask volume by “erasing” the target for one eye directly in VR, as done in our study.

4.2 Procedure of the Study

To gather more information about the behavior of Deadeye in real-world-conditions, we conducted a semistructured exploratory study based on medical volume rendering in VR. Our main goal was to determine possible limitations of our preattentive highlighting method and to generate a preliminary set of design guidelines for the application of Deadeye in such scenarios.

Since we were interested in feedback from people who are already familiar with volume rendering, we decided to recruit our participants manually via direct e-mail requests to limit the discard rate. The experiment took place in our VR lab and was supervised by two researchers. One of them was responsible for briefing and interviewing the participant during the experiment. All participants agreed that we would perform an audio recording of the whole session to facilitate the final evaluation. The second researcher monitored the volume rendering application and executed certain requests from participants, such as resetting the highlighted regions or switching between data sets. We intentionally reduced the direct interactions of the participants with the software to a minimum to focus on the highlighting aspect.

After assessing the demographic data and conducting a hole-in-the-card-test for eye dominance (Dolman method, e.g., [10, 60]), we equipped our participants with an HTC Vive Pro [12] with a wireless adapter and explained the remainder of the study. For volume rendering, we used a VR-optimized, GPU-based ray caster implementation that guaranteed a minimum refresh rate of 90 frames per second. We used two data sets to simulate common medical situations, a CT scan of a human head (+neck/shoulders), and a CT scan of a human torso; both are subsets from the frozen CT visible human data set [1]. In the first part of the study, participants explored both data sets with a medical greyscale transfer function. For the second part, we applied a transfer function that included color to explore its interplay with Deadeye. Each of these four scenarios (2x greyscale, 2x colored; cf. Figure 9) consisted of the following tasks:

• •

Detection - we displayed two different presets (scale, orientation, clip plane); each preset included between one and four Deadeye-enhanced regions. Assignment: Tell whether and how many parts of the data set pop out. Describe where the highlighted elements lie in depth relative to their surrounding.

• •

Application - participants were able to apply Deadeye on their own using the trigger button of the controller similar to 3D painting. Assignment: Pick two regions of interest and highlight them as precisely as possible.

• •

Comparison - we displayed two presets that utilized alternative popout techniques, color and flickering, to highlight between one and four regions. In the case of colored scenarios, only flickering was applied. Assignment: Locate the highlighted regions. Describe your perceived benefits and drawbacks compared to Deadeye.

During each assignment, we explicitly asked the participants to provide feedback in a think-aloud manner if possible. We did not impose any time limit for the scenarios and encouraged brief pauses between them. Upon completion of the experiment, we offered the opportunity to provide any additional feedback if desired.

4.3 Findings

The exploratory study included nine participants (three females, six males), aged 29 to 45 ($M=36.67,\,SD=5.68$) participated in the exploratory study. All had prior experience with volume rendering, and six of the nine participants had used VR HMDs before. The majority ($N=7$) had a right dominant eye, and all participants reported normal or corrected to normal visual acuity. In the remainder of this section, we group the obtained results by the respective task to provide a structured exposition.

Detection. In all four scenarios, participants delivered solid performances in spotting and pointing at highlighted regions via a selection ray. Only two participants made a false negative error each. Both participants missed the same small, thin region located rather deep within the data set. Four participants told us that they were able to spot target regions easier when the data sets were colored (P3). All participants were able to describe the relative depth of the highlighted objects relative to the surrounding elements. Four participants stated that they had to move the head a little bit to view the object from different angles to be 100 % sure (P7).

We received differing comments regarding the perception of the highlighted object. All but one participant utilized the adjective transparent in certain variations, e.g., pseudo-transparent (P1) or semitransparent (P6). All participants stated that they could see the target as well as the objects behind it (P3). Seven participants emphasized that they seemed to be able to decide whether they want to “see” the highlighted region or suppress it by focusing on the background objects (P4). Note that this observation aligns with general binocular rivalry behavior where we perceive both monocular images alternately instead of seeing a superimposition. Two participants were particularly intrigued by that behavior, describing it as a kind of willpower-dependent rendering where one could mentally alter the perceived visualization similar to ambiguous image puzzles (P8). Three participants also noticed that one could modify the transparency level of the highlighted object by rotating the head (P9). This phenomenon is caused by the rather limited field of view of the HMD, i.e., when we look at an object from a sufficiently large angle, that area is no more present for the eye that is further away. Hence, binocular rivalry becomes disabled, and we perceive a valid monocular image from the nearer eye that either contains the object or not.

At the end of the detection assignment, we also varied the eye for which the object would be removed. However, participants were indecisive, and only three of them expressed a weak tendency toward the object removal for their dominant eye. Other participants stated that they felt a slight difference, but cannot say which option is better (P4).

Application. All participants were able to manually apply Deadeye onto self-chosen regions of interest. However, the majority ($N=8$) perceived the task as rather challenging especially when moving the controller in the depth dimension (P2). One participant compared it to a VR painting tool: It feels the same as 3D painting in VR, which was not very intuitive to me, either (P5). Seven participants told us that they intuitively closed one eye from time to time to verify the result and speed up the highlighting because it feels more controlled that way (P3). No participants attested to any negative feelings about that one-eye trick. As expected, all participants stated that applying Deadeye is easier for colored data sets because one could just pick a region of a specific color which already has a clear visual separation from the surroundings (P1).

Comparison. In the case of greyscale scenarios, all participants preferred color over Deadeye for two reasons. First, color as highlight is more prominent, straightforward, and already known (P6). Second, as pointed out by six participants, color, in contrast to Deadeye, is not binary. One could utilize a meaningful color scheme for highlighting different regions and even have gradients and interpolations if needed (P9). Deadeye instead is either active or not, without any possibility of providing an additional meaning or more granular differentiation (P7). However, color is not applicable as a preattentive highlighting method in the case of colored data sets, which is an important limitation compared to Deadeye.

When comparing Deadeye to flickering as a highlighting method, the majority ($N=8$) preferred Deadeye; one participant remained indecisive. The most prominent reason for that preference was that flickering is often affected by aliasing and other artifacts (P2) that occur because such a VR experience is never a static scene due to the permanent view changes. Hence, the preattentiveness of temporal effects such as animation or flickering can be rather weakened in such setups, whereas the performance of Deadeye remains stable.

4.4 Discussion and Design Implications

The results of both studies support our vision that Deadeye is applicable as a preattentive visualization technique in real-world scenarios. Even in complex visualizations such as volume-rendered CT scans, we can still recognize the highlighted region and identify its depth. Although disparity-based depth perception is limited, our vision system is still capable of utilizing less straightforward cues such as occlusion geometry. Furthermore, if in doubt, we can easily refine our depth estimation of a region by slightly moving our head to generate different view angles.

One interesting aspect of Deadeye is that we can look behind the highlighted target by focusing on objects that lie behind it. This multistable perception—induced by binocular rivalry—can be regarded as a benefit or drawback depending on the use case. On one hand, such conditional perception allows us to gather additional information without any interaction with the underlying system. In particular, we think of scenarios with very limited interaction possibilities due to, e.g., asepsis requirements. In such cases, having the ability to see or fade out certain elements by simply focusing on the region of interest might be a valuable technique. On the other hand, the focus-dependent approach is mentally more demanding compared to alternatives such as semi-transparency or hiding/showing an object via user input. Although the fatigue results of the NASA-TLX questionnaire are quite reasonable, we expect that a longer, active use of the described switching between highlighted target and background might lead to significant exhaustion. As that phenomenon is less related to the highlighting aspect of Deadeye per se, we would rather consider a more fine-grained exploration as possible follow-up research.

Ultimately, the question remains whether one should pick Deadeye or one of the more common preattentive techniques to draw and guide attention in VR visualizations. According to our qualitative evaluation, people prefer color over Deadeye in the case of greyscale scenarios. In our opinion, the most important advantage of color is its nonbinary nature, i.e., we can encode additional information into the highlighted target and differentiate between multiple targets by utilizing diverse colors. However, many visualizations are not greyscale, and applying our technique in such cases has clear advantages over other preattentive approaches: Deadeye does not alter any visual properties such as size or shape; performs well in complex, heterogeneous environments; and is straightforward to implement in stereoscopic setups. Compared to temporal approaches such as flickering, Deadeye does not suffer from typical VR-related issues such as aliasing due to constant movements of the HMD. To summarize, even though color is still superior in greyscale use cases, we emphasize the applicability and usefulness of Deadeye in colored scenarios and consider the technique as a viable addition to our visualization toolbox.

5 Conclusion and Future Work

This work explored the performance of Deadeye in virtual environments. Initially, Deadeye was introduced as a preattentive technique for drawing attention in 2D visualizations by removing the object of interest for one eye only. The major drawback for real-world usage was the requirement for extra stereoscopic equipment for highlighting, without taking any further advantage of such a setup. Hence, as a natural next stage of our research, we have now applied Deadeye in stereoscopic environments to study its interplay with 3D visualizations.

The paper made two contributions. First, the results from our preattentive tests with homogeneous and heterogeneous distractors confirmed that Deadeye behaves preattentively in VR, and, even more important, maintains its robustness in more complex setups with distractors varying in significant visual properties such as depth, color, and shape. Second, we demonstrated how Deadeye can be integrated into VR visualizations using the example of volume rendering. The outcomes of the exploratory study underpinned the highlighting capabilities of Deadeye and confirmed our ability to estimate depth of objects highlighted in this manner. Especially in colored visualizations, Deadeye was perceived as a valuable extension to our visualization toolkit, as the technique does not alter any visual properties, maintains robustness under common issues such as aliasing (“shimmering”), and comes with a straightforward implementation.

For our future work, we are particularly motivated by an observation during the presented volume rendering study: our participants noticed that the highlighted object could be faded in or out depending on focus, i.e., concentrating on the objects behind the target allows the user to suppress the target completely and see the background only, and vice versa. As possible follow-up research, we suggest investigating this multistable perception phenomenon in more detail to show its full potential for scientific visualizations beyond pure highlighting.

Acknowledgements.

We are immensely grateful to Christine Pickett for her comments that greatly improved the manuscript. We would also like to show our gratitude to the anonymous reviewers of the original Deadeye paper and the IEEE VIS community for motivating us to explore Deadeye in virtual reality environments. This research was made possible in part by Award Number R01NR014852 from the NINR - National Institute of Nursing Research.