Perceived spatial presence and body orientation affect the recall of out-of-sight places in an immersive sketching experiment

TL;DR

This study shows that how people sketch maps of familiar places in virtual reality is influenced by their body orientation and perceived spatial presence.

Contribution

The study introduces 'immersive sketching' as a novel VR-based method to investigate spatial memory and body orientation effects.

Findings

Immersive sketching in VR confirms position-dependent recall of distant places.

Sketch map orientation depends on the subject's body orientation in VR.

Alignment with the target direction strengthens spatial recall, while opposite orientation weakens it.

Abstract

The orientation of sketch maps of remote but familiar city squares produced from memory has been shown to depend on the distance and airline direction from the production site to the remembered square (position-dependent recall, Röhrich et al. in PLoS One 9(11): e112793, 2014). Here, we present a virtual reality version of the original experiment and additionally study the role of body orientation. Three main points can be made: First, “immersive sketching” is a novel and useful paradigm in which subjects sketch maps live on paper while being immersed in virtual reality. Second, the original effect of position-dependent recall was confirmed, indicating that the sense of being present at a particular location, even if generated in a virtual environment, suffices to bias the imagery of distant places. Finally, the orientation of the produced sketch maps depended also on the body orientation of the subjects. At each production site, body orientation was controlled by varying the position of the live feed in the virtual environment, such that subjects had to turn towards the prescribed direction. Position-dependent recall is strongest if subjects are aligned with the airline direction to the target and virtually goes away if they turn in the opposite direction. We conclude that the representation of out-of-sight target places depends on both the current airline direction to the target and the body orientation.

Figures

The setup for the experiments consists of a modified rotating chair. Attached to the chair is a small desk for carrying a laptop and a tablet arm desk for drawing the sketch maps. Attached to the tablet arm desk is a camera stand with camera that captures a live video from the sketch paper that is displayed in the virtual environment

The panoramas of sketching locations S1, S2, and S3

Map of Tübingen city center with the two target locations T1 “market square” (green) and T2 “timber market” (magenta) and the sketching locations S1, S2, and S3 (yellow). Green and magenta arrows show route direction to target place T1 (green) or T2 (magenta), respectively. They coincide with the body orientation in the “towards” condition, while the arrow tails mark the body orientation in the “away” condition. The heavy white arrows show the canonical viewing direction of the target locations as reported by Röhrich et al. ([2014](#CR27)). The airline directions are not shown; they mostly coi

Maps and sketch map examples. a–c Target location T1. The green ellipses mark the city hall (“Rathaus”). b Oriented west (city hall up) and c is oriented east (city hall on bottom). d–f Target location T2. The magenta ellipses mark the long axis of the square together with the church and a fleet of stairs in front of it. e Oriented south (church up) while f Oriented east (church right). Note that the paper orientation while drawing is also defined by the scribbling. Map source: [www.openstreetmap.org](http://www.openstreetmap.org)Fig. 5Results of Expt. 1. Sketching and target locations as in F

Deviation of sketch map orientation from the theoretical directions airline to goal (a), route (b), and and canonical view (c). red: towards condition; cyan: away condition. Same data as in Fig. [5](#Fig5), accumulated over all tasks. Radial lines are circular means; histogram bars show frequency data

Results of experiment 2. a Phase 1, immersed before the break. b. Phase 2, in office environment after the break. For explanations, see Fig. [5](#Fig5). Map source: [www.openstreetmap.org](http://www.openstreetmap.org)

Deviation of sketch map orientation from the theoretical directions airline to goal, route to goal, and canonical view. Same data as in Fig. [7](#Fig7), accumulated over all tasks and body orientations. Colors indicate immersion condition. Radial lines are circular means; histogram bars show frequency data

Full text

Introduction

Imagery of distant places is based on the recall of contents from spatial long-term memory into a working memory stage sometimes called a “representational” memory. Even with a fixed set of spatial knowledge available in long-term memory, the imagined representation of a given target place will differ depending on a number of parameters including the subjects’ imagined heading and view point location as well as their actual position and body orientation at the time when imagination takes place. This has been impressively demonstrated in representational neglect (Bisiach & Luzzatti, 1978; Guariglia et al., 2013), a neurological condition in which subjects are able to recall places from the right side of a familiar city square, but not from the left, where “left” and “right” are defined relative to their currently imagined viewing direction. When the imagined view point changes, the recall pattern changes accordingly.

Representational space can be thought of as a local chart of the imagined environment in which the observer takes a fixed position at the center, while remembered objects or elements of the scene are represented at their respective egocentric position (see, for example, Bicanski & Burgess 2020). Alternatively, it can be conceived of as a graph of known views, each taken from a certain view point in the environment, and connected by descriptors of the egocentric movements required to change from one view point to another (Mallot et al., 2020; Mallot, 2024; Röhrich et al., 2014; Schölkopf & Mallot, 1995). While the general predictions made from both models are largely similar, the view-graph approach seems to lend itself more easily to the generation of pictorial imaginations and allows for anisotropies in representational space.

In behavioral studies with normal subjects, the structure of spatial imagery and representational space has been addressed with various versions of the “judgment of relative direction” paradigm (JRD) as well as with the position-dependent recall and production of sketch maps or three-dimensional neighborhood models.

Judgment of relative direction

In the judgment of relative direction (JRD) task, subjects are asked to imagine a known environment from a certain point of view, heading towards some remembered object within this environment. From the imagined heading thus defined, they are then asked to report the relative direction to other remembered objects (Shelton & McNamara, 2001). Recall fidelity depends on the imagined heading direction and is best if it aligns with an intrinsic axis of the imagined environment, in the experiment the long axis of a rectangular room. The intrinsic axis may also be defined by objects placed inside the room. Mou and McNamara (2002) placed several objects in an regular grid and found better JRD performance for mental tasks that involved perspectives that were aligned with one of the two grid axes. On a larger scale, the layout of streets in a city or the corridors in an office building may also provide intrinsic axes of reference (Montello, 1991; Werner & Schmidt, 1999). Reference axes for imagery of spaces on different scales such as an office space within an university campus may coexist and need not be identical (Wang & Brockmole, 2003).

Performance in the JRD task also depends on the subject’s body orientation during recall and is superior if their actual body orientation is aligned with the imagined body orientation or heading (May, 2004; Kelly et al., 2007; Riecke & McNamara, 2017). This “sensorimotor alignment effect” indicates that despite their engagement in imagery and the JRD task, subjects still maintain a sense of their actual bodily position as might be obtained from path integration and spatial updating. This is true even if both the “actual” and the imagined body pose are defined in a virtual environment only (Marchette et al., 2014). The representational space generated by imagery of a remote environment thus combines references derived from remembered cues of the imagined space itself (the intrinsic axis) and directional tracking or spatial updating between the imagined and the actual observer position (sensorimotor alignment; for overview, see Julian et al. 2018, Meilinger & Vosgerau, 2010).

Sketch maps and building tasks

Representational memory is organized along preferred reference axes even if no such axes are explicitly induced by the instruction given to the subject. Basten et al. (2012) have shown that the priming of the recall orientation by imagined travel influences the imagery of distant places. In this study, subjects located on an university campus have been asked to imagine walking in the down-town between two familiar city squares, thereby crossing the target square in one of two possible directions. When subsequently asked to produce a sketch map of the target square, the direction of the previously imagined travel primed the recall orientation in the sense that subjects were more likely to sketch the target square as is would have appeared during the imagined walk. The authors conclude that performing an imagined walk activates the representational memory due to automated spatial updating or mental travel or both.

In the Basten et al. (2012) study, priming is generated by the activation of visual imagery during imagined travel and may result from a visual memory. Alternatively, it may be a result of a spatial updating process in which memories of nearby locations are automatically activated as soon as the observer approaches that target location. Röhrich et al. (2014) therefore asked passers-by in the city center to produce sketch maps of nearby (walking distance) city squares which were out-of-sight from the interview location. Sketch maps were rated for orientation and showed significant variation with interview location. Sketch maps of nearby locations are predominantly aligned with the airline direction from the recall position to the target places as if subjects could look through the intermittent buildings. Subjects interviewed at more distant locations (about 2 km away) produced more homogeneous maps showing a standard or “canonical” view of the target independent of the interview location. Here, we use the term “canonical” in analogy of the canonical views known from object recognition (Bülthoff et al., 1995). The imagery of the distant target square thus depends on two directions, first, the airline direction, i.e., on how the square would look could we see through the intermittent buildings, and second, an intrinsic axis of the target square defining its canonical view.

In addition to these dependencies, the bodily orientation of the subjects might also play a role. This was demonstrated by Meilinger et al. (2016) using a building task instead of freely sketched maps. Subjects were given a set of cards naming popular locations around town and asked to rebuild the configuration. The study confirms the position dependence of the recall orientation and additionally demonstrates an effect of body orientation. Note, however, that the Meilinger et al. (2016) study uses landmark buildings in a larger area, not views of a single city square. The building task was transferred to a virtual environment by Le Vinh et al. (2020), again with the reconstruction of a specific target square as a task. The study confirmed the overall effect of airline direction; body orientation was not addressed.

Large-scale representational memory

Unlike object configurations in a room that can be perceived at a single glance, larger areas such as the down-town area of a city have to be explored in a step-by-step fashion, a distinction that has been discussed as vista and navigational spaces by Montello (1993). For the resulting representation, Meilinger (2008) suggested a network of local reference frames or charts, each with its own intrinsic axis. In a simple JRD experiment, only one such chart will be activated and imagery will be based on this chart’s intrinsic axis and the body orientation of the observer. In position-dependent recall, however, the actual and the imagined environments are usually part of different local charts each with their own intrinsic axis and the relative position of these charts will affect imagery as an additional factor. In the studies discussed above, this relative position is described by the airline direction and the distance to the target. In the present study, we present novel data on the role of body orientation in position-dependent recall.

VR methodology

Except for the Le Vinh et al. (2020) study, all studies on position dependent recall discussed above are based on real-world experiments. With the present paper, we also want to validate the VR technology as a tool allowing better experimental control. After decades of discussion, it is now generally assumed that virtual environments can provide valid experimental results in spatial cognition, both from non-immersive desk-top setups (e.g., Ruddle & Jones, 2001) and with motion tracking and head-mounted goggles (e.g., Avraamides & Kelly, 2008; Kelly et al., 2007; Marchette et al., 2014). While the fidelity of visual input does not seem to play a major role in the perception of presence as defined by Slater et al. (2009), it may still be relevant for navigation or the recognition of particular locations in a virtual environment (Lessels & Ruddle, 2005). This latter aspect is crucial for our study; we therefore use the term “spatial presence” to denote the sense of being at a particular location rather than somewhere else. Spatial presence is one dimension assessed in the presence questionnaire of Sanchez-Vives and Slater (2005). The other dimensions, immersion and the memories built from VR experience, are not addressed in our study. Besides the visual fidelity the quality of the feedback from the subjects’ bodily motion may compromise the validity of VR experimentation, especially in non-immersive setups (Kelly et al., 2018).

One additional problem arising in the transfer of the position-dependent recall paradigm in a virtual environment is the task to be performed by the subjects. Le Vinh et al. (2020) used an interactive building task in which a joystick was used to grab building blocks from a reservoir and place them on a work space. Here, we explore a novel approach called immersive sketching, in which the subject uses an ordinary drawing board while still immersed in the virtual environment. The sketching process is recorded with a camera and overlaid to the VR simulation as a live image.

Plan of study

Here, we set out to test the hypothesis that position-dependent recall occurs also in virtual environments and that in this case the relevant “position” is the participants’ simulated position in the virtual environment, not their physical position in the lab. In addition, we access the influence of body orientation. The experiment uses the novel “immersive sketching” paradigm in which participants produce sketch maps of remote places while immersed in a virtual environment. In a second experiment, we addressed the question whether the imaginary view point assumed during the immersion phase persists after leaving the virtual environment or is replaced by a direction corresponding to the participant’s physical position.

Methods

Participants

A total of 100 healthy adults, 37 male and 63 female, participated in our studies. All participants were students of the University of Tübingen aged at least 18 years. Further details were not recorded. All included subjects had been living in Tübingen for more than 2 years and successfully passed a pre-experiment testing their knowledge of the experimental area. Eighty participants were randomly assigned to 4 groups of 20 people each for experiment one and 2 groups of 10 people each for experiment two. Before the study, all the participants confirmed that they had no problem with virtual reality and wearing an Oculus RIFT head-mounted display.

Participants were informed that they could stop and quit the study any time during the experiment without providing any reason. All participants gave written informed consent and received a payment of 8 Euro for their participation.

Setup

Virtual environments were presented with an Oculus RIFT Development Kit 2 (DK2) head-mounted display with a total horizontal field of view of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$100^{\circ }$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$960 \times 1080$$\end{document} pixel resolution for each eye. During the presentation, subjects were seated on a rotating chair, as shown in Fig. 1. The chair has been prepared for the experiments in the following way:

1. An additional small desk for carrying a laptop has been attached behind the backrest. In the experiments, the subjects were able to rotate on the chair without any distraction with the cables of the video equipment.
2. A second desk is attached in the front of the chair similar to a tablet arm desk. For the experiments, a piece of paper is attached on the desk that allows the subjects to draw a sketch map. The paper is fixed with a clip on the desk.
3. A camera stand with a camera is attached to the tablet arm desk. The camera is used to record a live video of the paper fixed on the tablet arm desk. The live video is shown in the Oculus RIFT (using a Unity WebCamTexture) which allows the subjects to draw on the paper while being immersed in the virtual environment. Fig. 1. The setup for the experiments consists of a modified rotating chair. Attached to the chair is a small desk for carrying a laptop and a tablet arm desk for drawing the sketch maps. Attached to the tablet arm desk is a camera stand with camera that captures a live video from the sketch paper that is displayed in the virtual environment

We have used square pieces of paper for the experiments, so that the subject is not biased for drawing horizontally or vertically. As the piece of paper was fixed with a clip to the desk, it was not possible for the subjects to move or rotate the paper while sketching.

Videos for the pre-experiment were captured on site in Tübingen using a Canon Power Shot G7 camera. They were presented in the Oculus RIFT, which in this case was used just as a simple displaying device, i.e., in open loop.

For the main experiments, we have set up a virtual environment in Unity using C#. For each sketching location (S1, S2, S3), we created a 360-degree panorama of the location by taking 12 pictures at 30^∘^ intervals with the Canon Power Shot G7 camera mounted on a tripod. The 12 pictures were stitched to a cylindrical panorama using Microsoft Image Composite Editor. The three panoramas are shown in Fig. 2. All pictures and videos were taken during early morning hours to avoid imaging of passers-by.

The sketching locations were modeled by placing the panoramas as textures in the inside of a virtual cylinder. For the top of the cylinder, we have used matching images of skies, for example blue sunny sky or cloudy sky. For the bottom of the cylinder, we have chosen images of a matching ground, for example cobblestone.

For the immersive sketching procedure, the live picture of the sketching paper fixed on the tablet arm desk is displayed in a square work space (virtual sketching board) appearing in the bottom part of the cylinder. This allows the subjects to draw a sketch map on the paper without taking off the goggles. The texture of this square is set to Unity’s WebCamTexture that takes and displays the camera’s live image on the virtual sketching board instead of a static texture or plain color. The participants thus see their drawing hand and the produced sketch while still immersed in the VE. The work space for sketching is placed either towards or away from the target locations, such that subjects have to produce their sketches while bodily oriented towards or away from the target square.

The subject’s point of view in the experiments was placed in the center of the cylinder. The Oculus RIFT allowed for accurate tracking of the movement of the subject’s head, so that the subjects could visually explore the virtual environment. We have only tracked the rotation of the head; translational movement produced with the upper body while seated was not fed back to the VR simulation, i.e., if the subjects did produce translational movements, the cylinder would virtually move with them and the image remained unchanged.

During the experiment, the experimenter could switch between the three sketching locations using the space bar of a laptop computer, thereby “teleporting” the subjects from one location to another.Fig. 2. The panoramas of sketching locations S1, S2, and S3

Task

In all experiments, we used three sketching locations (S1, S2, S3) and two target locations (T1: market square, T2: timber market); see Figs. 2 and 3. All locations were well-known places in the historic center of Tübingen. All locations were in walking distance of each other but mutually out-of-sight.

Each of the two target locations was sketched from two different directions provided by the two closest sketching locations. This results in the four sketching tasks S1-T1 (at S1, sketch T1), S2-T1, S2-T2, and S3-T2; see Fig. 3. In addition, we varied the subjects’ body orientation by placing the live image of the sketching paper either towards the street leading from the sketching location directly to the target (“towards condition”) or offset from this direction by 180^∘^ (“away condition”). In the “towards” condition, subjects might therefore imagine to just move forward to walk to the target. The “exit direction” is generally similar to the airline direction, but slight deviations may occur due to the city street raster. In the “away” condition, subjects were required to produce their sketches while imagining a target located behind them.

In total, these variations result in eight different tasks. To avoid interactions between repeated sketching tasks, each subject produced two sketches only.

Data analysis

The maps have been categorized by two raters according to the cardinal directions north, east, south, and west and intercardinal directions northeast, southeast, southwest, and northwest, i.e., for one of eight nominal categories. For this task, the two raters could use any aid necessary, such as their own local knowledge of the area, pictures, or maps, as well as the orientation of lettering included in the maps. Maps were not rated for geometric quality, completeness, or any other parameter. The raters worked independently from each other and both raters rated all sketch maps. Cohen’s \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\kappa $$\end{document} (Cohen, 1960) was used to assess inter-rater reliability.

Directional ratings were analyzed in two ways, first as categorical data using \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} tests, and, in a follow-up analysis, with the V-test of circular statistics. Circular statistics transforms angular data ( \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha _i$$\end{document} ) into unit vectors of the form \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\vec {x}_i = (\cos \alpha _i, \sin \alpha _i)$$\end{document} ; see for example Batschelet (1981) and Berens (2009). The cardinal directions N, E, S, W occurring in our data would thus translate to the vectors (in mathematical angle convention) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N = (0,1)$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$E = (1,0)$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S=(0,-1)$$\end{document} , and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$W=(-1,0)$$\end{document} . The average of a set of unit vectors, also called the resultant vector, is given by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\vec {r} = (\sum _{i=1}^n \vec {x}_i)/n$$\end{document} ; its length \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$||\vec {r}||$$\end{document} is close to zero if the individual \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\vec {x}_i$$\end{document} point into different directions and approaches one if the sample is uniform. For a given “theoretical” direction \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\vec {x}_o = (\cos \alpha _o, \sin \alpha _o)$$\end{document} the V-test tests the hypothesis that the sample is drawn from a population not concentrated in the theoretical direction, i.e., that data cluster around another direction or do not cluster at all. It rejects the null hypothesis if the data are clustered around the theoretical direction.

Procedure

Pre-experiment

Before the main experiment started, we performed a pre-experiment to test whether the subjects had a good knowledge of the Tübingen city center and were familiar with the locations that we used in our experiments. Subjects were shown three videos exploring the sketching locations S1, S2, S3. For this, the video goggles were used in open loop. Each video started at a small distance from the sketching location and showed a walk through the immediate environment including views into streets connecting to the target locations. However, the target locations themselves were not visible in any of the three videos. After watching the video, the subject answered the questions: “Do you know this place? Do you know what this place is called? When was the last time you visited this area? Why did you visit this area?”

Three subjects who failed to name all three sketching locations correctly or reported to have last visited them more than 30 days ago were excluded from the main experiment.

Experiment 1

After successfully passing the pre-experiment, the experimenter explained the experiment to the subject by reading a note about the procedure of the experiment. The subjects received this note also in writing. The experimenter pointed out the equipment including the rotating chair, the camera, and the fixed square paper. They clarified that the subjects were allowed to rotate on the chair during the experiment whenever necessary and that they would be asked to draw sketch maps of the Market Square (“Marktplatz”, target T1) and the Timber market (target T2) on the paper while still wearing the goggles. Subjects were also informed about a relaxation room which they might want to use should they feel sick during the virtual reality experiment.

At the beginning of each sketching task, subjects were teleported to the respective sketching location facing north. They would then explore the environment by looking around (closed loop VR simulation for rotations only) until they indicated that they had made themselves familiar with the location. Exploration duration was not recorded. The experimenter then asked the participants to imagine the target location and produce a sketch on the virtual sketching board. To do this, they needed to turn into the direction in which the virtual sketching board had been provided. Beyond the sketching board, they would still see the scenery of the current sketching location, facing either towards or away from the target.

Each subject completed two sketching tasks, one for each target location. They were randomly assigned to one of four groups of 20 participants, called A-towards, A-away, B-towards, and B-away. The tasks for each group are listed in Table 1. After finishing the first sketching task, the second was started without delay.Table 1. Design of experiment 1groupA-towardsA-awayB-towardsB-away1st sketchS2T1 towardsS2T1 awayS1T1 towardsS1T1 away2nd sketchS3T2 towardsS3T2 awayS2T2 towardsS2T2 awayFor details see text

Fig. 3. Map of Tübingen city center with the two target locations T1 “market square” (green) and T2 “timber market” (magenta) and the sketching locations S1, S2, and S3 (yellow). Green and magenta arrows show route direction to target place T1 (green) or T2 (magenta), respectively. They coincide with the body orientation in the “towards” condition, while the arrow tails mark the body orientation in the “away” condition. The heavy white arrows show the canonical viewing direction of the target locations as reported by Röhrich et al. (2014). The airline directions are not shown; they mostly coincide with the route directions except for task S3T2 where it is offset by some 30^∘^ to the right. Map source: www.openstreetmap.org

Experiment 2

To understand the effect of immersion and spatial presence on the representational memory, we conducted a second experiment in which sketching was repeated after a delay period and outside the virtual environment. We have tested 20 participants (sex not recorded) who were assigned to two groups C-towards and C-away with 10 subjects per group.

Experiment 2 consisted of two phases: Phase 1 for groups C-towards and C-away is identical to experiment 1 of groups A-towards and A-away, respectively, i.e., subjects perform sketching tasks S2-T1 and S3-T2. After the subjects have finished sketching and the experimenter collected the sketch papers, the subjects put away the video goggles and took a 10 min break during which they are were asked to stay in the experiment room. They were offered refreshments or might have a look at books available in the room which were irrelevant to the study. However, they were not allowed to browse in the internet or talk on the phone. After the 10 min break, phase 2 started in which the subjects were asked to draw the layouts of the same target locations as in phase 1 again, but this time without being immersed in the virtual environment nor with any additional information from the experimenter. Finishing the drawing of the target locations concluded the second phase of the experiment and the experiment itself.

In total, each subject produced four sketch maps: two maps of locations T1 and T2 before the break and within the virtual reality environment, and again two maps of the two target locations after the break.Fig. 4. Maps and sketch map examples. a–c Target location T1. The green ellipses mark the city hall (“Rathaus”). b Oriented west (city hall up) and c is oriented east (city hall on bottom). d–f Target location T2. The magenta ellipses mark the long axis of the square together with the church and a fleet of stairs in front of it. e Oriented south (church up) while f Oriented east (church right). Note that the paper orientation while drawing is also defined by the scribbling. Map source: www.openstreetmap.orgFig. 5. Results of Expt. 1. Sketching and target locations as in Fig. 3. The circular histograms show the frequency of sketch map orientations for the four tasks S1T1, S2T1, S2T2, S3T3 as indicated also by the location markers touching each panel. Red and cyan bars indicate towards and away condition; north is upwards. The red and cyan lines in the center of each histogram show the circular means. Each histogram shows data from 20 sketch maps. Map source: www.openstreetmap.org

Results

Experiment 1

Figure 4 shows four examples of the produced sketch maps together with their orientational ratings. Inter-rater agreement was 100 %, resulting in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\kappa = 1$$\end{document} (Cohen’s \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\kappa $$\end{document} , see Methods section). All map orientations are listed in the Appendix. Figure 5 shows the distribution of the orientational ratings for sketch maps produced at the three sketching locations S1, S2, and S3, both for the towards and away conditions. For the statistical analyses, the orientation ratings were binned into four classes north, east, south, and west. The rare intercardinal ratings were counted as 0.5 for each of the two adjacent cardinal directions.Table 2. Position-dependent recall (Expt. 1)Target T1Target T2towardsS1T1 towards vs. S2T1 towardsS2T2 towards vs. S3T2 towards \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2(3,N=40) = 36.19$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p <.001$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V=.95$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2(3,N=40) = 38.05$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p <.001$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V=.98$$\end{document} awayS1T1 away vs. S2T1 awayS2T2 away vs. S3T2 away \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2(3,N=40) = 4.56$$\end{document} , n.s \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2(3,N=40) = 1.757$$\end{document} , n.sThe table shows test statistics for the data appearing in Fig. 5Note that all comparisons are between different subject groups as specified in Table 1V is Cramér’s V indicating effect strength

Sketch map orientation in the towards condition for both target locations depends on the sketching location (position-dependent recall). This was revealed by two separate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} tests comparing the orientation histograms for the two sketching locations used with each target location. Note that for each target location, the comparison is between different participant groups, as shown in Table 1. The effect is highly significant for both target locations as can be seen in the first row of Table 2. The effect is absent in the away condition.

Sketch map orientations also depend on body orientation. This is apparent from comparing the towards and away condition for individual tasks (pairs of sketching and target location). Again, all \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} -tests are between different participant groups as specified in Table 1. Test statistics for the effect of body orientation appears in Table 3.Table 3. Effect of body orientation (Expt. 1)Target T1Target T2S1S1T1 towards vs. S1T1 away– \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2(3,N=40) = 22.21$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p <.001$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V=.75$$\end{document} S2S2T1 towards vs. S2T1 awayS2T2 towards vs. S2T2 away \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2(3,N=40) = 17.80$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p <.001$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V =.67$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2(3,N=40) = 25.31$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p <.001$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V =.80$$\end{document} S3–S3T2 towards vs. S3T2 away \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2(3,40) = 13.32$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p =.002$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V =.58$$\end{document} The table shows test statistics for the data appearing in Fig. 5Note that all comparisons are between different subject groups as specified in Table 1V is Cramér’s V indicating effect strengthS1–S3 sketching locations

In summary, sketch map orientation is depended (at least) on two factors: sketching location and body orientation.Fig. 6. Deviation of sketch map orientation from the theoretical directions airline to goal (a), route (b), and and canonical view (c). red: towards condition; cyan: away condition. Same data as in Fig. 5, accumulated over all tasks. Radial lines are circular means; histogram bars show frequency data

A further analysis of the data concerns the preferred orientations of the produced sketch maps in relation to three theoretical angles, (i) the airline direction from the sketching location to the target, (ii) the route direction heading into the street that most quickly leads from the sketching location to the target (green and magenta arrows in Fig. 3), and (iii) the canonical view of the target as indicated by the white arrows in Fig. 3. Note that the route direction (ii) is also the body orientation of the subjects in the towards condition. Figure 6 shows the differences of the sketch map orientation and each of the three theoretical angles for all sketching tasks. Thus, if subjects would exactly align their maps with the airline direction, Fig. 6a would show an ideal peak at angular difference 0. Significant alignment with the theoretical angles was tested with the circular V-test (e.g., Berens, 2009). Results show significant effects of all three theoretical angles in the towards condition (airline: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V(80)=64.4, p<.001$$\end{document} ; route: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V(80)=60.8, p<.001$$\end{document} , canonical: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V(80)=27.5, p<.001$$\end{document} ), while in the away condition, significant map alignment was found only with the canonical viewing direction ( \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V(80)=24.0, p<.001$$\end{document} ). No distinction can be made between the airline and route direction, due to the close agreement of these two theoretical angles in the used sketching and target locations. Furthermore, all theoretical angles are virtually the same (i.e., west) for the S2-T1 task as can be seen from Fig. 3.

In summary, the results of Experiment 1 show that sketch map orientation depends on sketching location in the towards, but not in the away condition. The preferred sketch map orientation in the towards condition is a mixture between the airline or route directions to the target location and a canonical view of the target location, while the sketch maps produced in the away condition show some alignment with the canonical view, but not with the airline or route directions.Fig. 7. Results of experiment 2. a Phase 1, immersed before the break. b. Phase 2, in office environment after the break. For explanations, see Fig. 5. Map source: www.openstreetmap.org

Experiment 2

Results of experiment 2 appear in Fig. 7. Inter-rater agreement was 100 %, resulting in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\kappa = 1$$\end{document} (Cohen’s \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\kappa $$\end{document} , see Methods section). All map orientations are listed in the Appendix. Participants from the C-towards and C-away groups produced sketches of tasks S2T1 and S3T2 before and after the break, i.e., within and outside the virtual environment. In the “towards” and “away” groups, the number of sketch pairs produced with different orientations before and after the break was 12/20 and 8/20, respectively (see data table for experiment 2 in the Appendix). We therefore conjectured that subjects do not simply reproduce their first sketch in the second drawing and tested this hypothesis with two approaches.

First, we analyzed the orientation changes of sketch maps produced by each subject before and after the break using Hotelling’s \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T^2$$\end{document} -test. For this, the angular deviations between sketches produced before and after the break were transformed into unit vectors with components given by the cosine and sine of the angular differences. This generated a set of bivariate data which were compared to the value \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\cos 0, \sin 0) = (1,0)$$\end{document} expected under the null hypothesis of equal map orientation. The test did not reach significance for any of the tasks or body orientation conditions (S2T1 towards: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T^2(2,8)=4.47$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p=0.20$$\end{document} ; S2T1-away: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T^2(2,8) = 4.10$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p=0.22$$\end{document} ; S3T2-towards: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T^2(2,8) = 5.11$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p= 0.17$$\end{document} ; S3T2-away: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T^2(2,8) = 2.70$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p=0.35$$\end{document} ).

Second, we conjectured that if subjects would simply reproduce their sketches drawn during immersion also after the break, the effect of body orientation should be identical in the “before” and “after” conditions. In this approach, comparisons are made between the two subject groups C-towards and C-away, and are therefore again be analyzed by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} tests. Data were collapsed for sketching task but kept separate for immersion. The “north” bin in task S2T1 was empty for both body orientations in the immersed condition and was therefore deleted. A significant effect of body orientation was found for the sketch maps produced during immersion (before the break, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2(6,40) = 17.5$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p =.0051$$\end{document} , \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V= 0.66$$\end{document} ), but not after the break ( \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2(7,40) = 3.13$$\end{document} , n.s.). Indeed, after the break, when subjects are tested in the lab environment, the definition of the “towards” and “away” body orientations is no longer apparent.Fig. 8. Deviation of sketch map orientation from the theoretical directions airline to goal, route to goal, and canonical view. Same data as in Fig. 7, accumulated over all tasks and body orientations. Colors indicate immersion condition. Radial lines are circular means; histogram bars show frequency data

Figure 8 shows the alignment of the chosen view orientations with the theoretical directions airline to goal, route to goal, and canonical view, each with and without immersion in the virtual environment. While immersed in the virtual environment, highly significant alignment was found with all three theoretical directions (airline: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V(40)=30.8, p<.001$$\end{document} ; route: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V(40) = 24.4, p<.001$$\end{document} ; canonical: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V(40)=20.0, p<.001$$\end{document} ). This pattern is unchanged after the break, when subjects repeat their tasks in an office environment, although the significances are weaker (airline: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V(40) =9.6, p=.016$$\end{document} ; route: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V(40)=11.9, p=.004$$\end{document} ; canonical: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V(40)=12.0, p=.004$$\end{document} ).

To sum up, the results of Expt. 2 indicate that the imaginary view point assumed during immersion in the virtual environment partially persists when the participants return to the physical environment of the lab.

Discussion

When subjects produce sketch maps of distant target places from memory, they tend to align their productions with specific viewing directions. These depend on the airline or route direction from the sketching site to the target, the body orientation of the subjects at the sketching location, and a standard or canonical view of the target which is also activated when the distance to the target is large (Röhrich et al., 2014). The canonical view is “allocentric” in that it does not change as the observer moves around, while the airline and route axes are of course dependent on observer position, i.e., egocentric (see Klatzky, 1998). In our experiments, the airline direction was roughly identical with the direction of the shortest route connection to the target; we can therefore not distinguish the effects of these two factors.

The results confirm and extend earlier findings on position-dependent recall in a real urban environment by Röhrich et al. (2014) in two respects: First, the imaginary view point of position-dependent recall can be set by a virtual environment, while the subjects are physically in the lab. Second, position-dependent recall is modulated by body orientation which was not controlled in the Röhrich et al. (2014) study. If the subjects’ body orientation is aligned with the route direction to the target, sketch maps are oriented as if the subjects could look through the intervening buildings right to the target (“towards” cases in Fig. 5). One possible interpretation of this result is that to solve the task subjects perform a mental travel to the target during which they imagine oriented views along the route, and sketch the final view imagined upon arrival at the target (see Basten et al., 2012). In the away condition, sketch maps are preferably aligned with the target’s canonical view (Fig. 6f), i.e., the view produced also at large distances (Röhrich et al., 2014).

The difference between the sketch orientations in the towards and away conditions implies that subjects have at least implicit knowledge of their current body orientation relative to the target. This cannot be a result of path integration (as in the sensorimotor alignment effect of Kelly et al., 2007), since subjects were teleported between sketching locations. Rather, it must be obtained from spatial memory. Memory structures suited to provide this type of information are the view-graph suggested by Schölkopf and Mallot (1995) and Röhrich et al. (2014) as well as the network of reference frames of Meilinger (2008). In both cases, the orientation changes stored for each view or reference frame transition would have to be accumulated along a path connecting the sketching and target locations. Alternatively, local orientation relative to a global reference direction could be memorized for each location (Mallot et al., 2020; Mallot, 2024). Of course, information about the egocentric bearing of the target location from a sketching site would also be available from a complete metric map. In this case, however, position-dependent recall itself would be hard to explain.

In the original study by Röhrich et al. (2014), body orientation was not controlled and varied substantially among subjects. The fact that the strength of position-dependent recall was intermediate between the strong and absent effects found in the towards and away conditions in the present study may thus be a result of averaging over subjects with different body orientations.

One important result of this study is the overall equivalence of the real-world experiments by Röhrich et al. (2014) and our immersive sketching task in a virtual environment. The role of immersion was further addressed in experiment 2 which used the same paradigm as experiment 1 but added a second phase in which subjects were retested outside the virtual environment in an office setting. Subjects do not simply reproduce their previous drawings but still show the same pattern of orientational preferences, albeit with a higher level of noise. Immersion in the virtual environment thus seems to be sufficient, but not necessary to induce position-dependent recall. This is in line with the findings by Basten et al. (2012) who induced position-dependent recall simply using an imagined travel as a prime.

It is interesting to compare the various angles discussed in position-dependent recall with those studied in the literature on judgments of relative direction (JRD). The intrinsic axis of Shelton and McNamara (2001), i.e., the viewing direction in which an environment is most easily imagined, is a property of this environment. It does not change if the observer moves and is therefore allocentric in the sense of Klatzky (1998). The intrinsic axis is thus similar to our canonical view direction of the target place. The imagined heading direction is defined in the JRD task by the instruction to imagine facing towards a particular object. In our experiment no such instruction is used, but subjects may interpret the sketching task in a similar way. If they did, imagined heading would be the equivalent of our airline direction. It can, however, not be obtained from the imagery itself, but needs to take into account the current location and its remembered spatial relation to the target. Finally, sensorimotor alignment (Kelly et al., 2007) does not seem to play a role in our virtual environment experiments where subjects are teleported between sketching sites. Our angle of body orientation is defined by visual cues from the environment and knowledge of the spatial relations between sketching site and target. In real-world experiments, this may be combined with cues from path integration.

In conclusion, position-dependent recall is consistent with the idea that spatial long-term memory is organized as a graph of local places, views, or reference frames which can be “loaded” into a chart-like working memory stage for purposes of imagery and spatial planning. The representation in the chart has a specific orientation leading to view-like productions in sketching tasks. The orientations depend on the current situation and task, presumably in ways useful for spatial planning.