Interactive Narrative in Virtual Reality

TL;DR

This paper introduces a virtual reality tool for creating interactive narratives using a simple markup language, enabling content creators to craft immersive stories in VR environments and exploring future development directions.

Contribution

It presents a novel VR-based interactive storytelling tool that simplifies content creation through a markup language, expanding interactive fiction into virtual reality.

Findings

Enabled creation of interactive narratives in VR

Demonstrated use of markup language for storytelling

Discussed future development pathways

Abstract

Interactive fiction is a literary genre that is rapidly gaining popularity. In this genre, readers are able to explicitly take actions in order to guide the course of the story. With the recent popularity of narrative focused games, we propose to design and develop an interactive narrative tool for content creators. In this extended abstract, we show how we leverage this interactive medium to present a tool for interactive storytelling in virtual reality. Using a simple markup language, content creators and researchers are now able to create interactive narratives in a virtual reality environment. We further discuss the potential future directions for a virtual reality storytelling engine.

Figures

Full text

{CCSXML}

<ccs2012> <concept> <concept_id>10003120.10003121.10003124.10010866</concept_id> <concept_desc>Human-centered computing Virtual reality</concept_desc> <concept_significance>500</concept_significance> </concept> <concept> <concept_id>10003120.10003121.10003124.10010865</concept_id> <concept_desc>Human-centered computing Graphical user interfaces</concept_desc> <concept_significance>300</concept_significance> </concept> <concept> <concept_id>10010405.10010497.10010510.10010512</concept_id> <concept_desc>Applied computing Markup languages</concept_desc> <concept_significance>500</concept_significance> </concept> </ccs2012>

\ccsdesc

[500]Human-centered computing Virtual reality \ccsdesc[300]Human-centered computing Graphical user interfaces \ccsdesc[500]Applied computing Markup languages

\printccsdesc

1 Introduction

Since the beginning of time, people have been telling stories and created connections through them. Over time, these narrative took different shapes and forms, from oral storytelling traditions to modern technologies such as e-book readers. Given how central storytelling is to our collective psyche and cultures, it comes as no surprise that people constantly look for novel ways to express their imagination.

In this abstract, we propose a new and interesting paradigm for storytelling in Virtual Reality (VR). We present our interactive narrative concept, its implementation, and describe possible applications for our tool. This tool allows to create a modular and immersive experience in VR, with the goal of adapting a narrative to each reader.

2 Related Work

During the 70s, two forms of interactive narratives appeared: Interactive Fiction (IF) and Role Playing Games (RPG). IF encompasses both “Choose Your Own Adventure” stories, where readers select branches to advance the story, and “games” where readers type text in order to interact with the textual environment [8].

During this time, pen and paper RPG emerged, with games such as Dungeon’s and Dragons [7]. In such games, players work together to tell a cooperative story, bound by a set of rules and overseen by a "Dungeon Master" to give the story both structure and game elements. This combination of narrative freedom coupled with a grounded rule-base contains useful lessons on how to keep a person engaged and interested in the game.

One of the first IF game, “Adventure” [4] had readers interact with a world and characters reminiscent of the fantasy settings of most RPG at the time. In computer-based IF games, the Dungeon Master is replaced by specific algorithms embedded in the software.

Over the last decade IF has gained an increased attention [9], thanks in part to easier writing tools based on markup languages and graphical programming, such as Ink111https://www.inklestudios.com/ink/ and Twine222http://twinery.org/. At the same time, pen-and-paper RPG players started to investigate new formats, such as “Multi-User Dungeons”, a collaborative multi-player version of IF games. Nowadays, one of the most played platform of these games is maintained by Iron Realms333https://www.ironrealms.com/. Another venue consists of online forums such as “Quests”444https://forums.sufficientvelocity.com/, where one person plays the role of the author, generally telling a story based on a single protagonist. Other players then vote on the next set of actions, followed by the author continuing the story.

IF is a good framework to bring together creativity, flexibility, and adaptation. Yet, it suffers from readers having to explicitly interact with the story. Switching back and forth between reading the narrative and using its interface can impede readers’ immersion or the flow of the narration. RPGs possess a collaborative element which can further strengthen the narrative. However, with computerized versions, the role of the Dungeon Master, who interprets the story and serves as the narrator, is often diminished or even non-existent. One notable exception is Neverwinter Nights [2], a 3D game that lets one player endorse the role of the Dungeon Master and prepare the narrative. Yet, even there, the limited number of 3D assets can impede creativity.

A main advantage of interactive stories is that readers empathize with protagonists, as if they were part of the story. Adaptive stories have been envisioned as a way to help people connect with unfamiliar situations and characters [10]. In prior research literature, IFs have been used to study and train empathy. In Cavazza and Charles [3], brain recordings were used in a virtual environment to detect empathy and alter the narrative. A framework was also described to adapt a 3D narrative using physiology [6]. They used a “planning domain definition language”, a syntax coming the artificial intelligence domain. These examples are either focused on specific states, or do not intent to bridge the gap with lay writers.

Our work aims at giving more freedom to content creators, by integrating modern IF tools, and building a platform that can easily be expanded to meet a creator’s needs, as discussed in [5]. The following section describes our software tool.

3 Interactive Narrative in Virtual Reality

This section describes our proposed tool for textual storytelling in VR. We first explain the conceptual aspects of the tool, followed by its technical implementation.

3.1 Design

Our tool is implemented in VR, and the story is conveyed through regular text with words. Using text as a medium lets the creator or writer quickly and easily change any element of the story without specific technical knowledge.

Visually, we propose a simple nature scene in the background. The calming atmosphere of a forest meadow should help offset some of the discomfort that can arise for users new to VR. This meadow is also used as background to the area displaying the text. To increase the users sense of “presence” or “immersion”, we added a slight grass sway animation that simulates a soft breeze. We did not include sound effects in this first iteration of the tool.

This tool was conceptualized after the “Choose Your Own Adventure” paradigm, where the narrative can branch into several directions. In this first iteration, we hide the choice from the user, so that they are unaware that other paths exist. This choice was to simplify the user’s experience while allowing for more options in future work. The following section described the implementation stage.

3.2 Implementation

The architecture presents two principal components: a Display Engine (DE) and a Selection Engine (SE). The DE handles the content delivery to the user within the experience, and the SE handles the choice of the story path.

For the SE, we wanted to build a separate program that would allow someone outside of the DE to control the flow of the narrative. This ’Director’ would make the choices that affected what path the story follows, and thus influence users’ experience. We opted for a simple interface with the story options as buttons for this purpose.

We implemented the Display Engine using Unity, as it has the right balance of features and ease of use. The forest meadow scene background was built using Unity features, and its physics engine was used for the movement of the grass. A publicly available tablet mesh was then positioned at the user’s eye level to display the story. The text is displayed onto the virtual tablet using Text Mesh Pro Unity library, which allows advanced text features to be used in the stories. This includes high level HTML markup features, such as changing color, position, font, or alignment on the fly. To further support future research, we used the underlying mesh of each word to create a collider surrounding it. This collider can be used in future experiments for eye or head tracking, or to create dynamic “interactive areas” inside the story itself.

The SE component contains the interface for a Director, the bridge to the DE, and allows for control of the story. It was built using C#. The Director’s Interface was created using a Windows Form program that displays the current story status, and dynamically creates buttons for the Director to control the story flow. On the Unity side, a C# script connects to the Director’s Interface and follows its commands. Lab Streaming Layer (LSL) was used for the bridge, given its compatibility with both C# and Unity. Finally, we use the “Ink” scripting language to write the stories. Ink is a language specialized for this purpose, allowing for non-technical users to easily write branching and dynamic stories.

4 Applications

There are several avenues possible when considering the applications of this tool and technology, beyond supporting creative writers. One is to use it for research and user studies. For example, a researcher could build various versions of a story and test how the different versions affect people, such as when looking at emotional response or empathy. In this situation, the researcher would be able to easily and quickly build, test, and identify the practical effects of small changes within the story.

Another application relates to tailoring content to fit specific age or demographics. An author could create a story using several levels of language, and dynamically adjust it so that the reader can more easily comprehend the content. A news article could use simpler words for a teenager or a child, while a college student might get a more thorough description. This could allow for the same content to be absorbed better according to the readers’ needs.

5 Evaluation and Preliminary Results

After having performed several pilot studies, we planned to formally evaluate the performance of this tool in three stages: 1) a technical evaluation, 2) a first user study to see how various people react to different stories, some of which being personally tailored, and 3) a second user study to see how this tool could help support content creators.

The technical evaluation will support evaluating the effectiveness of the tool. This includes reviewing the usability of the tool by non-technical users, checking the viability of the Director component, and testing the Director’s LSL extension via an automated remote program. We will measure usability by asking non-technical creative writers to write stories that utilize the branching behavior. A usability questionnaire will be administrated to assess the interface ease of use. The viability of the Director will be tested by performing tests to estimate the minimal power and processing requirements of the system. To test the Director’s LSL extension, we wrote an automated script that can choose a story path at random for the reader.

Next, we will write a set of stories with several branching options, such as the ability to choose a male or female protagonist. We would then change the story to accommodate the reader, and investigate if a tailored story can promote empathy or foster playfulness. This result would be compared to a control group who has the story parameters set at random. Should this step be successful, we would then move on to stage 3, where we streamline the tool for use with existing content creators.

6 Future Work and Conclusions

There are a number of avenues to explore in future work. We envision reaching person’s empathy by tailoring the narrative. We will further explore the possibility of creating an artificial intelligence Director to control the scene. Such directors are common in gaming, and we believe this concept can be adapted to a literary experience.

In conclusion, we have presented a tool and a novel approach for creating interactive narrative fiction in virtual reality. Our tool is a stepping stone to a different means of engaging in storytelling. It allows for content creators with no technical experience to design relatable stories in VR. With its simple concept and design, more advanced versions of this tool can be created for specific applications.