Virtual reality technology is still in its infancy, but it’s already caught the attention of people around the world. This new medium offers so much potential for immersive storytelling, and people are just starting to scratch the surface of what’s possible. In this blog post, we look at some of the possibilities of VR and discuss how this technology could shape the future of narrative media. Stay tuned because VR is something you don’t want to miss!
My Personal Experience With VR and VR Storytelling
Over the past few months, I’ve been immersed in various virtual reality worlds. A journey that, for me, as probably for many others, began with the Covid 19 pandemic, when travel and outside experience became restricted.
This morning, using my Oculus (now Meta) Quest 2 headset, I played tennis on the Australian Open court in the First Person Tennis VR game. An incredibly immersive and real experience. It doesn’t matter that not every detail of the graphics is perfect. What matters are the physics behind the game, the mechanics and how the racket and balls react in different scenarios.
When I was competing against opponents around the world, it was as real to me as being on a real tennis court. As trite as that may sound. I can promise you that I worked up quite a sweat – so much so that I’m now investing in a small fan to at least cool down my facial area during the match!
Right now, I’m using the VR app Immersed to read and type this article because sometimes it helps motivation and focus if I head to my rainforest studio – where all distractions disappear. The keyboard is mapped, so it appears as a keyboard on my desk. When I need to move the screen (or have multiple screens), my virtual hands – mapped to my real ones – can select and drag objects in the environment.
Since VR is so “immersive,” one of the questions I’m most interested in is what role VR plays in storytelling. In other words: What’s VR storytelling?
To address this question, we need to look at some important aspects of storytelling in VR and give some concrete examples of apps, experiences, and games that illustrate the current state of play in VR content.
What Is Storytelling
Let’s define some terms quickly. If you search for a definition of ‘storytelling’, you’ll find many different answers. For now, let’s go with one from the Asian Development Bank that I rather like:
Storytelling is the vivid description of ideas, beliefs, personal experiences, and life lessons through stories or narratives that evoke powerful emotions and insights.Asian Development Bank
I understand that this is a business-oriented definition. Nonetheless, it encompasses much and can be applied to many creative stories – for example, in fiction writing- because most novels or short stories contain a message or belief that the author expresses through the tension and conflict that are part of any story.
How about VR, virtual reality? It’s important to know that VR is just one of several future technologies that will be very important in all fields in the coming years. They include Mixed Reality, Augmented Reality, and holographic technologies.
Mixed reality is a term that describes a fusion of virtual reality and augmented reality. Imagine using virtual reality to explore a new city, for example, and then finding yourself in a scenario where you’ve to interact with an entire building to get inside. An augmented reality scenario. In such a case, you’d have a mixed reality scenario where you’re only partly in virtual and augmented reality.
Holographic technologies are new ways of using digital light to create the illusion of three-dimensional space. Microsoft and other companies are working to develop holographic technologies that allow objects to appear to float directly in front of us or even around us.
For example, I could look at an object that seems to exist outside the screen. Holographic technologies are another candidate for the future of VR, AR, and MR.
It’s debatable which of these technologies will provide the most powerful experiences and immersion in the world of stories and storytelling, so I think it’s fair to group them into a category we could call “immersive technologies.”
The core of all technologies is that as a user, you’re immersed in a dynamic space and experience over which you often have a high degree of control.
Is There Such a Thing as a VR Story
Pixar founder Ed Catmull cautioned a few years ago that VR isn’t storytelling because VR is more about experiences and environments. The need to use three-dimensional space made it difficult to create linear narratives. He admitted, however, that he’s open to seeing how things will evolve.
Linear narrative is an artfully-directed telling of a story, where the lighting and the sound is all for a very clear purpose. You’re not just wandering around in the world.Ed Catmull, in an interview with The Guardian
I’d say that one of the tests of whether something is a good story or not is the sniff test: does it feel like you experienced a story after you leave it? Is VR storytelling a new art form?
The remarkable thing about VR is that I come out of different games and experiences feeling like I’ve been in an immersive story – I was one of the characters, a participant.
Various VR experiences aim for a linear narrative – much like a movie director. For example, I recently saw the powerful short VR film Surviving 9/11 – 27 Hours Under the Rubble, which in 360 video tells the story of the last person to be rescued from the rubble of the World Trade Center on September 11, 2001. It’s a great example of something that I believe will become a huge phenomenon in the future: immersive journalism.
Elsewhere, linear elements are incorporated into games or VR experiences to tell part of the story before the “experience” (story?) continues.
For example, the Resident Evil 4 game in VR uses brief interpolations at key moments to show, from a third-person point of view, what’s happening at that moment. It’s a decision that hasn’t met with universal approval from gamers, who feel it interrupts the immersive experience. Sniper Elite does something similar with its kill cam – it shows the bullet’s path through the enemy target when a successful shot is fired.
A Different Kind of Framing
One of the most important things to keep in mind when discussing VR is that, in most cases, there are no traditional “frames” for the narrative, such as those found in books, in movies, on the silver screen, etc.
There’s a big difference between a 360-degree or 180-degree camera and a full VR experience where the user navigates and interacts with the environment.
For example, when you enter a VR game, the user often becomes a character in the game story. Or at least controls one or more characters. In traditional storytelling, we’d probably divide these experiences into the first and third person.
Although linear storylines can and will be built into VR experiences and stories, there’s often a significant nonlinear factor where the user’s individual choices in a virtual environment determine the outcomes.
As the psychologists would say, the user has a stronger sense of “agency” in the story.
When we suspend disbelief in a story, such as a novel, we use our imagination to penetrate all the gaps in the narrative, description, plot, etc. Therefore, as readers, we have “agency” – even if we consider reading or watching a movie to be “passive.”
This suspension of disbelief and intervention of imagination happens in the same way when we have a VR experience or story – with the difference that the experience isn’t just emotional and psychological but much more physical. When we move and feel feedback through the haptic devices in the controllers, vests, etc., used on VR, we’re “in” the story to a much greater degree.
In a word, we experience it. In this sense, StoryTelling becomes StoryExperiencing. Abigail Posner, Google Zoo’s director of strategic planning, calls it “storyliving.” Another way to think about it would be to call it ‘experiential storytelling.’
The author of a book or a movie is a creator of experiences, a shaper of narratives. So is the author of a VR story – with the huge addition that the designer of a VR experience and environment is playing with many possible paths and interactions that can occur as the user navigates and interacts.
A New Form of Storytelling
Each new medium has spawned a new form of narrative.Frank Rose, scholar
VR has the potential to provide interactive experiences that engage users in a complex network of story elements, computer capabilities, and human interaction-and already does.
These narratives are already so realistic and engaging that people feel like they’re a part of the story. They’re able to control the outcome of the story. Over time, the possibilities will be endless.
Computer capabilities will enable an infinite number of stories, and human behavior will provide the emotional connection that makes the experience truly compelling.
The Importance of Abstraction in VR Storytelling
When it comes to VR experiences and VR stories, there’s an exciting and necessary line between realism and abstraction.
Suppose the storyline of a virtual environment is too concrete. In that case, the range of possible outcomes is reduced, and so is the journey and experience that the user – sometimes called “guest” or “participant” – can have.
Therefore, it can be important to make the story intentionally abstract to allow for different paths of action that depend on the user’s choices about where to look, where to stay in the virtual space, what to do with the interactive objects, and how to interact with other users in the game.
It’s a fine line because you usually have a sufficiently sharply drawn character and scenario to be captivating. Still, it gives the user the freedom to create a dynamic experience that completely suspends disbelief in the virtual environment.
Usually, a game or experience requires a series of different paths or solutions to get to a progressive moment in the story – I guess you could call it a “VR story beat” – before moving on with the experience.
This is in stark contrast to much of fiction, which is usually based on strict causality. This happened because this happened. Unless it’s a mystery novel, in which case false lures, red herrings are deliberately laid out to confuse and provoke the reader into guessing the solution. Then the rug is pulled out from under them, and the real villain emerges.
Immersion in Foreign Lands
One of the true strengths of VT is its ability to immerse readers in a fantasy world they could never experience in real life.
Thus, you can fight zombies in a post-apocalyptic Los Angeles (After the Fall). Or you’ll build a series of automata in a magic tree to harvest the plants’ energy so that the tree doesn’t drown and people no longer have to be present all the time (The Last Clockwinder).
One of the problems of VR is that no two-dimensional or even 3D representation of VR does it justice. There’s simply no substitute for actually entering a virtual environment, walking around in it, or teleporting and fully experiencing a digital world.
This, in turn, allows for characterization and perspective that can be completely fantastic – flying, shrinking, expanding, entering different dimensions throughout a game (The Room, for example), or solving puzzles on a magical island (Myst – the reason I first got a VR headset).
The truth is that VR is still in a very early stage of development, even after 40 years of experimentation and evangelizing. Right now, you’ve to use a head-mounted display (HMD) to enter VR. The Oculus Quest has lowered the barrier to entry considerably. For a few hundred dollars, you can enjoy VR experiences and stories wirelessly. However, it still feels clunky and isn’t as seamless and effortless as I imagine it’ll be shortly.
When the first immersive VR /AR glasses hit the market that is as simple as sunglasses and cost $100 or less, that’s when VR will hit the mainstream. Then a huge community of story creators and consumers will be tapped.
Game Engines Create Virtual Environments
Aside from the 3D mini-experiences in VR, there are currently two main immersive technology approaches to creating VR stories: film-based and game-based. In the former, 3D cameras are used in different positions or even with tracking to create a point of view that the user can orient to. The second method uses specialized game development programs such as Unity and Unreal Engine to create an entirely digital world.
Therefore, in my opinion, the games approach is currently more powerful and interesting than the movies. The real strength of VR is full immersion, not quasi-immersion.
The game approach can use the full range of imagination to defy the normal laws and conventions of normal human experience. Stories from the real world can be expressed in a surreal way, to great effect. The physics and mechanics of the medium continue to improve, offering more and more creative possibilities for game designers and VR experience creators.
Limitations for VR Storytelling
Aside from the linear vs. nonlinear debate mentioned above, the main obstacles to the growth of VR storytelling right now are as follows:
- The size of the user base. Once the VR world becomes truly mainstream, this problem will disappear.
- The processing power of the headsets. As excellent as the Quest 2 is, it uses every ounce of a processor equivalent to an Android phone. This, in turn, reduces the achievable native graphics resolution. Many get around this problem using PCVR: Computers that stream games and experiences to the headset, even wirelessly. However, this is expensive and requires an extra step (and sometimes cabling). However, if there’s one aspect of VR technology that’s certain, it’s that computing power and storage will increase – probably at an exponential rate.
- Speed of production. Right now, most high-quality VR experiences and games are developed by teams of developers. There are notable exceptions where a single developer has broken through (First Person Tennis is a good example). As authoring tools become more powerful and easier to use, I think we’ll see democratization and diversification of VR story experiences.
Getting Into the Experience
Want to experience this new frontier of experience and visual storytelling for yourself? I recommend you get an Oculus (Meta) Quest 2 headset in the 256G version and dive in.
Books to Read
One caveat is that the field of VR and virtual reality storytelling is rapidly evolving, so the publication date must be considered for all books and other references. Nonetheless, I’d recommend:
The End of Storytelling – The Future of Narrative in the Storyplex by Stephanie Riggs.
Storytelling for Virtual Reality by John Bucher