First things first, Virtual Reality cinema is a misnomer.
The reason being, in VR you create an experience and not just tell a story. Therefore, the language of cinema is simply obsolete.
Filmmakers are having a hard time coping with the uncertainty and lack of control, but they are learning quickly. This article pushes the bar a bit further by defining a Narrative AI (artificial intelligence) strategy for cinematic Virtual Reality.
In “Ghost”, the viewer is an unseen observer.
By contrast, “Character” is where the viewer is an actual participant in the drama – thus allowing interaction in the world.
Out of the four types of VR, 360 video is the least common denominator. It’s so low in fact that most VR purists don’t even consider it VR at all. In 360 video, you can move your head in a shot and see all around which is way cool but it’s entry level VR. The real goal of VR is to put the viewer in the scene as a participant who’s able to influence the story.
This is represented by the “Character with Impact” box in the image above and it’s the highest form of VR.
Later this year, VR penetration will start to accelerate and a thirst for quality content will follow in years to come. It’s now possible to create next generation VR … here’s how.
Step #1: Create a World
In real life we create stories out of experiences we’ve had with people and environments.
In VR the same is true so the goal is to create a world where this is ripe to happen. The central questions are:
- What type of world will give the viewer a rewarding experience?
- What type of experience do I want them to have?
- What characters inhabit the world?
- What are the character backstories?
- Where will these characters take the viewer and what emotions will they experience as a result?
At face value, these questions appear similar to questions you ask at the beginning of any story outline but the difference is that the emphasis is on experience.
VR is inherently non-linear which demands experience-based storytelling over linear storytelling. In experience-based storytelling an emergent narrative reigns over a linear based narrative and that starts with the world.
Step #2: Enhance Emotion through Immersion
Many psychological studies have been conducted on VR’s ability to enhance emotional connection to material.
The sheer fact of putting on the headset allows for far greater emotional responses than we get from cinema. However, the key to unlock VR’s potential relies on the creators ability to maximize the viewers immersion in the world. This is another argument for aiming for the “Character with Impact” square in the image above.
Allowing the viewer to become an active participant who impacts the story is the key to immersion maximization. The research shows a direct correlation between immersion and emotional impact.
Step #3: Create a Narrative Artificial Intelligence
This is the secret sauce of immersion. This is how you make the viewer a participant that affects the story – through collect nodes and story modes.
The Collect Node
Collect nodes learn about the viewer by collecting inputs from the user in order for the story nodes to later make decisions to maximize immersion.
They collect tactile input (controller, or movement), audio input (speech, laughter), and visual information from the head mounted display (HMD). The collect nodes are placed at strategic places in the world in order to learn the greatest about the viewer.
For instance, if you’re making a comedy and you’ve placed a comedic character in the world you may want to test if the viewer is engaged and also if they’re laughing. Collect nodes can get this type of information.
The Story Node
Story nodes change the experience for the viewer based on the inputs collected from collect nodes.
To continue with the previous example, if we’ve collected information from the viewer that suggests that they like a particular type of comedy (via laughter and attention … i.e visual input) then the story node may branch the experience in a way to highlight this character and this type of comedy.
While in graduate film school at USC, I made a film that performed this task in a simplistic form. It was a comedic film that listened for laughter and changed the story if laughter was detected or not. It was a test on whether a particular form of comedy (physical, slapstick etc.) worked for the viewer. It was developed in Flash using filmed elements and actionscript. The project was ahead of its time but it was useful as a precursor to the theory presented here.
Narrative AI Experience Space
The above narrative AI experience space shows how collect nodes and story nodes work together to shape the experience over time. The flow appears linear because I put time on the X axis but the flow is really a non-linear experience. Story nodes are allowed to loop back and across into others – thereby creating a web of interactivity and possibility.
Now let’s dive a bit deeper.
Hello I’m a binary experience space 🙂
I’m a Narrative AI tool used to create interactive non-linear stories for virtual reality
Experiences spaces are where the rubber hits the road for emergent narrative AI. It’s a tool that’s used to plan the paths through a cinematic VR experience. Each path through the experience space represents a story the viewer/participant can choose.
For simplicity let’s define a super node.
Super Node = Collect Node + Story Node
The building block of the experience space is the super node. Super nodes are just a mash up of collect and story nodes…… but basically it’s where the story diverges based on subconscious input from the viewer/participant.
Binary Super Node
Let’s consider a binary super node where the viewer/participant can make one of two decisions. If we expand the experience space out to three levels… we end up with a 3 bit experience space with 8 possible endings.
3 bit Binary Experience Space
3 bit experience spaces are interesting for storytellers because we’re used to developing stories in three parts.… Therefore it’s convenient to think of the experience space as a compounded three act structure. Imagine building an experience space with a unified theme but concluding in eight distinct endings….. each highlighting a nuance of the experience.
Why stop at binary experience spaces?
5 bit Super Node
It’s doable to expand to more complex number systems but let’s not get carried away. The computer industry has made binary systems easier to discus so I chose it for simplicity.
A call to action
It’s up to us… the content producers…. to make VR a medium that lasts the test of time. This means we must get away from the simple linear narratives we’re used to and embrace interactivity via emergent narratives. Let’s get out of our comfort zone and create truly revolutionary experiences that stretch the narrative paradigm filmmakers have been accustomed to. I’m sharing this theory in hopes that other content users will use it to march toward global agency emergent narrative VR. All aboard!