Evolving The Art Of Storytelling in Virtual Reality, Mixed Reality And Beyond

A Trip Through My Brain…

Often while I’m working on new emerging fields and spaces, I hit upon subjects I don’t have the initial answers to. It’s at that moment I turn to writing to help me focus and think about the problem. This could be any type of problem from design to theory to sorting out functional flows. The item I had in my mind this morning was specifically about  immersive stories. Get ready for a wild ride!

Core Immersion

When we talk about immersion in games and in the realm of user experience we talk about the user being consumed in the moment. The outside world fades away. For the most part I feel this is truly the main goal of virtual reality. You want your user to experience something other worldly, but on the other hand when we think about immersion in relation to Alternate Realities,  (I may just use this term for AR / Mixed Reality coined here), we have to break down what this means. Let’s take a look at writing this article as writing a story.

I apply some basic UX principles. Simple items I teach and use every-day when engaging in design thinking.

  • What activities is the person trying to accomplish and why?
  • What experiences best suit these activities?
  • What technology best allows these experiences to play out?
  • What constraints are both in the technology as well as in the experience?

That being said when you look at these activities I believe it gives you a clue as to the power of a story.

Find The Practical In The Fantastical

I’m always trying to find the practical application in every technology no matter how new it is. In this instance, I want to provide an analysis of what crazy ideas are in my brain so as to help think through the concept. So let’s dissect this experience and see what happens to the very nature of the activity and how it’s born. How meta, the story of a story!

Writing An Article

  •  Traditionally we defined this in the past as sitting down with a tablet (not that kind) and etching in words. (maybe even cave drawings)
  •  Then this evolved into writing on papyrus and other “paper materials”
  •  Then this evolved into the paper that we still use today, though much of the writing may have moved to Typewriters / word processors
  •  Then this evolved into computer tools and then into possibly using tablets (yep those kinds)

Yes, it’s true the words that were written could take you in and transport you to new places (including the author).  So how could we take and move this experience to an Alternate Reality scenario?

The primary activity for the above you might think is documenting. I would disagree and identify the primary active as storytelling, we use common language, terms, words and patterns to connect with our audience and convey meaning and thus tell the story.

Here is the challenge with words, not everyone can read, nor can everyone read the same language. This builds in inherent divisions in how we go about connecting with new people foreign to our own concepts. This is where holographic immersion provides a multi-dimensional window into the creators soul.  As opposed to to just watching a video there needs to be multi-channel, and multi-sensory connections.

A Story That Re-Writes Itself

I dream of creating experiences in these new dimensions that allow multi-faceted, multi-tiered stories, that weave in and out of current reality, anticipating design, and environmental shifts. What does this mean?

I have to be quite honest that I’m still working this out in my own head. Imagine entering into a plot point in a book and you find it boring, dull, or not compelling. You might flip ahead or switch to a new more compelling story. What if you could anticipate the viewers interactions(I know far out Neuro AI, and Cognitive Design Combined), and connect to this disconnection by flipping the story.Think of choose your own adventure on steroids? Perhaps, the protagonist becomes the antagonist.  If the lights are off in your room maybe the words / and visual imagery of your Alternate Reality (or just current reality at that point) shift to match that mood. If a new object enters your active environment it becomes an integrated component into the fabric of the creation. An object falls off the shelf and suddenly becomes a new pivotal point in your experience. Your response is measured and further procedurally crafted into the story with new interesting twists.

Mind Blown! – The things is I can see how these interconnected pieces, come together to create a new living experience and perhaps a new medium and mode for storytelling!

In this way we could start explore a new dimension of storytelling that incorporates environmental context, emotion, words, visual images, animations, touch and more. Yes, it’s also a little scary because by that time would reality breakdown? The very concept of storytelling in this dimension is absolutely intriguing. Even though this may be fantasy now, some of these things could be done with a lot of programming, and massive drawings that connect the dots (a future article)…

It’s crazy, but think about how crazy other creations have been throughout history (locomotion, steam engines, microchips) and what they might have sounded like on paper. The sad thing is the words above can only scratch the service of what it is in my head. A combination of mediums, skills, and disciplines have to unite to bring this concept to the masses. That is a huge challenge as we a world that believes in specialization, but that also makes us blind to a multitude of different perspectives – Connective tissues that we are blind to in everyday life, but suddenly popup when approached with collective lens.

For now if you made it through this and your brain doesn’t hurt, you win!