In my first article, I offered a brief glimpse into the some of the experiments I had been working on as well as a just very quick overview of the holographic UX design world. Starting with this article we will begin to apply my practical UX approach towards crafting a holographic experience that is design driven and not development driven. This is not to dismiss developers this is a fact that the technical know-how requires a lot of developmental understanding to achieve the desired design solutions. This is by no means meant to be exhaustive. Also while some of these patterns, methods, and techniques will apply to other devices (meta, leap motion, Vive, Oculus), this article is meant to focus on the HoloLens. Other device articles, as well as those on the topics ofVR, will be coming later.
By understanding these patterns and technological cross-over it will help you to craft first class world experiences. Keep in mind this is still very experimental and there are a lot of challenges we must overcome. The least of which is the technological evolution of the devices and their capabilities. Many companies have started to embrace the HoloLens as a system for business use. That can only mean good things as we move into this new design evolution.
First Time Explorers Welcome To A New Design Medium
We are about to enter a brave new dimension of design. I couldn’t be more excited to be immersed in the day to day evolutions of these new design systems that in many ways won’t hit the mainstream for several more years. It’s been my dream to create application experiences that co-exist in our real world. This involves pulling together not one technology but many forms to create, drive and craft new ways for levels of computers & humans to work together. This goes far beyond using just holograms, but AI, Machine Learning, and all of these interconnected, eco-systems. It starts to tap heavily into neuro and cognitive science.
The techniques to craft these experiences are still extremely developer friendly which means it’s hard for those who don’t want to face the daily errors and the constant barrage of problems to get a glimpse into these systems that make up the constructs of these new worlds. The terminology is a hotbed of discussion right now. As a note, I’ll be throwing in my own terms because I need a common language to communicate and as you may know I like to create things!
The Usage Challenges
For many of us in the techno-bubble, we push on past the uncomfortableness of both using and wearing these devices. Believe me, I have wanted to toss the device many times, but due to cost, I refrain. The battery occupies a lot of the device and many people after wearing it for a while have the distinctive red Rudolph mark above the bridge of the nose. Much like the pain of existing in VR experiences, we must push on! Beyond putting the device on there are a few things that people tend to do that are unfamiliar with the HoloLens the first time they use it. This all adds to the challenges facing this technology.
My favorite activity is training people in the usage. I also like to just watch to see what first-time users do. There is also a fun component of seeing how others react to the technology. When someone is wearing a HoloLens It’s bold and out there. I think that tends to be less scary and pushes us out of the Uncanny Valley and away from adoption issues that we ran into with Google Glass. Now we just need to get past the price barrier and comfort.
First Time Responses Witnessed
- Most people have problems with device adjustment and as a consequence can’t see the initial holographic experience.
- I’ve run into a few people that the device barely fits their head.
- Upon immediately seeing a Windows start menu they try to air click furiously
- They attempt to apply other types of gesture-based interaction (grabbing space, touching, even punching)
- They stay single point focused on the current field of view VIP (virtual interaction point). I call this the refrigerator effect and is very similar to early child behavior and cognitive responses of object permanence. Basically, if I don’t see the item in the refrigerator right in front of me it must not exist!
- They tend to quickly get lost in an application experience because of improper wayfinding cues. (where do I go next? what can I do?). This is very similar to letting someone else use your computer and finding that now your Netflix account is a combination of Children Shows & War Movies. We really need profile systems at the OS level.
- They tend to get lost in application shift (applications are housed in their own space). How do I switch activities?
- They forget the “bloom” command to bring up windows.
- They don’t consider voice as an option.
I find it takes an average person about 3-5 hours to feel comfortable using the HoloLens. I personally spend about 10 -20 hours in the device a week.
Now that we understand a bit about how a first-time user is going to potentially react to this system. Let’s take a deeper dive into some of the constructs you want to be familiar with in this next design evolution.
The Core Components Of Multi-dimensional VEX (Virtual Experience)
Once we move beyond these challenges we can start to open up our mind to the potential of the multi-dimensional world. The world where you sit around and wonder “where did I put that hologram?”
We all like to understand where we are in our world. Navigating through an application experience can be the same in a holographic situation as it is in a two-dimensional app world. The biggest challenge is the user is now directly manipulating and working with VIPS in an area that is generally familiar to them – A room, a house, an office space. Instead of just laying information on top of this domain we can do so much more! These are entirely new concepts of personal customization of VIPS. Ares of customization that until now we have not even begun to scratch the surface. I like to think of this as the ultimate evolution of anticipatory design (more on that later perhaps). At the root of our initial experience, we should start out by doing these things.
- Understand our core devices technical capabilities
- Provide methods for onboarding and orientation
- Effectively apply context & meaning to situational interaction points (SIPS),
- Indicate Interaction Capabilities
- Provide direction, context, and clues, that avoid initial cognitive dissonance to design patterns and objects existing in the VIP. This includes the existence of even removal of real world objects.
- Understand object to surface collisions, occlusion, and temporary surface displacement and potential impacts of misplaced interaction objects to the overall application experience.
- Draw connections and context between connected SIPS, and VIPS, to have a well thought out understanding of potential Virtual Experience (VEX).
- Contemplate the capabilities of transformational interfaces that cross the boundaries of real and intangible objects.
- Direct attention to view appropriate design constructs (no popups please)
- Don’t overuse text! think about illustrating visually stunning interface interactions
This is just the start and is based on hours of research as well as hands-on experience watching first-time, intermediate, and advanced users interact with a HoloLens. Here are several items we want to consider when we are crafting our onboarding experience. If you are curious about the inner-workings of the HoloLens including how the light system works check out this article.
- Spatial Size
- How big is the room or area?
- Are there multiple involved spaces?
- How many meters is the expected work area? (unity measures in meters)
- Is the area generally cluttered (like my office) which means planar surface detection will be a challenge
- Is the area outside (In that case you will need a bit of shade as direct sunlight doesn’t work well).
- Surface Type & Detection
- Environment & Lighting
- What’s the average luminance in the area?
- Will this light vary?
- How many windows are in the space?
- Color usage In Objects & Models
- Object Occlusion
- Modes of Interaction
- Gaze is the primary interaction that acts like a traditional mouse cursor. Where you look is typically where the interaction will occur.
- Will you be interacting with other devices?
- Will there be cross-over between your interaction systems
- How will you let people know about the capabilities of your interface
- Will you use voice commands?
- Will you use sound cues to orient and position people, places, things
- Will you use animation cues to signal a change in events?
- Understanding the FOV
- Base Principles of 3d Design
- Creating and understanding colliders
- Understanding rigid bodies
- Poly counts
- Crafting & Animation Object Status
- Menu Systems & Canvas Object
- Total Virtual Objects
- Special Effects Use & Optimization
- Rapid Prototyping In New Tools
- Unity (learn it, master it, love it – and hate it)
- Streamlined Processes (more on this in the future)…
- Blender, Maya, Zbrush, And many others…
In the next article, we will look at how to craft a potential Onboard experience. This means understanding what we can, should and shouldn’t do in crafting a VEX (Virtual Experience). To learn more about the definition of VEX’s and VIPs I’ll be working on a new design talk as well as another article on the thought I’ve put behind these virtual constructs.