New Blog

Back to all posts
  • Holograms, Magnets, and Cher Horowitz' closet: reflections on augmented reality and virtual real

    Holograms, Magnets, and Cher Horowitz' closet: reflections on augmented reality and virtual real

    A couple months ago, I started working at Normative—a future-focused software design firm. My main role is to research the customers/users of our corporate clients, so that we can design more meaningful and impactful design solutions for these clients and their customers. 

    Specifically, my focus is on research through interviewing, direct observation, focus groups, and other qualitative research methods. And so, my understanding of how the research will be used—what kinds of software products and solutions could be created—shapes what I look for in the field. The better my understanding of what is possible with software, the better I can do my job. 

    But I'm pretty new to software. And I'm definitely new to Augmented Reality and Virtual Reality. My exposure to AR/VR was limited to iphone games like Flash Invader or Pokemon Go. So, I jumped at the opportunity to participate in the Spotlight AR/VR Conference with my colleagues a couple weeks ago.

    There were tons of ideas, info, case studies, questions, and demos at the conference that were illuminating, surprising, and exciting. Below is a quick & dirty summary of my favourite bits.

    First, a note on definitions

    Augmented Reality: still mostly sensing (seeing, hearing, touching, tasting, etc.) in the real world, but it is being augmented somehow using technology. For ex. You are walking down a street, looking at the ground in front of you through your smartphone screen, so that when you spot a monster on your screen, you can flick balls at the monster on your screen and gain points. AR requires a smartphone, tablet, or some other specialized device.

    Virtual Reality: your eyes and ears are completely encased. All you see and hear is the virtual world. However, you are still walking around or touching things in real life. It’s like you are in a video game - you look down and you have pixel hands, you look up and around and the virtual world moves as you would expect (except when it doesn't quite line up and ends up making you feel nauseous). Until Westworld becomes a reality, VR requires Oculus Rift, Google Cardboard, or one of a number of other VR goggles on the market.

    Mixed Reality: somewhere in the middle. You are seeing the real world but the virtual world is overlaid. For ex. in your home office you might put a floating screen and your favourite Beyonce music video / in your kitchen you might put a hole in the ground that peaks through to a hidden world as if you are up in the clouds. For now, MR requires Microsoft HoloLens. But Magic Leap may be creating something that fits into the MR category as well.

    "Talking about virtual reality is like dancing about architecture." -Chris Milk

    Normative colleagues demoing the hardware and software

    Reflections / fave bits

    Small, Medium, and Large sized bits...

    - A game that enables kids to receive faster medical diagnoses. Don’t Stumble Tumble is a game, and an assessment and preparation tool, that helps kids practice lying still during an MRI scan. This is important because MRI scans require the subject to lie very still for an extended period of time. And, MRI scan waitlists can be months long, and they are even longer when a child requires anesthesia to help them stay still during the scan. Thus, this AR game fastracks kids medical treatment, let's doctors know that the child is ready (achieves a certain high score), and saves hospitals time and money. I'm interested to see how else health care leverages AR/VR technology.

    - Virtual Reality gaming as the carrot, to get people talking. VR gaming can create opportunities for mingling and networking. For example, at a tech conference, conference goers lined up to play a two-player VR game. There was a leaderboard, and while waiting in line to play it becomes apparent that the way to get a high score is to team up with the two people behind you in line - so that you can use a super move. Eeep, that means talking to strangers! The game got people chatting and building a rapport with one another. Thus, VR can be used as a tool to incentivise behaviours.

    - Augmented Reality is not limited to vision. There is also AR for the other senses, including: hearing, smell, taste, touch. I was especially excited by the AR earbuds that enable you to, for example, tune out voices but keep the other noises in your environment. These would be awesome for the street car; I could listen to music while still clearly hearing the stop names and the noises acound me all without having to hear people's phone conversations. The Taste Machine was also interesting, where any taste can be electronically simulated / could be interesting for people who are dieting. Or along the same vein, the Smell Machine, which could be an interesting way to augment a movie experience, like the folks at Here There do with actual scent viles during film screenings.

    - Real-time hologram chat. Holoportation is a virtual meetings that feel as natural as face to face. The potential of this technology was mind boggling. It seems very Total Recall (1990). A hologram of the person you are “meeting” with is projected in front of you and you can see them, their facial features and gestures, as if they are right there. This demo video, showing a father and daughter interacting as if they are in the same room, blew my mind.

    - Tweak the real world to create a videogame-like feel. With roomscale VR, staff may act as “Ghosts” to help guide a VR player by subtlely shaking a prop in the player's peripheral vision, in order to hint that it may be an item worth checking out. Or magnets are added to objects the player picks up during game play, so that when they put it in the right place there is a videogame-like "click" in to place (for ex. when you place an ancient artifact on pedestal in Escape Tomb). But our minds will also do a lot of the work for us. In the example of a haunted house game that takes place in a 6 foot by 6 foot space, the game layout included multiple floors and rooms that were layered on to of each other. However, players would be walking in circles within the small space, so immersed in the game that they don’t notice that the house's blueprint doesn’t make sense.

    - You can "lose your keys" in mixed reality. There is such thing as Mixed Reality clutter or forgetting where you put something. For example, you might forget that you put a youtube video up over somewhere that you seldom go, then stumble upon it later - much like we misplace things in in real life.

    - Make AR help you acheive your own goals. AR can be used as a tool for self expression, to work as we each want it to in order to give us value in our lives. Ex. check out these hilarious videos by Jeremy Bailey. Jeremy's message was that: It is still early days, and the potential for AR and VR is largely unknown. It is up to the early adopters to play, and break, and hack, and create ways of using the technology that was not part of the original intent. This will push the field and enable even more uses that right now cannot be imagined. Yes!

    I also got thinking about...

    - What if AR and VR was used to help people overcome phobias? There are lots of studies showing how exposure therapy is effective and AR/VR could certainly support this kind of therapy. The treatment exposes you to the very things one is afraid of, in small managed doses. Over time, the fear is reduced or ceases to exist. Leveraging AR/VR for this would likely work with certain kinds of phobias like sharks, heights, flying, or maybe even cucumbers

    - What would Cher Horowitz’s AR/VR closet look like? Serious question. Will choosing what to wear via a computer screen (rather trying on a million combinations) soon be ubiquitous? Virtual try-on is already a thing, especially when shopping for glasses. And we’ve all had a meeting wearing a pirate hat, thanks to google hangouts. How could this technology be translated to daily outfit considerations? 

    - How is our notion of realness is changing? The lines between what we consider real, and what is augmented or virtual reality, are being blurred (this is one of the themes of the series Westworld, which I am currently watching!). Is an interaction I have with a hologram real? How can we distinguish what’s real and not? How will our definition of realness change?

    Still chewing on...

    - I had not considered the unconscious bias of many avatars, that may exclude the visible diversity of people using the software / for example entire genders or skin colours. A conference goer brought up this issue, and how it can be jarring to have to play a game as a white man if you’re black or if you have to play as a man if you’re a woman; in other words, to not see yourself as a possibility in the game as you see yourself. Interesting question about: diversity + inclusion x avatars.


    What is exciting you about AR/VR? What applications do you find interesting or surprising?

    - Satsuko