The worlds we can now enter with virtual reality technology are so stunningly authentic, it's hard to tell where true life ends and the virtual landscape begins. At the University of Southern California's Institute for Creative Technologies, researchers like Mark Bolas are dreaming up new ways to improve the immersive experience of virtual worlds. In the Mixed Reality lab, he and his colleagues are developing environments that not only look realistic, but feel realistic, both from a sensory-motor and a cognitive-emotional perspective. They have also developed an ingenious way to keep individuals who are fully immersed in a virtual world from running into walls and other obstacles that exist in the real world, by taking advantage of our own perceptual errors. And just this year, the Mixed Reality lab won an award for the FOV2GO, a 3D viewer that you can build yourself, transporting your smartphone into a handheld virtual reality experience.
CARA SANTA MARIA: This is crazy! Okay, so I see a Hummer.
MARK BOLAS: Great. So, between the Hummer and the left there, you'll see a little hole in the wall, then the gravel path. So, go ahead and step on the gravel path.
CSM: Mmkay. [laughs]
MB: And then, just walk down the path.
CSM: [gasps] Oh my god!
MB: Okay, so you've been down this path a couple times.
CSM: Yeah. I've moved, like, down. You know, looking at different rooms.
MB: Alright. So close your eyes for a second. Think about where you are on the path.
CSM: Okay.
MB: And I'm gonna take the virtual world off of you, so you can enter the real world here.
CSM: Okay.
MB: Alright, and now open your eyes.
CSM: I'm like, where I started.
MB: We can run you down this path multiple times, and you keep thinking that it's going down that way. But we're kind of changing where you walk a little tiny bit, so that we keep reusing the same piece of ground over and over again.
CSM: How are you doing that? I just walked all the way down this path. [laughs]
MB: So, when you entered the room, you went through a door. When you turned around to go back through the door, we moved the door from one wall to the other wall within the same corner.
CSM: So I didn't notice it?
MB: So, you turned around and knew you had to go through the door, and you went through the door. But, it was 90 degrees rotated. That 90 degrees lets us send you right back at the start and not sends you back right over here.
CSM: [laughs] That's brilliant! So I can see the Hummer with the gun on top.
MB: You can look over the wall. Look at the rocks on the ground.
CSM: Oh! This is so weird! [laughs] Okay!
MB: So, as a result, we can just make this path as long as we want in one axis, and you can just keep going down it.
CSM: Was that your idea?
MB: It was not. It's a thing called unintentional blindness, and Evan Suma is a postdoctoral researcher we have, and he worked on figuring out some of these perceptual tricks we can use in virtual environments. So, you can think about all of these optical illusions that we know about. What we're trying to do is figure out how can we use these illusions to create more effective spaces? So, for example, we literally have a 20 foot by 20 foot space here. Through a perceptual illusion, we can make it seem like an infinitely long space.
CSM: This is so cool. So this is where, kind of, cognitive psychology and neuroscience meets computer science?
MB: It's exactly that. You take an optical illusion and you replace it with a perceptual illusion, and then we can do all sorts of magical things in these virtual environments. I mean, really, this is just one big perceptual illusion when you really think about it. The fun part of doing science these days is you just, you can't be a scientist and be narrow anymore. When it comes to dealing with a person, we have to worry about what you would call haptic perception, which is the feet on the gravel. We have to worry about your visual perception, which is what you're seeing through the lenses. We have audio. We have how you feel about the space. We have your emotions to consider. So, it really becomes a whole new form of authoring environments. And, you're starting to see that mature in the video game area, and now you're starting to see it in displays like this.
CSM: So, this was really fun--kind of like I was a character in a video game. Is that really the main application of this, is for gaming purposes? Are there other real world applications?
MB: When you think about it, we can create any environment. We can make anything. So, the real question is, what is it you can't do in this environment? Looking at the big ones, medical training is obviously one. You can do one to have surgeons plan what they want to do before a surgery. The army would really like to be able to do simulations of a lot of the situations they find themselves in before they find themselves there. So, the applications really go from scientific visualization to medical visualization to all training applications.
CSM: This could kind of help train people to do anything, right?
MB: The next thing we want to do is start looking at training for high school kids in physics. You can't always afford physics labs in the schools anymore. But through a simple device, I could see mechanical engineering, I could see devices, I could see potential energy. Why not do things like that?
CSM: That's amazing. And chemistry too. We could be doing experiments that maybe aren't that safe to do within a high school chemistry lab. [laughs]
MB: Right. It is a bummer to lose the real explosion though.
CSM: Do you think that we are going to have these, like, in home, take-with-you kits in the future?
MB: We have take-with-you kits right now. I can show you some if you want, which we can put this sort of display (not quite as good quality), but just on a mobile phone, like an iPhone.
CSM: Really?
MB: Yeah.
CSM: Cool! Let's check that out!
MB: Okay. So here we have the phone, and you see it's rendering two images. So, that's how we get stereo.
CSM: Yeah.
MB: We just slide it right into this cardboard viewer, and then you've got your stereo world that you can look at any time you want.
CSM: So the way that you trick somebody into thinking that they're seeing in 3D is that you show an image to one eye slightly different than the image to the other eye.
MB: Right. The technology is just like the old 1800 stereoscopes. We use two images, one for the left eye, one for the right eye. We slip it in, and you trick the two eyes into seeing different images.
CSM: That's crazy.
MB: It's this. You can cut it at home with a razor blade. And then, these lenses you can buy. It's actually a dollar for two of them. You fold it up, and it folds right around. Here's one. It folds right around the phone. Drop the phone in, close it up--
CSM: You just slip the phone in. And it doesn't even have to be an iPhone? Like, any smart phone with a big screen?
MB: It works for Droids, it works for iPhones, and we're working on a Windows phone version of it.
CSM: I mean, this is almost as immersive as that big getup that I had. You can get lost in this really easily.
MB: And what's amazing is that the mobile phone is doing all the computing itself. We're really entering an age where the devices we hold in our hands can put us in alternative worlds.
CSM: Yeah, I mean, this is a computer.
MB: Right.
CSM: Yeah, this is insane.
To learn more about the FOV2GO virtual reality viewer and accompanying apps for your smartphone, click here.
See all Talk Nerdy to Me posts.
Like Cara Santa Maria on Facebook.
Follow Cara Santa Maria on Twitter.