There's an awful lot of science that goes into producing cutting-edge movie magic. Have you ever wondered just how they make digital characters like Benjamin Button? Apparently, the trick is all in the lighting. Paul Debevec of the University of Southern California's Institute for Creative Technologies knows this well, as he won an Academy Award in 2010 for developing the Light Stage technologies, which helped in the process of creating the Na'vi characters in Avatar. He invited me to visit his beautiful and brilliant "Light Stage X" to be captured and converted into a photorealistic digital character. To see for yourself, watch the video above and/or click on the link below for a full transcript. And don't forget to leave a comment at the bottom of the page. Talk nerdy to me!
CARA SANTA MARIA: What inspired you to build this light sphere?
PAUL DEBEVEC: We wanted to have a way of digitizing people's faces to turn a real person into a digital person that would be just as realistic as the real thing.
CSM: And what would be the need to do something like that?
PD: Well, we have more and more digital entertainment and simulations these days that need to depict everything you can see in the real world: environments, objects, vehicles, and people. We've all seen digital people in video games, like from you know, early Lara Croft that looked like a bunch of pixels and polygons. At some point, we're gonna need them to look like real people. And we wanted to build a device that would make it possible to translate that into the digital world as realistically as possible.
CSM: Coming in here, you know, with the cameras that are set up, all the lights that are set up, what's the process to capture my face?
PD: Well, the most standard thing that we do with this device right now is get high resolution digitized versions of actors. And so, what we typically do is we have somebody come on in here, like we are...
CSM: Mhmm.
PD: ...sit on the seat, we'll adjust the headrest so you can keep your head still, and then we're going to light you with 20 different lighting patterns in about two seconds (so, you'll hear some cameras clicking). And that'll give us all the information that we need to process to generate a 3-dimensional scan of your face ear-to-ear and get all of the texture maps and reflectance maps and everything that we need to render you under any kind of illumination afterwards.
CSM: So you mentioned that there have been many, kind of, incarnations of this light sphere. This is the most recent one, obviously. How long has this one been in practice?
PD: We call this one "Light Stage X" because we're basically up to number 10.
CSM: Oh wow.
PD: This one has been used for just about over a year at this point. The very first light stage that we had was nothing more than a 250 watt spotlight that was on the end of some plastic tubing and a bunch of 4x4 lumber that we got at Home Depot. And we had to pull it on ropes to get it to go to all of the different directions. We didn't have enough money when I was a postdoc at Berkeley to buy all these lights. People hadn't even invented bright white LEDs at the time, so we had to do it with this, you know, with this light bulb basically. It took 60 seconds to capture somebody, so we couldn't get particularly natural facial expressions. It was really hard to get eyes-open expressions 'cause you'd have this, like, 250 watt light in your eyes.
CSM: No, it's hard enough when it takes 2 seconds. [laughs] I can't imagine sitting there for 60 seconds without blinking.
PD: Exactly. Nowadays we're so happy to have these great white LEDs--so computer controllable. They will turn on and turn off in a millionth of a second. And we have complete control of the light from the entire sphere at the same time.
CSM: That's amazing. So where do you see this going? I mean, what does the future hold for this kind of technology?
PD: We're gonna see more and more of lead actors being used--using this kind of technology. At this point, if we can get somebody in 30 different facial expressions, we can actually rig up a digital face that can act with a lot of the subtlety of the original performance. We don't know how to have a computer create acting performances yet. We're gonna have to get better on our artificial intelligence for that (that's a long ways off, I think). But, we're getting to the point where these, essentially these digital puppets of people that you can drive with real actors' performances are gonna look totally photoreal, even in close-up, and relevant to the video game industry, even in real time.
CSM: [gasps]
PD: And here we go!
CSM: This is crazy!
PD: Let's take a look at 'cha.
CSM: So...
PD: This is actually you in 3-D, and...
CSM: Oh my god.
PD: ...we can change the lighting around. Now eyes and teeth are not gonna be perfect. What we'd do for a normal, true treatment is we'd actually model your eyes. These are just the eyes that we get from the geometry of the skin.
CSM: I mean, they look pretty damn good. This is nuts. You can see, like, just all the coloring and the freckles and--Oh god! [laughs]
PD: [laughs] So here we can do-- I actually turned the shine of your skin way, way down. But you can kind of see in here, this is sort of the level of detail that we get.
CSM: No, this is me! This is--I mean, it looks like a photo, a 3-D photo! But it's not. This is a completely digital recreation.
PD: It is indeed.
CORRECTION: An earlier version of this article stated that Paul Debevec won an Academy Award for his work on Avatar. In fact, his Academy Award was specifically awarded for “the design and engineering of the Light Stage capture devices and the image-based facial rendering system developed for character relighting in motion pictures.”
See all Talk Nerdy to Me posts.
Like Cara Santa Maria on Facebook.
Follow Cara Santa Maria on Twitter.