Preserving History in Interactive 3D

Award-winning speech for Andrew Jones, Senior Research Associate, Vision and Graphics Lab at the Institute for Creative Technologies

Imagine if you could meet anyone in the world — anyone. Who would you choose? What questions would you ask? And would you record the meeting so that years from now you could relive it?

As a computer graphics researcher at the USC Institute for Creative Technologies, my main focus has been to make faces, in both video games and blockbuster films, appear less computer generated and more real.

Back in 2010, my team and I were invited to a conference to showcase a project we’d invented — essentially, a floating head in a box.

Now, this display was cool for multiple reasons —

The first was the depth of the face could be seen without 3D glasses. Secondly, this was live teleconferencing where the head could look you directly in the eye, or she could turn away to look at the person next to you… just like in face to face conversations. In fact, it’s these spatial cues that made it feel like my 3D colleague was actually in the room — a feature today’s video chat still can’t deliver.

At first glance, most visitor’s reaction was, “Oh wow, holograms are finally here!” While others leaned in and asked her to repeat, “Help me Obi Wan Kenobi — you’re my only hope.”

Just as I was getting tired of hearing the same exchange over and over again… a woman from the USC Shoah Foundation grabbed my hand, looked me in the eye, and said “ .. would it be possible to record a 3D conversation… but with a Holocaust survivor? ”

And I thought, I don’t know … would it be?

All we had was this floating head that could only talk in real-time.

As a result, the most we’d ever recorded was 60 seconds. That is not much of a conversation. In order to have impact, we would need to display not just a face, but the entire body — head to toe.

And if that wasn’t enough, there was also a tremendous urgency to get this right because the remaining Holocaust survivors are well into in their 80s and 90s — within the next decade many of these firsthand accounts will be lost, forever.