Professional Blog

 

I have seen the face of the future, and it was blue. For the first time, the facial performance capture technology in James Cameron's movie Avatar made computer generated image (CGI) characters come alive. The ability to capture the nuances of facial expressions, gestures and body movements and transpose them onto CGI avatars made me forget they weren't real. The spectacular, nearly photorealistic, 3D scenery and lifelike action were a plus, but it was the realism of the faces that brought the animations to life. The CGI characters came alive with the nuances of the actors' performances.

Like the "Jazz Singer," the first major full length film with sound, Avatar is a total game changer. The first time I saw it, all I could think of was how it marked a turning point for the film industry. Before long any actor may be able to appear in any body they need and expensive location filming will move to the sound lot and processing cloud. But somehow it seemed to me that the change was even more fundamental.

It took me two more times seeing the movie before I realized that it wouldn't just change the films, but will revolutionize the way we interact each other and the world around us. In the mid 90s I worked on a project to use 3D and VR (Virtual Reality) technology to create a Virtual Command Center. It offered a great way to integrate the presentation of information, with a shared sand table in the center, and various real time data displays on the surrounding walls. Avatars of the command team members from different locations could stand around the table and work together interactively. For its time, it was visually quite impressive; but it soon became clear that it had a critical flaw.}

The users complained the virtual command center couldn't replace face to face meetings. They needed to see each other in real time. They felt they couldn't adequately communicate without being able to seethe nuances of facial expressions, gestures and posture. It was obviously far beyond the capabilities of the current technology, and the project was soon abandoned. For the next 15 years the only alternative to face-to-face meetings were video teleconferences and webcams that lack the intimacy and realism of a shared 3 dimensional environment.

When I made the connection between these new Avatars and our Virtual Command Center I realized that the major impact of the performance capture technology was not in films. It isnt even in the Avatars in the 3rd person online games, although these will become amazing. The future is the social media; in particular in its impact on first person online collaboration and communications. It can enable people to come together in an immersive, information rich, virtual environment. Users will be able to observe this environment as if though their own eyes, as they are sitting or standing among the other participants.

There is a confluence of technologies of technologies that may make this possible in the very near future. These include broadband internet, facial and gesture recognition software, multi-cored GPUs (Graphical Processing Units), 3D capable digital TVs and monitors and cloud computing. A simple high definition web camera can capture motions and gestures of participants seated at their desktops. Facial recognition software can identify the key points on the face, shoulders and arms and map them to 3D models. For each user these coordinates of the key points in the models can be passed to the 3D environment models and renderers in a service cloud. The gestures could be used to track the users point of view and as input devices for manipulating the virtual environment. They, in turn can send either stream custom user point-of-view (POV) video to each user or send model data points to graphics renderers in the users computers.

This will not only revolutionize video conferencing, but also the social media. The Facebook and texting generation (like my teenage daughter) have virtually abandoned the one-on-one experience of the telephone for the many-on-many of virtual chat. They do this even though short one liners offer a very impoverished information flow bandwidth. A 3D virtual chatroom with lifelike avatars and 1st person POV will definitely put the "social" into the social media big time.

This isn't just a pipe dream. The technical pieces are falling into place and commercial applications may only be a two or three years away. It may be even sooner for some critical big budget applications. The pornographers probably won't be far behind. Social websites like YouTube and FaceBook may take a little longer if a new generation of graphics hardware is needed on the client computers.

It won't be long before "I see you" in our own virtual world.

I'll be the one looking like Errol Flynn.