How digital characters replicate human action and emotions – Motion Capture

1 November 2017

Fiction and fantasy go hand in hand. It is impossible for fiction to exist without often drifting to imaginary worlds or situations. If every fiction narrated only tales of regular human characters, it would be boringly repetitive and would have long lost its popularity. All master storytellers are aware of that and that is why today we are never short of aliens, talking animals or fantasy creatures. Switch on your television any day and unless it is a day of total power outage you will find at least one movie or show that feature such a story world! They are a regular in our living rooms today. Along with the story world we are quite often mystified by the remarkable humanness in such digital creatures. The best of such characters, though out of this world, still retains a human nature. The reason is quite simple. One might be dealing with fantasy creatures, but a smart filmmaker understands the people who watch them are human beings and not aliens! For human beings to relate to a character, there has to be some human shades in the character. But how is this achieved? How are fancy creatures given a human touch?

Motion Capture, face capture, emotion capture

You might have seen stage acts where a performer carries out the actions exactly like in the footage behind him. Well.. replace the footage with a digital image and the stage to a live action performance and then we will have what we call the Motion Capture or Mocap in short. To explain, motion capture refers to recording live actions of human actors and then using that information to animate digital character models in 2D or 3D computer animation. Quite simply, it transfers the movement of an actor to a digital character. In other words, that scene of monsters attacking a bus that made you skip a beat was actually performed by human actors on set! 

Earlier, digital animations were drawn by hand using a method called keyframing which was based on the notion that only the key conditions describing the transformation of an object needs to be shown and all other immediate positions can be figured out from them. For example, imagine Tom and Jerry. How often have seen them taking a single step and in the next instance we are shown where they have reached. This way, we can assume the conditions that happened before they reached their spot. As time went by that process got automated and that is Motion Capture. 

Motion Capture was initially introduced towards late 1970’s to facilitate biomechanics research but it later expanded to sports, video games, movies, and television. To facilitate motion capture, from the beginning, LED, Acoustic, inertial, magnetic or reflective markers are worn by the performer near each joint and their movements are tracked at least two times the frequency of the desired motion. The prosperous technological era of 21st century saw several changes in the motion capture technique and now, systems can even obtain a silhouette of the performer from the background and by connecting a mathematical model to the silhouette all the joint angles can be calculated.

Motion capture systems that make use of tracking cameras are called optical while systems that use inertia or mechanical motion are non-optical. Optical systems track position markers or features in 3D and then assembles it to the semblance of the actor’s motion. Active Optical systems use markers that lights or blinks in a unique way and in this a battery or charger pack must also be worn by the performer. Passive optical systems are the most commonly used method in the industry and they involve retroreflective markers that are tracked by infrared cameras. Optical systems can also be markerless and this technique depends on software to track the performer’s movement. Once tracked, the motion is then plotted onto a virtual skeleton of the animated character. Inertial technique requires computer only as a localization tool. The performer wears inertial sensors and the data from sensors are wirelessly transmitted to a computer.

But a character is not just about movements or actions. Just moving around like a human will not be enough for us to reach out for the character. We did not like Caesar in Rise and Dawn of Planet of the Apes, The Hulk in Avengers or Gollum in Lord of the Rings just because their movements had some resemblance to humans. Did we? It’s not just the flesh that matters. It’s who they are inside that really counts. We want to see how they express their compassion, anger, fear, joy, tension, disappointment and all other set of emotions. We love actors who time their emotions perfectly onto the face, don’t we? So, it is quintessential for any digital character to emote their feelings with humanness. This is achieved through facial motion capture.

As the name implies facial motion capture is electronically converting the movements of a person’s face into a digital database. Facial capture can be two dimensional or three dimensional. Two dimensional capture is also called expression tracking but to fully capture movements such as head rotation, three dimensional technique has to be used. Like in motion capture, facial capture can also be marker based and marker less. Remember Tom Hanks in Polar Express? That was an example of marker based facial capture. In this technique, upto 350 markers are applied to the actor’s face and the marker movement is tracked using high resolution cameras. The markerless technology is a vision based approach that tracks the minute features of the face such as wrinkles, nostrils, pupil movement, eyelids etc frame by frame. This technology is almost similar to that of Facial Recognition System. Though the technology is quite advanced, it still needs an efficient performer to deliver it in the most powerful manner. It is the performer’s act that gets transferred to the digital image and thus the performer has to be extremely careful about his minute reactions. As someone once said, 90% of directing is casting and it is true for most CGI heavy films too! 

Seemingly impossible things have now become a regular norm due to technological breakthroughs and this has dared storytellers to push all their limits. Fictions can be set anywhere and it can be about anyone. One just needs the right creativity to imagine stellar characters and worlds. And when you have such creativity and vision in you, technology is there to let you do anything. Some day, when aliens take over the inhabitants of Earth, they might stumble upon the movies humans made about them! Marveled by the creativity they might just let us go. Now that’s a story!

Recent Posts

45th Annie Awards – Winners of the World Animation Industry

5 February 2018

Image source: Annie Awards Forty six years ago, renowned American voice


Visual Wonders Decoded: VFX Festival 2018

20 January 2018

There are quite a number of film festivals and game