An Open Letter To Andy Serkis

Dear Mr. Serkis,

If you deserve to be considered for an Academy Award nomination for Acting in regards to your performance motion capture, then every animator who has ever animated a character in any movie deserves consideration as well.

Sincerely,

Tim Borrelli

P.S., Let me clarify:

Recently, you have been quoted as claiming that performance capture actors deserve to be considered for the Academy Awards in Acting categories:

http://www.guardian.co.uk/film/2011/aug/12/andy-serkis-motion-capture-acting

Before I even start, let me say that I feel that you are a great actor. I don’t doubt your acting ability, both on stage and on film. But that’s not the debate here.

From what I gather, here is what you are suggesting. You seem to feel that performances like yours in Lord of the Rings (Gollum), King Kong (King Kong), and Rise of the Planet of the Apes (Caesar) should be recognized by The Academy as an individual effort in excellence of acting performance.

Wait. What?

Let’s ignore the fact that animators have been doing this without motion capture longer than you have been suiting up for it.

Performance capture is the digital capture of a performance of an individual actor, to be later applied to a digital character. Yet according to you, “…there are two parts to the process. The first part is capturing the performance. Only later down the line do you start seeing the characters being painted over frame by frame using pixels."

First, that doesn’t sound like an individual performance to me.

Second, painted over? Using pixels? For a guy who has positioned himself to be the spokesperson for performance capture, it sounds like you don't quite understand what goes into the entire process.

Ignoring the fact that there is nothing “being painted over frame by frame using pixels” (almost) anywhere in the process, you seem to be ignorant of what happens to your performance data after you walk off the set. Many times, chunks of data need to be thrown out entirely and done by hand. Also, it is quite often that the actor’s proportions don’t match that of the digital characters, requiring a remapping of the motion. This may not seem like it affects a performance, but it in fact does. Different proportions means poses don’t read the same. It means a slouch on a short actor is a hunchback on a tall character. It means delicate interactions often need to be heavily modified or redone with animation due to differing limb lengths. I could go on.

Long story short, it means the performance is not 1-to-1 from performance capture to screen.

Furthermore, you claim that "Performance-capture technology is really the only way that we could bring these characters to life… It's the way that Gollum was brought to life, and King Kong, and the Na'vi in Avatar and so on and it's really another way of capturing an actor's performance.”

You then go on to say, “That's all it is, digital make-up."

What. The. Hell.

Well, makeup artists HAVE an Oscar category. So are you also suggesting that the people behind taking his performance to the big screen be considered in that category? When you say “that success using the technique can be rewarded with current accolades,” is that what you mean? Should the modelers, animators, painters, shader TDs, lighters, etc., be considered for Makeup and Costume Design?

Makeup and Costume Design teams do amazing work. I just have trouble seeing how modelers, animators, painters, shader TDs, lighters, etc. fit into those categories.

Or are you referring to the VFX category (which, while valid, is a much broader category than acting), or even the lesser known, non-televised technology categories? Are you basically saying that your performance, which wouldn’t even be viewable without those aforementioned teams of people, is more deserving of public recognition?

I, as well as many others, won’t argue that motion capture data is only as good as the actor in the suit. I have directed and worked with motion capture data from actors on both ends of the talent spectrum. I agree that without the proper direction and performance, the end result that I produced wouldn’t be as emotional, as powerful, or as accurate.

However, I also know that without a talented digital character team (animators, modelers, TDs, etc.), that performance will NEVER look as intended.

What you've done here, Mr. Serkis, is downplay the contribution that the whole team makes to bring a character like Gollum to life. What’s worse is that you aren’t alone. In this featurette on the making of Rise of the Planet of the Apes, the animation team is completely overlooked!

Is the technology that Weta developed awe-inspiring and exciting? Hell yes it is. I’d love to be on set just for a day and see what the technology is like from start to finish. It would be amazing (and after writing this I may never get the chance). But to see the contribution of an entire discipline glossed over so readily by both a recognizable name (your own, Andy Serkis!) AND a production team is disheartening and frustrating.

Yet, as infuriating as that may be, this is not the point I want to make here. That point is:

If you deserve to be considered for an Academy Award nomination for Acting, then every animator who has ever animated a character in any movie deserves consideration as well.

Animators, both hand-keyed and motion capture artists, breathe life into their characters. They push performances of their characters to an artistic limit, based on the direction they are given. Many even use video reference- animators often of themselves performing (yes, ACTING) the scenes they are working on, mocap artists using video shot on set.

Not to single one person out, but some do it REALLY WELL, like this example (password: education):

Rio Comparison Reel from jeff gabor on Vimeo.

And this one:

It should be clear that this guy is an amazing animator. He’s also a great example of an animator using his own performance to bring characters to life (in the case of Rio, a female lead, and supporting male, and a bird.) As animators, we’ve been taught that video reference is a powerful tool. Like any tool, however, it requires training and practice to get right.

Some things may come more naturally (in a male animator’s case, the supporting male). Some things may take more creativity (like humanizing a creature, such as a bird). Even other things may take a bigger investment into the movement and emotion of the character (the female lead).

However, the end result in Rio didn’t come from just an animator’s performance. It came from the ability to translate that acting into what the digital character warranted.

Like you, Mr. Serkis, animators use their performance to improve and sell the characters they are acting for, in the interest of the whole story.

So my question for you is this:

Don’t animators also deserve individual recognition from the Academy for Acting?

Mr. Serkis, please leave a comment here, or drop me a line. I welcome the discussion, as would many others who do and do not share my opinion.

Designing Animation For Speed

This was cross-posted to AltDevBlogADay. Look for a powerlifting corollary sometime soon!

Everything I ever needed to know about animation, I learned from watching Looney Tunes.

Seriously.

Sure, I went to school. I learned HOW to animate there and from other sources, but none of that taught me what I needed to know about animation as much as watching Bugs outsmart Elmer, Daffy play the second banana, Foghorn Leghorn fumble about, or Wile E. Coyote fall to yet another demise.

Gravity Doesn’t Exist Until You Look Down

Bugs is the popular one. Daffy is the memorable one (Stuuuuuuupor Duck!). Foghorn, well, I say…  I say, he was fun to watch with his contraptions. However, Wile E. Coyote is the one that sticks out the most to me.

Not only is it entertaining to watch poor Wile E. try and try again to catch the Roadrunner (with a seemingly unlimited credit line with the Acme corporation), it is also a very good study of how developing a formula can allow an animator to work efficiently while keeping a high quality bar.

Take the timing of Wile E.'s fall, for instance. Did you know that every time he falls, it happens in the same exact number of frames? He tends to walk/fly/float/propel himself over a ledge, and doesn't fall until he looks down (for Wile, gravity doesn't exist until he does this). He then falls over N frames until you see a puff of smoke and hear a distant "splat."

This comedic timing (combined with a happy accident regarding the audio level of his "splat") allowed the Looney Tunes animators to build up to this punchline without worrying if the punchline itself would feel right. It also ensured that re-takes weren't going to be needed- once that formula was set, it was followed.

Set Your Pace

For game animation, I take the same approach, especially for transition animations. In fact, on our current project, we try to animate all transitions to occur on 6, 12, 18 or 24 frames. Each of these frame counts corresponds with the desired perceived pacing of the character's thought process. For example:

"Ready Now!" - 6 frames

"Get moving and STOP!" - 12 frames

"I gotta be ready before I get there" - 18 frames

"OK, I'm gonna pick my spot and get ready when I'm there." - 24 frames

These frame numbers allowed us to build a visual language with these transitions. For example, the faster ones are more immediate and needed for faster gameplay responsiveness (say, for aiming and shooting). Having them all on a 6-frame count holds the player's expectation that if “I've gotta shoot, I'll get there in .2 seconds.” It also allows us to later add interruptible follow-through for an extra 6 or 12 frames without that extra motion feeling out of place.

In contrast, the 18/24 frame transitions typically denote the end of longer, deliberate movement- the player knows they are moving at a certain speed and when they have reached their destination, the transition will carry that speed to a stop. These extra frames allow us to sell the character's weight at that stop, which in turn creates a mental acceptance by the player that it took longer than the 6/12 frame motions.

Fast Iteration

Sweet, sweet consistency

The next benefit to the frame counts is that they are easily scalable from one to the other. Personally, I set poses every 3-6 frames. This lets me easily slide the keys around if I need to make timing adjustments after the animations are implemented. These adjustments can also be easily scriptable (for fast tests), so I can adjust the timing of 160 animations very quickly to see if the new length feels right before going ahead and hand-tweaking them.

Since I've spaced out my keys in a uniform way, I can re-time the motions much more efficiently.

Lastly, these frame formulas give us efficient consistency. Since a uniform length exists, once we nail down the first key animation, we can modify it to create other motions in the same pool. For example, If we need to transition from a non-shooting pose to a shooting pose, we’ll work on a key motion that gets across the character’s intent and the player’s feel. Once design deems the look and feel to be good for gameplay, we can create every other transition in that pool much quicker.

Happy Accidents

Much like the audio level on the splat of Wile E. Coyotes's fall (see the book Chuck Redux), coming up with the multiples of 6 formula was actually a happy accident- the Source Engine's default engine blend value is .2 seconds (6 frames!) In order to not disrupt existing gameplay, we started animating to that frame number. After implementing many transitions, it was apparent that not all of them could or should work on 6 frames. The easiest thing to do was scale the motion by 200% and work back from there. Lo and behold, 200% worked great, so we redid all of those motions with twice the frame count.

Since that pacing looked good, we extrapolated that 18 and 24 frames would be logical to try next. It worked, and now we have a (undocumented till now) visual language for our transitions. We've since extended this philosophy to our movement system, and the animators who work on that pump out high quality motions like nobody's business.

It's Still Creative!

Purist animators out there might be shocked that we take shortcuts in order to produce efficient, high-quality animation at a fast pace. The fact of the matter is Warner Brothers, and even Disney, did the same thing! Read Chuck Redux, or watch this:

The key is to not get so married to those formulas that we are unwilling to change them when they no longer work, or get stale. If that happens, we play with new formulas and do more timing tests until we get it right. In the end, the better we can nail down the punchline, the more time we can spend on the fun part of getting there.

Feedback!

I'd love to hear from other animators and designers what their processes for this are, as well! So speak up, tell your friends to read this and rip me to shreds, tell me how awesome I am or somewhere in between. I'm always looking for different techniques and opinions!