Deep Compositing

Have You Seen Those Amazing Composited Images In The Movies?
For a long time, the movie industry would hire visual effect houses to create these beautiful Scenes and add in elements to a movies story. Things like set extensions, overlays, masks and other sisual effects were often done by major names like Weta Digital, Animal Logic, MPC, ILM, FUEL VFX and others.


But what happens when after a shot is considered finished and now it's time to get the directors opinion? Often, when a shot is delivered to the main production studio, the directer with "I like the shot, but it would be better if..." "that character's face wasn't so dark?", or "if I could see through that dust cloud a little bit easier" or " if the camera were a little more to the left?" yadayadayada.....
Usually, these minor changes are the major headache for Visual Effect Compositors because they would make the small change. and have to render the entire scene to see if it was any better. (Just imagine how much time that would take on a major battle scene where hundreds of characters are fighting, and we need to just move one of the characters to the left!)

In 2010 a group of compositors reviled a remedy to this headache. After researching for a number of years, their solution is today known as Deep Compositing. In a Deep Composited image, every pixel in an image has not only a Red Green Blue and Transparency value, but also a depth value in the scene. This minor change makes the entire work flow unbelievably faster. The video below describes the concept:


Using Deep Compositing, making and update to a little part of the scene was now faster than ever before.
This Semester, I am taking an experimental Course in Maya Animation, and a majority of the curriculum is based on what technical achievement by some group of people made a program like Maya function the way it does today. In the same way I see that there are compositing techniques that are still being developed and seeing that research is still being done inspires me to continue my pursuits as an Emerging Media degree.

History of Pixar

Pixar Studio, originally a step child of Geourge Lucas has been popping up in conversations around me recently, so I decided I might look into the history of the creation of this animation studio.
Lucas Arts, an American film and television production company, was focused on innovative special effects since it was founded in 1971. Working on major motion pictures like Labyrinth (1986), Indiana Jones(1981), and the Star Wars Trilogy (1977-1983) Lucas film had a reputation as one of the most advanced special effects houses in all of Hollywood.

Lucas saw special effects as something to paint your films with. The technologies we had were the colors, and the effects were only the parts that added flourish to the story lines. Lucas had found himself always looking for ways to make his effects better, and cheaper.

Although Lucas was never "interested in computers" his company was one of the first to try using this new color in filmmaking.

At the time, in order to do any kind of digital image processing, you needed to find a way to get your image into the computer. Lucas was trying to solve the issue: "How do you get in and out of the Computer." digital Image processing was the promised land for filmmakers because not only would film scanned-in and manipulated in the computer be free from any generational loss (Negative Quality) but mistakes in the original photography (such as lighting on a blue-screen element that was inconsistent with its background plate) could be corrected without the costs and time delays of reshooting.

Lucas approached Ed Catmull in 1979 and asked him to head up a group to bring computer graphics, video editing, and digital audio into the entertainment field. With funds dwindling from Dick Shoup, a millionaire in New York (The majority of his money from starting the New York Institute of Technology) Catmull was looking for a way to incorporate, and continue research in 3D modeling. Remember at the time 3D was very basic, and thinks like texture mapping, lighting, and polygon manipulation were almost nonexistent. In 1979 Catmull became the Vice President at the seminal Industrial Light & Magic computer graphics division at Lucasfilm.

A few years later (1984) John Lasseter, also joined the Pixar team at LucasFilms after working on Tron (1982). 3D is a very expensive animation style, and at the time George Lucas was in the process of splitting funds with he recently divorced wife, so he was forced to cut funding to the Pixar group.

As Lucas's assets were being disseminated, Steve Jobs stepped in. Jobs had already turned down a position with the pixar team in 1984, and seized this opportunity to acquire the division instead. With new funding, Pixar found itself back on it's feet and became it's own company.

For more information, and a more in-depth timeline see: The Pixar Story

Star Wars Lava Planet Sequence

So shooting in studio this week got me thinking about how major companies do it, and so my post this week is a featurette from Star Wars Episode 3.  It's primarily about building the effects for the Mustafar sequence, a fight scene on a lava planet. While it is mostly an ego stroking for Lucasfilm and Industrial light and Magic, there are a lot of cool tidbits in here. There is a lot of cool stuff about the compositors and matte artists. The matte artists build the background for mustafar, which is a space hilariously based around a gravel parking lot, but obviously with a ton of assets and art. The assets that are involved in the sequence itself, because they are a giant mix of different sources. Some of the lava is from a shot of an actual volcano, some if it is built with particle effects. The use of models along with CG is also really interesting to me. ILM puts a lot of emphasis on making real rocks and setpeices because it can be easily manipulated in a real scene. So what comes down is what parts of the scene need to be real models and what can be made in CG, and someone sitting down to composite the two. The same went for the lava, a lot of it is done with real shots in a miniature set. Lighting and real effects like smoke are key, and then they just film it as their overall wide shot.

Clearly there is a ridiculous amount of elements here, but I just find it really cool how they all come together.

Before Modern Graphics

Stop motion is a well known form of animation that is used today in many shorts and feature films.  The process involves taking still images of characters and moving them slightly between shots to give the illusion of movement.  There is a draw back however, since the image is taken while the object s still there is no motion blur, and things look unrealistic.   That had to be over come for the technique to be used in a very well known and technically advanced film, Star Wars.  They wanted to film the TaunTauns, but wanted them to have motion blur.  So a new technique was invented by Phil Tippett and ILM, Go Motion.  This involves hooking the puppets up to computer driven rods that will move the puppet as the picture is taken creating the motion blur necessary for a realistic shot.  The technique is incredibly complicated, difficult and is not really used now since adding motion blur in post is much easier.  Older technology like this is incredibly interesting and makes one appreciate how easy effects today truly are.  

Jurassic Park

When I was little, my cousin used to be obsessed with dinosaurs, just like almost every other child. But when we would watch Jurassic Park, I would be scared to death thinking that the T-Rex used in the movie would break out from Hollywood's captivity center and come raging through my neighborhood. That's when my father had to explain to me that the dinosaurs used in the movies were not actually real, but instead animated robots with computer created effects.

Well 15 years later, and I'm learning the basic process of how these dinosaurs were created. The amount of work that goes into every scene with these creatures amazes me still, and I believe that with the technology available at that time, Jurassic Park is a beautiful example of how visual effects can make a movie. The team responsible is ILM, Industrial Light & Magic, who are known for working on many of the films we consider classic today, such as Raider's of the Lost Ark, E.T., and The Empire Strikes Back. In this article you can see some of the behind-the-scenes work with Dennis Muren, the creative director of Lucasfilm's visual-effects, and the incredible process that they would go through to create such imaginative worlds.

Now while I am truly amazed with the scenes that involve the Jurassic Park dinosaurs physically interacting with the actors, the scenes that you wouldn't necessarily expect to have CG were beautifully and artificially created and amaze me all the same.





I found a whole walk-through of the process in "Making of the Lost World: Jurassic Park" on YouTube, where the videos go into detail of the Pre-Production, Production, and Post-Production stages for Jurassic Park. These videos show the interviews with different people involved in the project and show how certain scenes were constructed on every level. The fifth video of the seven can be found here (embedding was disabled) and talks about how each of the dinosaurs was modeled and created.  But if you go back through each video, you can see bits and pieces of CG used in each scene.

I found these videos very interesting because they show the process of researching the dinosaurs, creating the models, using the live-action animatronics, filming the scenes that didn't have moving models, building the dinosaurs on computers, and combining the footage with the CGI.

The Lost World: Jurassic Park came out in theaters almost 15 years ago in 1997. Comparing what I know about computer graphics now and what was available back then, it's no wonder to me why Jurassic Park is considered one of the greatest examples of visual effects and why Industrial Light & Magic movies will forever be considered classic.