Enabling 3D Moviemaking

By Lee Purcell

Producing animated feature films presents daunting challenges for storytellers, but not nearly as daunting as those posed by stereoscopic 3D (S3D) moviemaking. Whether dealing with special effects in a film with live actors or computer-generated character movements in an animated feature, many of the most difficult computing tasks are literally doubled—to support two individual streams of images, one slightly offset from the other, creating a sense of depth.

With the release of Autodesk® Maya® 2009, the developers at Autodesk have generously equipped the latest version of their 3D design and animation software with features and capabilities to enhance stereoscopic 3D digital content creation. In support of evolving animation pipelines and the advanced physics algorithms that underlie special effects, the Autodesk development team capitalized on next-generation multicore processing advances from Intel and engineered their code to use parallel threads for boosting performance and efficiency. 3D moviemakers enjoy faster rendering, higher quality previewing, richer end results, and additional perks that enhance creative storytelling.

Enabling moviemaking in S3D requires rethinking many of the cinematic conventions that have held sway for decades and then building tools and processes that help storytellers exercise their craft. With Maya 2009, the innovators at Autodesk have done just that.

Growing Studio Commitment to Stereoscopic Entertainment
Studios have demonstrated a growing interest in S3D production, particularly as moviegoers have flocked to theaters in record numbers to watch 3D films. Kevin Tureski, product director at Autodesk, said in an exclusive interview with Intel® Visual Adrenaline, "We've been working very closely with the leading studios creating stereoscopic films. These include Sony Picture Imageworks, the studio that created Beowulf; Disney Animation, the makers of Meet the Robinsons, Chicken Little, and Bolt; as well as DreamWorks Animation, the company behind the upcoming Monsters vs. Aliens."


"The quality of the content in stereoscopic productions will very much depend on the filmmakers," said Sebastian Sylwan, senior industry manager for film at Autodesk. "One of Autodesk's main drivers," he continued, "is to give our customers what they need to make the best production possible. All the enabling technologies—and the current resurgence of stereo filmmaking—are really driven by a particular alignment of knowledge: knowledge about digital cinema, single projector stereo, and digital projection techniques. Many of the technical hurdles that were in the previous iterations of projecting and creating stereo, as well as obstacles with digital cameras, are being intrinsically solved by these technologies."

Efforts coordinated by industry groups, including the Digital Cinema Initiatives LLC ( www.dcimovies.com ), are paving the way toward wider distribution of 3D content. On the home front, Intel is actively working with companies, including DreamWorks Animation SKG, to bring premium stereoscopic entertainment to audiences through big screens in the movie theater and computer screens in the home. Another venue where 3D continues to gain momentum is computer gaming, in which immersive worlds acquire an extra measure of reality with the addition of stereoscopic depth.

Strengthening Workflow with High-Performance Platforms
Through close collaboration with Intel, Autodesk developers have tuned their product performance to the platform capabilities of multiple processing cores, multi-threading many of the most demanding, data-intensive tasks (of which there are many in the 3SD pipeline). Supporting two separate streams of digital image data to support the stereoscopic model clearly requires a truckload of processing power. Intel and Autodesk found many areas where Maya's 10-million lines of code could be refined to take advantage of parallelism.

"One of the things that has been great—now that Maya is part of Autodesk—is that we can take a pipeline-wide view of stereoscopy," Tureski said. "Instead of just providing a point solution in a single product, we've taken a very holistic, productline, wide view toward stereoscopy. We made sure that data Maya creates can be taken into Autodesk Toxik, our 3D compositing system."

"So Autodesk offers a nice seamless stereoscopic workflow between your 3D package, your compositor, and even our color grader, Autodesk Lustre®," Tureski explained. "We made sure to add stereoscopic capabilities to all of these products, so that when you're working in stereo you're not just working in 3D. You're not just compositing. You're not just color grading. Being able to carry stereoscopic data between all these different tasks is required to provide the best experience for the audience. Film studios need this stereo continuity to manage the complexity of the extra dimension."

Phil McNally, the global stereo FX supervisor at DreamWorks Animation, said, "Using Maya, we have developed a 3D toolset and workflow that gives our artists a highly interactive environment for working in stereo. Using the various 3D display systems at DreamWorks, artists are able to view in stereo throughout the production pipeline."

Stereoscopy, or stereoscopic imagery, uses the characteristics of human binocular vision to create the illusion of depth, making objects appear to be in front of or behind the cinema screen. The technique relies on presenting the right and left eyes with two slightly different images, which the brain automatically blends into a single view. Subtle right-left dissimilarities in the images create the perception of depth and can be manipulated to creative advantage. Therein lies the art of stereoscopic filmmaking."

— From Autodesk® Stereoscopic Filmmaking Whitepaper: The Business and Technology of Stereoscopic Filmmaking

"In Maya," McNally continued, "we can interactively control the 3D effect and see these changes happen in real-time. We can quickly adjust the scene composition and even adjust a character to be in front of the screen or behind the screen-all in a highly interactive environment. The example would be something like: if you're a sculptor working in clay, now we have our hands on the clay as opposed to being in some remote room writing up parameters of what we want the clay to be like."

The prevailing trend in today's animation pipelines—which are undergoing rapid evolution in the industryleading studios—is to integrate processes throughout the pipeline for more fluid workflow and greater visualization possibilities. Performing complex operations concurrently helps speed overall workflow performance. Advanced multi-core processing architectures, as provided by Intel® Core™ i7 Processors enable the performance-driven platform capabilities that are revolutionizing digital cinema production techniques.

"In the past," McNally explained, "what you were trying to do is take the idea of the storyboard and create an animated version of that storyboard, which supports the same compositional idea. The choices include which lens to use and what's the actual framing of the shot going to be. That's a very two-dimensional workflow—based on drawn images in a two-dimensional view. Even though we're in a 3D world, it's still a two-dimensional view of that world. What we're trying to encourage now is different. Although the storyboard is still there as the basis of the idea, in Maya the space becomes three-dimensional. We're suddenly within a spatial world and we can compose spatially. Ultimately, audiences will see this as a spatial composition. It's no longer going to be seen as a flat 2D composition."

"If you think of three dials with the lens choice, framing, and stereo depth," McNally said, "those are the three things we're trying to achieve a more fluid balance among."

"Giving our film makers these highly interactive 3D tools is enabling a whole new element to story telling. For DreamWorks Animation, we couldn't be more excited about the potential for 3D," said Ed Leonard, CTO, DreamWorks Animation.

Monsters vs. Aliens, scheduled for release by DreamWorks Animation in March 2009, builds on the stereoscopic techniques that have been refined over the past months, inspiring the latest capabilities in Maya 2009 for pre-visualizing and previewing scenes on 3D-capable display systems or projectors during editing.

Maya can also support computer-generated effects that rely on content imported from live image capture. Sebastian Sylwan, senior industry manager for film at Autodesk commented, "If you have a live-action shot filmed with stereoscopic cameras, you can track the cameras and apply them inside Maya, or inside a compositing package, and then composite them together. The next step for us in the pipeline is Toxik, which actually has just been introduced with stereoscopic features. Toxik can read the Maya cameras (that have been tracked or processed to identify the camera moves) and produce those same camera moves onto the CG element. That element can then be brought into Toxik and composited with live action footage that comes from cameras."

"These systems are designed to be extremely flexible," Sylwan said. "This is one of the reasons why partnerships with our clients are extremely important: because almost every project has a specific workflow that needs to be followed. We need to be very aware and very active in supporting those workflows and enabling the client. One of the main capabilities of both Maya and Toxik is to be extremely extensible and extremely flexible—to be very adaptable tools for pipelines. That's why big facilities, our largest clients, have built their pipelines around these tools."

Pre-Visualization Adds Creative Flair to Productions
Under the direction of DreamWorks Animation CEO Jeffrey Katzenberg, every film produced by the studio will be produced in 3D beginning with Monsters vs. Aliens in 2009. At this year's SIGGRAPH, DreamWorks Animation proudly showed off the 3D test sequence of Kung Fu Panda as a taste of future CG film making. The expertise that DreamWorks Animation is acquiring is being reflected in a new pipeline designed to better accommodate stereoscopic workflow and new tools that have been created in collaboration with Autodesk to better explore the creative possibilities of the medium.

Unlike traditional animation, where the workflow follows a very serial process, larger studios are increasing the use of digital design tools to perform pre-visualization of scenes. Pre-visualization allows the luxury of being able to explore the effectiveness of different approaches to a scene and decide what works and what doesn't before going deeper into the animation work. Reporting from SIGGRAPH 2008, 1 Anne Hall notes that DreamWorks Animation formed a Pre- vis Department for the creation of Kung Fu Panda.


"The traditional Ô2D' animation approach of hand off from Story/Storyboard Dept. to Layout Dept. to Animation Dept. is extremely regimented. This was necessary when bringing a scene to life required hundreds of labor-intensive drawings and handmade background paintings, but in a fully 3-D feature where the cameras and characters are comparatively easy to move around it seemed unnecessarily ," Hall commented. " John [Stevenson, the director of Kung Fu Panda] seems to agree in his insistence that the opening fight sequence of Kung Fu Panda benefited greatly from their ability to use Pre-visualization techniques to choreograph that complex sequence."

"More and more studios," Hall continued, "seem to be adopting previs into their animation and layout process, changing cameras and angles to work with new ideas directors and animators come up with on the fly. It will be interesting to see how this new process will affect filmmaking in the future. If Kung Fu Panda is any indication, it could quickly become part of a new standard in the creation of animated films."

In stereoscopic works, this type of flexibility extends to determining how effective the stereo effects are, providing cues to the creative director as to camera movements, intraocular distances, and scene composition.

Gauging the Popularity of 3D
Nick Dager, who produces the Digital Cinema Report* ( www.digitalcinemareport.com ) and keeps a thumb in the wind on new developments, is unwavering in his optimism for this new technology. "I believe stereoscopic 3D is going to transform the entire movie experience from top to bottom," Dager said, "and, in fact, it will cause a greater change than the introduction of sound. The kinds of movies that are made, how they are made, and the intensity of the movie-going experience are about to get bigger and better than ever."

"I strongly suspect," Dager continued, "that once the public at large has a taste of this, people will start demanding more stereoscopic 3D than they already are. The success of the Beowulf example wasn't atypical. So far every digital 3D movie that has come out has done at least a factor of 2.5 times the box office return of the same film in 2D."

As long as the public continues to demonstrate their interest in S3D movies, voting at the box office through ticket purchases, the studios will continue to improve their storytelling skills in this format. Tools such as Autodesk Maya, with integral application features that make it easier to design, pre-visualize, create, and render 3D, help drive down production costs and enhance the creative edge, giving animators and moviemakers a fresh palette of techniques to hone their craft. As interest in 3D expands into gaming and television, opportunities will likewise blossom for those innovators at the peak of their game. Digital content creators looking for a new medium to reach audiences could discover that 3D is the way to go.