‘FlashForward’ Flashback: Zoic Studios’ Steve Meyer on the Award-Nominated VFX for the Pilot

Posted by Erik Even in I Design Your Eyes on April 1, 2010

Property of ABC; screencap from the Zoic Television Reel.

Based on the science fiction novel by Robert J. Sawyer, ABC’s FlashForward tells the story of the aftermath of a bizarre global event. For 137 seconds, every person on Earth (except perhaps one) loses consciousness, and experiences visions of their own future.

The pilot episode presents the immediate aftermath of the worldwide disaster, with the consequences of the worldwide blackout – millions of deaths due to traffic collisions, crashed aircraft, and other accidents. Star Joseph Fiennes, portraying FBI agent Mark Benford, survives an auto wreck and looks out over a chaotic Los Angeles cityscape. Culver City, California’s Zoic Studios was tapped to create the disastrous tableau; the company’s work on the episode was nominated for two VES awards, for Outstanding Supporting Visual Effects in a Broadcast Program and for Outstanding Created Environment in a Broadcast Program or Commercial.

Zoic VFX Supervisor Steve Meyer discusses the creation of the complex scene, which required a tremendous amount of rotoscoping and motion tracking. The amount of roto was necessary, Meyer says, “because they shot in downtown LA, looking from the 4th St. overpass, over the southbound and northbound 110 Freeway. So naturally you can’t stop all that traffic, or get a greenscreen up.

“There’s a big, swooping hero shot following Joseph Fiennes as he jumps up on a car and looks down, and it’s a huge vista. Then we reverse it and look northbound. Everything in the foreground on that overpass we had to roto out. We had a team of seven or eight people going for weeks, just rotoing that and a bunch of other shots.

“We ended up having to remove all the traffic on the freeway; then added in overturned cars, cars burning, flames, smoke, helicopters crashing, just debris everywhere. We had to build a 3D matte painting in that environment. There’s a lot of detail – you can look at the shot over and over, and always see something new.”

The production brought on an experienced feature film matte painter, Roger Kupelian (2012, Alice in Wonderland) to create a “road map” of the shot. “Kupelian took a still of the freeway overpass shots, and he just dialed it in with Kevin Blank, the VFX supervisor – we want smoke here, we want the helicopter to hit here, we want fire and destruction here, we want this tree burning. They gave us a template – this is what we need it to look like. Kupelian sent us the files, and sometimes we used his elements.

“There are about 45 shots we ended up doing for the pilot, and the majority of them were for that overpass sequence. For most of the shots, nothing was locked off – every shot had some sort of roto, because there was no greenscreen. We had to roto everything to build the shots, and then try to match the smoke, fire, debris, people and other elements from shot to shot.

“We were working with different formats — stock footage, film footage, the Red Camera — trying to mix all these different formats to create one environment.”

All the scenes were tracked in Andersson Technologies’ SynthEyes camera tracking software, from which the team was able to build a 3D environment. The artists used this 3D information in Adobe After Effects to rebuild the plate with clean pieces of freeway, overpasses, signs, etc.

“The smoke was a combination of digital photographs, Google images, CG smoke, and moving elements that we had in our vault. Some of the smoke was a dust cloud that we slowed down. One of the smoke passes was a photo of a brush fire I took up by my house with my iPhone. I took the image, gave it to one of my compositors, and said ‘this will look good off in the distance.’ It’s so far off you don’t see it moving, so it fit in fine.

“They shot lots of people on greenscreen, and they all needed to have the right camera lens perspective. They’re way off in the distance; you have to get up close to an HD screen to see them. But we didn’t want any nuances to be overlooked. We don’t want to shoot a person head-on when the camera is going to be looking down at them.

“Another complication with the overpass sequence was that it was shot on a bright day, so we had a lot of technical problems. If you look at someone up against a bright sky, the sun wraps around them a bit, like a halo – and we were trying to put a dark smoke cloud behind them. It just doesn’t work right. We had a lot of technical things to try to work through when we ran into those kinds of problems. Every shot had to be 3D tracked. We took that 3D track into our environment, and we placed things in our 3D world shot by shot by shot.

“We also had a CG tanker in there that blew up. They actually had a real tanker with a big hole in it, and they threw in six gallons of gasoline and lit it and boom! It was huge. We had to put the shell of the tanker on there before the practical explosion; and then we just blew it up in Autodesk Maya and added CG debris, camera shake, heat ripple and dynamic smoke trails; plus glass shattering on the buildings and other background effects.”

Property of ABC; screencap from the Zoic Television Reel.

Wreckage from the tanker explosion strikes an overturned car and knocks it off the overpass onto the freeway below. Zoic created the car in CG. “They shot everybody running up to the guardrail and looking over,” Meyer explains. “We had to remove the railing and put in our own CG railing, so when the car goes down it takes it with it. So we had the complicated roto of recreating the people’s bodies that were behind the railing. We had to rebuild lots of people’s legs and waists. We put in the smoke and stuff that dynamically reacts to the car, so when it gets sucked down it creates a vortex and pulls the smoke down. Also, there’s an orange cart right nearby. We try to get every detail right, so when the car goes down we have a couple of oranges that roll away with it.

Property of ABC; screencap from the Zoic Television Reel.

“Then we had the falling LAPD helicopter. We took a panoramic image of a building, so we’re working on one frame and can do a pan-and-tilt in post. The helicopter has already crashed into the building, and we needed to have the smoke barreling out through most of the sequence, with the rotor blades still spinning — then we get to a certain point, and there’s an explosion that pushes the helicopter out. It tumbles and it’s scraping the building, tearing it apart and opening it up.

“Roger Kupelian labored intensely on a matte painting of the inside of the building, with what would be exposed – wires, beams, pipes, office equipment. As the CG helicopter was falling down the face of the building and opening it up, our compositor just revealed it with little mattes. At the same time we threw in sparks, debris, dust and smoke. It ended up pretty good. It was tough making an animation that made everyone happy, but in the end it looked great on the big screen at the viewing.

“Some of the shots were fun, because you can really push the envelope — let’s see what happens when we do this or when we do that. It took a lot of planning and careful choreography to between our 2D and 3D teams to keep the action and look continuous. Our teams worked tirelessly to create seamless product because anything out of place would be glaring.

“This isn’t ‘sci-fi’ with spaceships and aliens,” Meyers says, “which allow a bit of imagination – but rather, real-life vehicles, smoke, fire, people and buildings that have to look real.”

More info: FlashForward on ABC.com; the latest Zoic Studios Television Reel on ZoicStudios.com; Steven Meyer on IMDb.

Zoic Brings Photo-real CG to Broadcast TV with ESPN NASCAR “Dominoes”

Originally published on IDesignYourEyes on 2/2/2010.

ESPN NASCAR "Dominoes" spot

To the opening riffs of Metallica’s “Master of Puppets,” two NASCAR drivers jostle for position at the front of the pack. One cuts off the other by the wall, and the rear car speeds up, smashing into the front car. As the front car drifts from the wall, the rear car makes its move, attempting an aggressive pass on the right. But it’s no good – he sideswipes the front car and spins out. He’s slammed by another car and flips high into the air, triggering a massive pile-up. And straight through the smoke and chaos of the pileup – a third driver makes his move and takes the lead. “It’s anybody’s race.”

The 30-second spot for ESPN (see it here), promoting the NASCAR Nationwide series, was created by advertising agency Wieden+Kennedy New York and Culver City, California’s Zoic Studios. The commercial is significant because, despite its unique and stylized black-and-white look, it appears to have been shot in live action. In fact, it’s entirely CG.

Zoic co-founder Loni Peristere, who directed the spot, talks about why the commercial was created digitally, and how Zoic was able to create the illusion of perfect realism.

“The question from Wieden+Kennedy was, ‘we have a project, two scripts, which take place on the track, and would require significant action and stunt work. We’re trying to decide whether we should approach this from a live-action standpoint; or should we approach this from an animation standpoint.”

Wieden+Kennedy insisted the final product be photo-realistic; the agency did not want a commercial that looked like a video game.

But Wieden+Kennedy was insistent that the final product must appear perfectly photo-realistic. Peristere says the agency did not want a commercial that looked like a video game. “It was really important to them that it had the energy, grit and testosterone of the track. They were not interested in making a spot that didn’t have the reality of NASCAR.”

The agency was well aware how far CG realism has recently progressed. “Even in the last 12 months it has come a long way,” Peristere says. “With the advent of motion pictures like Avatar or The Curious Case of Benjamin Button, we are seeing the potential for photo-real characters, photo-real environments, and photo-real action. But could we actually achieve that for a commercial, and could we afford it? What would the timeline be?

“We got boards for both spots, and it became readily apparent why they were even asking this question – they had a 40-car pileup in the middle of the first spot, and a pretty significant crash in the second. Now when you looked at the second spot, you thought ‘well, from a production standpoint you could probably pull that off’; in fact we’d done something similar for Budweiser the year before. But the 40-car pileup featured just an enormous amount of damage to an enormous number of vehicles, which from a production standpoint would be very expensive.

“And the ability to control the lighting and the camera and the art direction would be limited in a live action production. You would be fighting against the sun, making you rush through the shots, allowing you limited control over your color palette. And you would have the expense of wrecking an enormous number of vehicles.”

Peristere discussed the project with other principals at Zoic – fellow co-founder Chris Jones, commercial creative director Leslie Ekker, commercial executive producer Erik Press, and CG supervisor Andy Wilkoff. “We thought it would be fun to rise to the challenge,” Peristere says. “We knew the team we had been building over the last several years had the potential to do incredible photo-realistic work. We’d seen large leaps in the realm of photo-real characters. We came back to Wieden+Kennedy and said ‘yes, yes we can.’”

ESPN NASCAR "Dominoes" spot

Deciding to do the spot in CG led to the first question – should the drivers’ faces be represented in the spot? Human characters are the most difficult thing to create realistically in CG. “From a directorial standpoint,” Peristere says, “I felt it was absolutely essential to see the drivers, to understand who they were, and to know what their motivations were so we had a personal connection to the race. I had the ever-present voice of [Buffy the Vampire Slayer and Firefly series creator] Joss Whedon in my head, who says ‘it’s all about the story; it’s all about the people.’

“We enlisted the help of some incredibly talented artists, including Brad Hayes, Brian White, and Michael Cliett.” Hayes and White had worked at Digital Domain on Benjamin Button and more recently on Tron Legacy, and had been a part of the development of a character-based VFX pipeline.

The technique used for “Dominoes” involved projecting the actual NASCAR drivers’ faces onto CG characters, allowing Peristere complete control over movement and lighting while still getting full, photo-realistic facial performances.

“Andy [Wilkoff] and I went to the very last race at Daytona, and after race day we met with the eight stars of our two commercials. We ran them though some technical setups, which involved a three-camera shoot against a greenscreen. I directed them through a series of emotions and actions that related to the story we were telling. We then took those performances back to Zoic, made editorial selects based on those performances, and gave them to Brad and Andy and the smart people to make something cool with.”

Dmitri Gueer, founder and senior editor of Zoic Editorial, was involved in the “Dominoes” spot from the pre-viz stage through the final product. He describes the editorial process as “non-stop,” and uses the facial performances as an example of Editorial’s involvement at each step.

“The pre-viz had the drivers, but we didn’t see their faces,” Gueer explains. “So the drivers were just a placeholder in the cut. When we later got the driver plates, we started picking the selects and placing them in the cut. Since the pre-viz already existed, you needed to find takes that worked for the placeholders.

“When you have the drivers’ faces mapped in the shots, it becomes apparent when we need to give them a little bit more time, or take a little time from them, because something’s not working out; and once you have a set of almost-final shots, the edit takes on a different spin. You need to pick the sweetest spots in the shots; you need to reestablish the pacing; you need to make sure there’s continuity from shot to shot; and that the edit comes together not just as a story, but also that it gels with the music and is captivating to watch.”

“We had the added complexity of a 40-car pileup,” Peristere says, “which involved extensive damage to CG vehicles, but which had to happen organically. That was hand-developed and designed by Brian White, another Digital Domain veteran with an intimate knowledge of physics and kinetics, who was able to use both animation-by-hand and procedural techniques to bring these cars into collision. You’ll see that every vehicle reacts and behaves just as a real car would as it impacts. When we have our big moment where we t-bone the hero car, you actually see it break where it should break, and that’s because Brian White made it so.”

I was looking to invoke the German Expressionist period, so I wanted these incredibly long shadows, with crushed blacks.

The spot also required an enormous smoke simulation. “Whenever these cars spin they generate tons of smoke. We worked closely with Zoic Vancouver, and a number of technical directors up in that office who specialize in smoke; they did the phenomenal nuclear explosion scene in the forthcoming movie The Crazies, for which they developed a lot of the pipeline for this — which involves Maya fluid dynamics, along with some techniques in RF4 Real Flow — so they could generate authentic smoke elements that gave the illusion and sense of a full-scale car accident on a NASCAR track.

nascardominoes3_630x354

“Kevin Struckman, Mike Rhone, and Trevor Adams all put in an incredible number of hours to make these smoke simulations incredibly spectacular, concluding with the hero car penetrating the giant smoke cloud, creating those beautiful little vortices that you see. That’s something that’s pretty tricky in a fluid simulation, and they were able to do a really nice job with that.”

In order for the spot to come together organically, there was an immense amount of compositing. “We brought in real smoke, spark, and pyro elements to underline the CG elements. Also, every single one of the 27 shots in this 30-second spot had upwards of hundreds of passes– lighting, reflections, highlights, lens flares, vignettes, grain – all of this stuff that had to be added as a secondary layer.”

The spot was rendered in full color, but the end product was always intended to be in a highly-stylized black-and-white. “That was a choice we made with Wieden+Kennedy, to create a style, a more graphic look. For me it was heading towards the films Alfred Hitchcock made in the 40s and 50s, and looking back even further to F.W. Murnau and Sunrise, and Fritz Lang and Metropolis. I was looking to invoke the German Expressionist period, so I wanted these incredibly long shadows, with crushed blacks. You’ll see a low sun – I call that the Ridley Scott sun, because Ridley Scott shoots at the magic hour all the time, and we wanted to put that in every shot. You’ll see these incredibly long film-noir shadows with bright brights, and black blacks.

nascarconcept_630x354

“Then we wanted to include the branding of Nationwide; so we applied the Nationwide presence as a design element. We had an illustrator, Eytan Zana, who did a phenomenal job setting the tone and palette.” Zana worked with Wieden+Kennedy, and with Derich Wittliff and Darrin Isono of Zoic’s design department, applying the Nationwide Pantone color to the stickers, the cars, and the track.

Peristere says, “I think overall, this black, white and blue we put together in the compositing really lends an original look to this spot that’s unlike anything we’ve seen before.”

Zoic VFX supervisor Steve Meyer handled the final finish, color grading and color treatment. “We wanted to have sort of a Raging Bull kind of look, high contrast black-and-white. So the compositors left things a little bit more on the flat side to give range; and then I took that, got the style Loni [Peristere] was looking for, and added some of those little nuances like the road rumble, the extra shake when something flies by camera, that kind of overall stuff.

“It’s a stylized look that you could attribute to real photography. I’ve been in the business for a bit, and it blows me away when I see it. Wow, that’s frickin’ all CG? It’s a very impressive spot. I was glad to be a part of it, because I think it’s going to have some legs.”

In the end, it was up to editor Gueer to assemble the finished shots into the final product. “It was a non-stop editorial process, from the beginning when Loni was assembling the story, to the time when we had all the final shots on the Flame. One of the things Steve [Meyer] did was add camera shakes to the shots, which made them look much better; but it changes the nature of what you’re seeing, even the slightest shake. You go well, wouldn’t it be better if we cut a few frames from this, or extended it by a few frames? When we had the final shots on the Flame, we literally did editorial on the Flame, making it better and better and tighter and tighter.”

“With this giant team of 40 some-odd people who worked on this spot, it’s certainly one of Zoic’s finest hours,” Peristere says, “and we’re incredibly proud to have put it together.”

People look at this spot and say “where did you guys shoot this?” Well, we didn’t shoot it!

Press is thankful to Wieden+Kennedy for trusting Zoic with the production of such an innovative and risk-taking spot. “They had faith in us and patience with us, and that was really great, because it really took that to produce this spot. It was a great experience on both sides. They gave us a lot of creative freedom, to really bring out the best in us. We pushed ourselves really hard to the level of realism and level of detail.

“I mean this kind of work, this animation, the quality level, is something very new for broadcast,” he says. “The extent to which we have gone to produce this spot in a visual style, in CG animation, has really never been done before. It’s a full 100% photo-real CG spot.

“NASCAR is very concerned about representing their world accurately, which was a big challenge for all of us, both from an agency side and a production side. Down to the decals on the cars, and the physics of the accidents, what would really get damaged and what wouldn’t, where would skid marks be made on the track… So people look at this spot and say ‘where did you guys shoot this?’ Well, we didn’t shoot it!

“The music was Metallica – my understanding is they’ve never licensed their music for broadcast commercials before. That was exciting from the get go — definitely a driving force creatively, no pun intended, the kind of energy that brings to the spot.”

Press says the spot has exceeded everyone’s expectations. “We’ve seen that response all the way around, from the agency, from our colleagues in the advertising world, and from ourselves as well – it’s really some of our best work. We’ve really set the bar anew; there’s a new target for us now, which is fantastic.”

More info: ESPN NASCAR “Dominoes” on Zoic Studios; Wieden+Kennedy.

Zoic Studios’ ZEUS: A VFX Pipeline for the 21st Century

Originally published by IDesignYourEyes on 1/7/2010.

zeus_v_greenscreen_630x354Actors Christopher Shyer and Morena Baccarin on the greenscreen set of ABC’s V; the virtual set is overlaid.

Visual effects professionals refer to the chain of processes and technologies used to produce an effects shot as a “pipeline,” a term borrowed both from traditional manufacturing and from computer architecture.

In the past year, Zoic Studios has developed a unique pipeline product called ZEUS. The showiest of ZEUS’ capabilities is to allow filmmakers on a greenscreen set to view the real-time rendered virtual set during shooting; but ZEUS does far more than that.

Zoic Studios pipeline supervisor Mike Romey explains that the pipeline that would become ZEUS was originally developed for the ABC science fiction series V. “We realized working on the pilot that we needed to create a huge number of virtual sets. [Read this for a discussion of the program’s VFX and its virtual sets.] That led us to try to find different components we could assemble and bind together, that could give us a pipeline that would let us successfully manage the volume of virtual set work we were doing for V. And, while ZEUS is a pipeline that was built to support virtual sets for V, it also fulfills the needs of our studio at large, for every aspect of production.

“One of its components is the Lightcraft virtual set tracking system, which itself is a pipeline of different components. These include InterSense motion tracking, incorporating various specialized NVIDIA graphics cards for I/O out, as well as custom inertial sensors for rotary data for the camera.

“Out of the box, we liked the Lightcraft product the most. We proceeded to build a pipeline around it that could support it.

“Our studio uses a program called Shotgun, a general-purpose database system geared for project shot management, and we were able to tailor it to support the virtual set tracking technology. By coming up with custom tools, we were able to take the on-set data, use Shotgun as a means to manage it, then lean on Shotgun to retrieve the data for custom tools throughout our pipeline. When an artist needed to set up or lay out a scene, we built tools to query Shotgun for the current plate, the current composite that was done on set, the current asset, and the current tracking data; and align them all to the timecode based on editorial selects. Shotgun was where the data was all stored, but we used Autodesk Maya as the conduit for the 3D data – we were then able to make custom tools that transport all the layout scenes from Maya to The Foundry’s Nuke compositing software.”

By offloading a lot of the 3D production onto 2D, we were able to cut the cost-per-shot.

Romey explains the rationale behind creating 3D scenes in Nuke. “When when you look at these episodic shows, there’s a large volume of shots that are close-up, and a smaller percentage of establishing shots; so we could use Nuke’s compositing application to actually do our 3D rendering. In Maya we would be rendering a traditional raytrace pipeline; but for Nuke we could render a scanline pipeline, which didn’t have same overhead. Also, this would give the compositing team immediate access to the tools they need to composite the shot faster, and it let them be responsible for a lot of the close up shots. Then our 3D team would be responsible for the establishing shots, which we knew didn’t have the quality constraints necessary for a scanline render.

“By offloading a lot of the 3D production onto 2D, we were able to cut the cost-per-shot, because we didn’t have to provide the 3D support necessary. That’s how the ZEUS pipeline evolved, with that premise – how do we meet our client’s costs and exceed their visual expectations, without breaking the bank? Throughout the ZEUS pipeline, with everything that we did, we tried to find methodologies that would shave off time, increase quality, and return a better product to the client.

“One of the avenues we R&Ded to cut costs was the I/O time. We found that we were doing many shots that required multiple plates. A new component we looked at was a product that had just been released, called Ki Pro from AJA.

“When I heard about this product, I immediately contacted AJA and explained our pipeline. We have a lot of on-set data – we the have tracking data being acquired, the greenscreen, a composite, and the potential for the key being acquired. The problem is when we went back to production, the I/O time associated with managing all the different plates became astronomical.

“Instead of running a Panasonic D5 deck to record the footage, we could use the Ki Pro, which is essentially a tapeless deck, on-set to record directly to Apple ProRes codecs. The units were cost effective – they were about $4,000 per unit – so we could set up multiple units on stage, and trigger them to record, sync and build plates that all were the exact same length, which directly corresponded to our tracking data.”

We found methodologies that would shave off time, increase quality, and return a better product to the client.

Previously, the timecode would be lost when Editorial made their selects, and would have to be reestablished. “That became a very problematic process, which would take human intervention to do — there was a lot of possibility for human error. By introducing multiple Ki Pros into the pipeline, we could record each plate, and take that back home, make sure the layout was working, and then wait for the editorial select.” The timecode from the set was preserved.

“The ZEUS pipeline is really about a relationship of image sequence to timecode. Any time that relationship is broken, or becomes more convoluted or complicated to reestablish, it introduces more human error. By relieving the process of human error, we’re able to control our costs. We can offer this pipeline to clients who need the Apple ProRes 442 codec, and at the end of the day we can take the line item of I/O time and costs, and dramatically reduce it.”

Another important component is Python, the general-purpose high-level programming language. “Our pipeline is growing faster than we can train people to use it. The reason we were able to build the ZEUS pipeline the way we have, and build it out within a month’s time, is because we opted to use tools like Python. It has given us the ability to quickly and iteratively develop tools that respond proactively to production.

“One case in point – when we first started working with the tracking data for V, we quickly realized it didn’t meet our needs. We were using open source formats such as COLLADA, which are XML scene files that stored the timecode. We needed custom tools to trim, refine and ingest the COLLADA data into our Shotgun database, into the Maya cameras, into the Nuke preferences and Nuke scenes. Python gave us the ability to do that. It’s the glue that binds our studio.

“While most components in our pipeline are interchangeable, I would argue that Python is the one component that is irreplaceable. The ability to iteratively making changes on the fly during an episode could not have been deployed and developed using other tools. It would not have been as successful, and I think it would have taken a larger development team. We don’t have a year to do production, like Avatar – we have weeks. And we don’t have a team of developers, we have one or two.

While most components in our pipeline are interchangeable, Python is the one component that is irreplaceable.

“We’re kind of new to the pipeline game. We’ve only been doing a large amount of pipeline development for two years. What we’ve done is taken some rigid steps, to carve out our pipeline such a way that when we build a tool, it can be shared across the studio.”

Romey expects great things from ZEUS in the future. “We’re currently working on an entire episodic season using ZEUS. We’re working out the kinks. From time to time there are little issues and hiccups, but that’s traditional for developing and growing a pipeline. What we’ve found is that our studio is tackling more advanced technical topics – we’re doing things like motion capture and HDR on-set tracking. We’re making sure that we have a consistent and precise road map of how everything applies in our pipeline.

“With ZEUS, we’ve come up with new ways that motion capture pipelines can work. In the future we’d like to be able to provide our clients with a way not only to be on set and see what the virtual set looks like, while the director is working — but what if the director could be on set with the virtual set, with the actor in the motion capture suit, and see the actual CG character, all in context, in real-time, on stage? Multiple characters! What if we had background characters that were all creatures, and foreground characters that were people, interacting? Quite honestly, given the technology of Lightcraft and our ability to do strong depth-of-field, we could do CG characters close-to-final on stage. I think that’s where we’d like the ZEUS pipeline to go in the future.

“Similar pipelines have been done for other productions. But in my experience, a lot of times they are one-off pipelines. ZEUS is not a pipeline just for one show; it’s a pipeline for our studio.

“It’s cost effective, and we think can get the price point to meet the needs of all our clients, including clients with smaller budgets, like webisodes. The idea of doing an Avatar-like production for a webisode is a stretch; but if we build our pipeline in such a way that we can support it, we can find new clients, and provide them with a better product.

“Our main goal with ZEUS was to find ways to make that kind of pipeline economical, to make it grow and mature. We’ve treated every single component in the pipeline as a dependency that can be interchanged if it doesn’t meet our needs, and we’re willing to do so until we get the results that we need.”

For more info: Lightcraft Technology; InterSense Inc.; Shotgun Software; AJA Video Systems; IDYE’s coverage of V.

Zoic Breathes Life Into Cartoon Network’s ‘Ben 10: Alien Swarm’

Originally published on I Design Your Eyes on 11/27/09.

Ben 10: Alien Swarm

This week, Cartoon Network premiered Ben 10: Alien Swarm, its second live-action movie based on the popular animated children’s series Ben 10: Alien Force. Alien Swarm is the sequel to the first live action film, Ben 10: Race Against Time; both were directed by Alex Winter (Freaked, Fever).

Alien Swarm continues the story of ten-year-old Ben Tennyson, an ordinary boy who becomes part of a secret organization called “the Plumbers,” which fights alien threats. He possesses a wristwatch-like device called the Omnitrix, which allows its wearer to take the physical form of various alien species. Ben, now a teenager and played by 23-year-old Ryan Kelley (Smallville), defies the Plumbers to help a mysterious childhood friend find her missing father.

Winter, an experienced director more familiar to fans as an actor from the Bill & Ted films and The Lost Boys, chose effects supervisor Evan Jacobs (Resident Evil: Extinction, Ed Wood) to oversee the movie’s many effects sequences. Jacobs worked with Culver City, California’s Zoic Studios to produce character animation and particle work for a number of key scenes.

Ben as Big Chill, using his freeze breath.
Ben as Big Chill, using his freeze breath.

Zoic worked on three main characters – Kevin “Kevin 11” Levin (Nathan Keyes, Mrs. Washington Goes to Smith), an alien-human hybrid who can absorb properties of matter; Ben’s cousin Gwen Tennyson (Galadriel Stineman, Junkyard Dog), another hybrid who manipulates energy; and Big Chill, one of Ben’s alien forms, a creature that breathes ice.

Zoic’s Executive Creative Director, Andrew Orloff (V, Fringe), says that for the production, the filmmakers chose to stay away from motion capture as “too limiting.” With all the jumping, flying and other stunt work that would be required, performers hanging from wires would not produce as realistic a result as traditional keyframing, in which every frame of a computer animation is directly modified or manipulated by the creator. “All the characters were traditionally keyframed and match moved by hand,” Orloff says.

Orloff collaborated with Winter and Jacobs to turn the Big Chill from the cartoon, an Necrofriggian from the planet Kylmyys, into a 3D, realistic breathing character. Working with a model created by Hollywood, California’s Super 78 Studios, Orloff developed character and motion & flying studies for Big Chill before the filmmakers ever hit the soundstage.

“It was very important to Alex [Winter] that we stay true to the original series, and give it a little something extra for the live action series that’s a real surprise for the viewers, to see their beloved cartoon characters finally brought to life,” Orloff says.

Gwen blasts the alien swarm, as Big Chill hovers nearby.
Gwen blasts the alien swarm, as Big Chill hovers nearby.

“Based on the visual choreography of the scenes, we didn’t really do previsualization as pre-development of the character. We talked about the way that [Big Chill] can fly, the maneuvers it could do; and that allowed Alex to have in his mind at the storyboard phase a good idea of what the kind of movement of the character was going to be.

“He’s a seven foot tall flying alien, so to create that realism was definitely a challenge. To take a two-dimensional character and turn it into a three-dimensional character, you have to maintain the integrity of the two-dimensional design, but make it look as if it’s realistically sitting in the environment. So we added a lot of skin detail, we added a lot of muscle detail and sinews; it was tricky to get the lighting of the skin exactly right. We just had to make sure that the skin had that ‘alien’ quality, so it didn’t look like a manikin or an action figure. We wanted to give a realistic feel to the skin using Maya/mental ray to render that subsurface scattering.”

Much of the footage with Big Chill involved the character flying and fighting inside a warehouse. It wasn’t possible to shoot plates that would track exactly with the as-yet unrendered character, and the filmmakers could only guess how the character would move, and how quickly. So Jacobs provided Zoic with a variety of plates of a number of different moves, plus some very high resolution 360° panoramas of the warehouse interior. Zoic then used these materials to produce its own plates, rebuilding the warehouse from the set photos and creating the shots needed to flesh out the sequence. This process was time-consuming and difficult, as much of the blocking and choreography was highly detailed.

In addition to designing the character’s movements and rendering his actions, Zoic created the freeze breath effects for Big Chill. The character’s power required two kinds of effects. First, Zoic used heavy-duty particle and fluid simulations in Maya and mental ray to create the chunks of ice, smoke and liquid nitrogen that blast from Big Chill’s mouth. Then Zoic produced quite a bit of matte painting work to encase objects in ice, icicles and frost. These include the chip swarm tornado; the interior of the warehouse; and the villain, Victor Validus (Herbert Siguenza, Mission Hill).

Kevin, having taken on the properties of the metal girder, attacks the alien swarm.
Kevin, having taken on the properties of the metal girder, attacks the alien swarm.

The main antagonist in Alien Swarm is the alien swarm itself, a cloud of thousands of intelligent, flying alien chips that work together to harm the good guys.

The alien swarm was also created in Maya and mental ray. According to Orloff, “there needed to be thousands of chips that swarmed with a random yet directed attack. The idea that the chips were learning, so they would group together – first they try to go at Gwen and the kids, and Gwen blasts them away — then they reconfigure into a buzz saw and try to attack the kids that way — then they configure into a large tornado – you have to give a personality, but an evolving personality, to a swarm of objects.” In predevelopment, Zoic looked at fish schooling and insect swarming behaviors in nature, to give the swarm movement that felt organic without seeming contrived.

Zoic also produced the effects for Kevin, who absorbs the properties of matter from objects. “Kevin was a big challenge,” Orloff says, “because what we ended up doing was scanning the actor; as he touched something we would put a CG version of the model over the top of him; rotoscope those few frames where the transition occurs; take that model and map it with whatever the material was – a rusty metal beam, a wood desk, a concrete floor. We rotoscoped the CG version over the top until the transformation was done, and then we transitioned from the rotoscoped animation, based on the actor’s performance, to a fully CG character animation.”

The energy manipulation effects for Gwen were “a ‘two-and-a-half-D’ effect, using 3D particle generators and 3D scene-tracked cameras in Adobe After Effects to create the energy bolts and energy fields that Gwen uses. We wanted to give it a ‘Jack Kirby’ kind of energy feel to it. So it has a lot of character to it, it looks very organic, and it affects the background objects and produces heat ripple effects.”

Frost effects in the warehouse. All of the frost and ice are VFX.
Frost effects in the warehouse. All of the frost and ice are VFX.

While Zoic was providing visual effects for the movie, Zoic’s Design Group worked directly with Vincent Aricco and Heather Reilly from Cartoon Network’s On-Air department, developing both the show and promo packaging for Ben 10: Alien Swarm. The package was used to promote the film both on-air and online – as well as in the Comic-Con preview this past summer.

Design Group Creative Director Derich Wittliff worked with Zoic’s internal production team, lead by Producer Scott Tinter and Designer Darrin Isono, creating 3D environments and models based on the movie’s 2D logo and other references from the film. Elements were created in Maxon Cinema 4D, Autodesk Maya and Adobe After Effects. The final product was a show open and modular promo toolkit which allowed Cartoon Network’s in-house team to create custom endpages, IDs, bumpers, and other elements.

Because the Zoic Design Group worked under the same roof as the team that produced effects for Alien Swarm, they had access to the best elements available from the show, like the “swarm” effect itself, as soon as they were created, allowing for an efficient process which produced finished elements for special uses – like Comic-Con – far in advance of customary production schedules.

Zoic Design Group Executive Producer Miles Dinsmoor says Zoic was excited to have the opportunity to work directly with Cartoon Network, acting as both a visual effects and digital production studio for the main production, and as a creative design shop for the promotional package, exploiting Zoic’s fully integrated media and design department. His goal is to offer Zoic’s in-house design and creative expertise industry-wide, and not just to Zoic’s existing VFX clients.

Orloff says he is proud of the work Zoic did on Ben 10: Alien Swarm, and looks forward to future collaboration with everyone involved – and hopefully, another Ben 10 movie.

More info: Ben 10: Alien Swarm at Cartoon Network; on Amazon.