‘FlashForward’ Flashback: Zoic Studios’ Steve Meyer on the Award-Nominated VFX for the Pilot

Posted by Erik Even in I Design Your Eyes on April 1, 2010

Property of ABC; screencap from the Zoic Television Reel.

Based on the science fiction novel by Robert J. Sawyer, ABC’s FlashForward tells the story of the aftermath of a bizarre global event. For 137 seconds, every person on Earth (except perhaps one) loses consciousness, and experiences visions of their own future.

The pilot episode presents the immediate aftermath of the worldwide disaster, with the consequences of the worldwide blackout – millions of deaths due to traffic collisions, crashed aircraft, and other accidents. Star Joseph Fiennes, portraying FBI agent Mark Benford, survives an auto wreck and looks out over a chaotic Los Angeles cityscape. Culver City, California’s Zoic Studios was tapped to create the disastrous tableau; the company’s work on the episode was nominated for two VES awards, for Outstanding Supporting Visual Effects in a Broadcast Program and for Outstanding Created Environment in a Broadcast Program or Commercial.

Zoic VFX Supervisor Steve Meyer discusses the creation of the complex scene, which required a tremendous amount of rotoscoping and motion tracking. The amount of roto was necessary, Meyer says, “because they shot in downtown LA, looking from the 4th St. overpass, over the southbound and northbound 110 Freeway. So naturally you can’t stop all that traffic, or get a greenscreen up.

“There’s a big, swooping hero shot following Joseph Fiennes as he jumps up on a car and looks down, and it’s a huge vista. Then we reverse it and look northbound. Everything in the foreground on that overpass we had to roto out. We had a team of seven or eight people going for weeks, just rotoing that and a bunch of other shots.

“We ended up having to remove all the traffic on the freeway; then added in overturned cars, cars burning, flames, smoke, helicopters crashing, just debris everywhere. We had to build a 3D matte painting in that environment. There’s a lot of detail – you can look at the shot over and over, and always see something new.”

The production brought on an experienced feature film matte painter, Roger Kupelian (2012, Alice in Wonderland) to create a “road map” of the shot. “Kupelian took a still of the freeway overpass shots, and he just dialed it in with Kevin Blank, the VFX supervisor – we want smoke here, we want the helicopter to hit here, we want fire and destruction here, we want this tree burning. They gave us a template – this is what we need it to look like. Kupelian sent us the files, and sometimes we used his elements.

“There are about 45 shots we ended up doing for the pilot, and the majority of them were for that overpass sequence. For most of the shots, nothing was locked off – every shot had some sort of roto, because there was no greenscreen. We had to roto everything to build the shots, and then try to match the smoke, fire, debris, people and other elements from shot to shot.

“We were working with different formats — stock footage, film footage, the Red Camera — trying to mix all these different formats to create one environment.”

All the scenes were tracked in Andersson Technologies’ SynthEyes camera tracking software, from which the team was able to build a 3D environment. The artists used this 3D information in Adobe After Effects to rebuild the plate with clean pieces of freeway, overpasses, signs, etc.

“The smoke was a combination of digital photographs, Google images, CG smoke, and moving elements that we had in our vault. Some of the smoke was a dust cloud that we slowed down. One of the smoke passes was a photo of a brush fire I took up by my house with my iPhone. I took the image, gave it to one of my compositors, and said ‘this will look good off in the distance.’ It’s so far off you don’t see it moving, so it fit in fine.

“They shot lots of people on greenscreen, and they all needed to have the right camera lens perspective. They’re way off in the distance; you have to get up close to an HD screen to see them. But we didn’t want any nuances to be overlooked. We don’t want to shoot a person head-on when the camera is going to be looking down at them.

“Another complication with the overpass sequence was that it was shot on a bright day, so we had a lot of technical problems. If you look at someone up against a bright sky, the sun wraps around them a bit, like a halo – and we were trying to put a dark smoke cloud behind them. It just doesn’t work right. We had a lot of technical things to try to work through when we ran into those kinds of problems. Every shot had to be 3D tracked. We took that 3D track into our environment, and we placed things in our 3D world shot by shot by shot.

“We also had a CG tanker in there that blew up. They actually had a real tanker with a big hole in it, and they threw in six gallons of gasoline and lit it and boom! It was huge. We had to put the shell of the tanker on there before the practical explosion; and then we just blew it up in Autodesk Maya and added CG debris, camera shake, heat ripple and dynamic smoke trails; plus glass shattering on the buildings and other background effects.”

Property of ABC; screencap from the Zoic Television Reel.

Wreckage from the tanker explosion strikes an overturned car and knocks it off the overpass onto the freeway below. Zoic created the car in CG. “They shot everybody running up to the guardrail and looking over,” Meyer explains. “We had to remove the railing and put in our own CG railing, so when the car goes down it takes it with it. So we had the complicated roto of recreating the people’s bodies that were behind the railing. We had to rebuild lots of people’s legs and waists. We put in the smoke and stuff that dynamically reacts to the car, so when it gets sucked down it creates a vortex and pulls the smoke down. Also, there’s an orange cart right nearby. We try to get every detail right, so when the car goes down we have a couple of oranges that roll away with it.

Property of ABC; screencap from the Zoic Television Reel.

“Then we had the falling LAPD helicopter. We took a panoramic image of a building, so we’re working on one frame and can do a pan-and-tilt in post. The helicopter has already crashed into the building, and we needed to have the smoke barreling out through most of the sequence, with the rotor blades still spinning — then we get to a certain point, and there’s an explosion that pushes the helicopter out. It tumbles and it’s scraping the building, tearing it apart and opening it up.

“Roger Kupelian labored intensely on a matte painting of the inside of the building, with what would be exposed – wires, beams, pipes, office equipment. As the CG helicopter was falling down the face of the building and opening it up, our compositor just revealed it with little mattes. At the same time we threw in sparks, debris, dust and smoke. It ended up pretty good. It was tough making an animation that made everyone happy, but in the end it looked great on the big screen at the viewing.

“Some of the shots were fun, because you can really push the envelope — let’s see what happens when we do this or when we do that. It took a lot of planning and careful choreography to between our 2D and 3D teams to keep the action and look continuous. Our teams worked tirelessly to create seamless product because anything out of place would be glaring.

“This isn’t ‘sci-fi’ with spaceships and aliens,” Meyers says, “which allow a bit of imagination – but rather, real-life vehicles, smoke, fire, people and buildings that have to look real.”

More info: FlashForward on ABC.com; the latest Zoic Studios Television Reel on ZoicStudios.com; Steven Meyer on IMDb.

Zoic Brings Photo-real CG to Broadcast TV with ESPN NASCAR “Dominoes”

Originally published on IDesignYourEyes on 2/2/2010.

ESPN NASCAR "Dominoes" spot

To the opening riffs of Metallica’s “Master of Puppets,” two NASCAR drivers jostle for position at the front of the pack. One cuts off the other by the wall, and the rear car speeds up, smashing into the front car. As the front car drifts from the wall, the rear car makes its move, attempting an aggressive pass on the right. But it’s no good – he sideswipes the front car and spins out. He’s slammed by another car and flips high into the air, triggering a massive pile-up. And straight through the smoke and chaos of the pileup – a third driver makes his move and takes the lead. “It’s anybody’s race.”

The 30-second spot for ESPN (see it here), promoting the NASCAR Nationwide series, was created by advertising agency Wieden+Kennedy New York and Culver City, California’s Zoic Studios. The commercial is significant because, despite its unique and stylized black-and-white look, it appears to have been shot in live action. In fact, it’s entirely CG.

Zoic co-founder Loni Peristere, who directed the spot, talks about why the commercial was created digitally, and how Zoic was able to create the illusion of perfect realism.

“The question from Wieden+Kennedy was, ‘we have a project, two scripts, which take place on the track, and would require significant action and stunt work. We’re trying to decide whether we should approach this from a live-action standpoint; or should we approach this from an animation standpoint.”

Wieden+Kennedy insisted the final product be photo-realistic; the agency did not want a commercial that looked like a video game.

But Wieden+Kennedy was insistent that the final product must appear perfectly photo-realistic. Peristere says the agency did not want a commercial that looked like a video game. “It was really important to them that it had the energy, grit and testosterone of the track. They were not interested in making a spot that didn’t have the reality of NASCAR.”

The agency was well aware how far CG realism has recently progressed. “Even in the last 12 months it has come a long way,” Peristere says. “With the advent of motion pictures like Avatar or The Curious Case of Benjamin Button, we are seeing the potential for photo-real characters, photo-real environments, and photo-real action. But could we actually achieve that for a commercial, and could we afford it? What would the timeline be?

“We got boards for both spots, and it became readily apparent why they were even asking this question – they had a 40-car pileup in the middle of the first spot, and a pretty significant crash in the second. Now when you looked at the second spot, you thought ‘well, from a production standpoint you could probably pull that off’; in fact we’d done something similar for Budweiser the year before. But the 40-car pileup featured just an enormous amount of damage to an enormous number of vehicles, which from a production standpoint would be very expensive.

“And the ability to control the lighting and the camera and the art direction would be limited in a live action production. You would be fighting against the sun, making you rush through the shots, allowing you limited control over your color palette. And you would have the expense of wrecking an enormous number of vehicles.”

Peristere discussed the project with other principals at Zoic – fellow co-founder Chris Jones, commercial creative director Leslie Ekker, commercial executive producer Erik Press, and CG supervisor Andy Wilkoff. “We thought it would be fun to rise to the challenge,” Peristere says. “We knew the team we had been building over the last several years had the potential to do incredible photo-realistic work. We’d seen large leaps in the realm of photo-real characters. We came back to Wieden+Kennedy and said ‘yes, yes we can.’”

ESPN NASCAR "Dominoes" spot

Deciding to do the spot in CG led to the first question – should the drivers’ faces be represented in the spot? Human characters are the most difficult thing to create realistically in CG. “From a directorial standpoint,” Peristere says, “I felt it was absolutely essential to see the drivers, to understand who they were, and to know what their motivations were so we had a personal connection to the race. I had the ever-present voice of [Buffy the Vampire Slayer and Firefly series creator] Joss Whedon in my head, who says ‘it’s all about the story; it’s all about the people.’

“We enlisted the help of some incredibly talented artists, including Brad Hayes, Brian White, and Michael Cliett.” Hayes and White had worked at Digital Domain on Benjamin Button and more recently on Tron Legacy, and had been a part of the development of a character-based VFX pipeline.

The technique used for “Dominoes” involved projecting the actual NASCAR drivers’ faces onto CG characters, allowing Peristere complete control over movement and lighting while still getting full, photo-realistic facial performances.

“Andy [Wilkoff] and I went to the very last race at Daytona, and after race day we met with the eight stars of our two commercials. We ran them though some technical setups, which involved a three-camera shoot against a greenscreen. I directed them through a series of emotions and actions that related to the story we were telling. We then took those performances back to Zoic, made editorial selects based on those performances, and gave them to Brad and Andy and the smart people to make something cool with.”

Dmitri Gueer, founder and senior editor of Zoic Editorial, was involved in the “Dominoes” spot from the pre-viz stage through the final product. He describes the editorial process as “non-stop,” and uses the facial performances as an example of Editorial’s involvement at each step.

“The pre-viz had the drivers, but we didn’t see their faces,” Gueer explains. “So the drivers were just a placeholder in the cut. When we later got the driver plates, we started picking the selects and placing them in the cut. Since the pre-viz already existed, you needed to find takes that worked for the placeholders.

“When you have the drivers’ faces mapped in the shots, it becomes apparent when we need to give them a little bit more time, or take a little time from them, because something’s not working out; and once you have a set of almost-final shots, the edit takes on a different spin. You need to pick the sweetest spots in the shots; you need to reestablish the pacing; you need to make sure there’s continuity from shot to shot; and that the edit comes together not just as a story, but also that it gels with the music and is captivating to watch.”

“We had the added complexity of a 40-car pileup,” Peristere says, “which involved extensive damage to CG vehicles, but which had to happen organically. That was hand-developed and designed by Brian White, another Digital Domain veteran with an intimate knowledge of physics and kinetics, who was able to use both animation-by-hand and procedural techniques to bring these cars into collision. You’ll see that every vehicle reacts and behaves just as a real car would as it impacts. When we have our big moment where we t-bone the hero car, you actually see it break where it should break, and that’s because Brian White made it so.”

I was looking to invoke the German Expressionist period, so I wanted these incredibly long shadows, with crushed blacks.

The spot also required an enormous smoke simulation. “Whenever these cars spin they generate tons of smoke. We worked closely with Zoic Vancouver, and a number of technical directors up in that office who specialize in smoke; they did the phenomenal nuclear explosion scene in the forthcoming movie The Crazies, for which they developed a lot of the pipeline for this — which involves Maya fluid dynamics, along with some techniques in RF4 Real Flow — so they could generate authentic smoke elements that gave the illusion and sense of a full-scale car accident on a NASCAR track.

nascardominoes3_630x354

“Kevin Struckman, Mike Rhone, and Trevor Adams all put in an incredible number of hours to make these smoke simulations incredibly spectacular, concluding with the hero car penetrating the giant smoke cloud, creating those beautiful little vortices that you see. That’s something that’s pretty tricky in a fluid simulation, and they were able to do a really nice job with that.”

In order for the spot to come together organically, there was an immense amount of compositing. “We brought in real smoke, spark, and pyro elements to underline the CG elements. Also, every single one of the 27 shots in this 30-second spot had upwards of hundreds of passes– lighting, reflections, highlights, lens flares, vignettes, grain – all of this stuff that had to be added as a secondary layer.”

The spot was rendered in full color, but the end product was always intended to be in a highly-stylized black-and-white. “That was a choice we made with Wieden+Kennedy, to create a style, a more graphic look. For me it was heading towards the films Alfred Hitchcock made in the 40s and 50s, and looking back even further to F.W. Murnau and Sunrise, and Fritz Lang and Metropolis. I was looking to invoke the German Expressionist period, so I wanted these incredibly long shadows, with crushed blacks. You’ll see a low sun – I call that the Ridley Scott sun, because Ridley Scott shoots at the magic hour all the time, and we wanted to put that in every shot. You’ll see these incredibly long film-noir shadows with bright brights, and black blacks.

nascarconcept_630x354

“Then we wanted to include the branding of Nationwide; so we applied the Nationwide presence as a design element. We had an illustrator, Eytan Zana, who did a phenomenal job setting the tone and palette.” Zana worked with Wieden+Kennedy, and with Derich Wittliff and Darrin Isono of Zoic’s design department, applying the Nationwide Pantone color to the stickers, the cars, and the track.

Peristere says, “I think overall, this black, white and blue we put together in the compositing really lends an original look to this spot that’s unlike anything we’ve seen before.”

Zoic VFX supervisor Steve Meyer handled the final finish, color grading and color treatment. “We wanted to have sort of a Raging Bull kind of look, high contrast black-and-white. So the compositors left things a little bit more on the flat side to give range; and then I took that, got the style Loni [Peristere] was looking for, and added some of those little nuances like the road rumble, the extra shake when something flies by camera, that kind of overall stuff.

“It’s a stylized look that you could attribute to real photography. I’ve been in the business for a bit, and it blows me away when I see it. Wow, that’s frickin’ all CG? It’s a very impressive spot. I was glad to be a part of it, because I think it’s going to have some legs.”

In the end, it was up to editor Gueer to assemble the finished shots into the final product. “It was a non-stop editorial process, from the beginning when Loni was assembling the story, to the time when we had all the final shots on the Flame. One of the things Steve [Meyer] did was add camera shakes to the shots, which made them look much better; but it changes the nature of what you’re seeing, even the slightest shake. You go well, wouldn’t it be better if we cut a few frames from this, or extended it by a few frames? When we had the final shots on the Flame, we literally did editorial on the Flame, making it better and better and tighter and tighter.”

“With this giant team of 40 some-odd people who worked on this spot, it’s certainly one of Zoic’s finest hours,” Peristere says, “and we’re incredibly proud to have put it together.”

People look at this spot and say “where did you guys shoot this?” Well, we didn’t shoot it!

Press is thankful to Wieden+Kennedy for trusting Zoic with the production of such an innovative and risk-taking spot. “They had faith in us and patience with us, and that was really great, because it really took that to produce this spot. It was a great experience on both sides. They gave us a lot of creative freedom, to really bring out the best in us. We pushed ourselves really hard to the level of realism and level of detail.

“I mean this kind of work, this animation, the quality level, is something very new for broadcast,” he says. “The extent to which we have gone to produce this spot in a visual style, in CG animation, has really never been done before. It’s a full 100% photo-real CG spot.

“NASCAR is very concerned about representing their world accurately, which was a big challenge for all of us, both from an agency side and a production side. Down to the decals on the cars, and the physics of the accidents, what would really get damaged and what wouldn’t, where would skid marks be made on the track… So people look at this spot and say ‘where did you guys shoot this?’ Well, we didn’t shoot it!

“The music was Metallica – my understanding is they’ve never licensed their music for broadcast commercials before. That was exciting from the get go — definitely a driving force creatively, no pun intended, the kind of energy that brings to the spot.”

Press says the spot has exceeded everyone’s expectations. “We’ve seen that response all the way around, from the agency, from our colleagues in the advertising world, and from ourselves as well – it’s really some of our best work. We’ve really set the bar anew; there’s a new target for us now, which is fantastic.”

More info: ESPN NASCAR “Dominoes” on Zoic Studios; Wieden+Kennedy.