Zoic Brings Photo-real CG to Broadcast TV with ESPN NASCAR “Dominoes”

Originally published on IDesignYourEyes on 2/2/2010.

ESPN NASCAR "Dominoes" spot

To the opening riffs of Metallica’s “Master of Puppets,” two NASCAR drivers jostle for position at the front of the pack. One cuts off the other by the wall, and the rear car speeds up, smashing into the front car. As the front car drifts from the wall, the rear car makes its move, attempting an aggressive pass on the right. But it’s no good – he sideswipes the front car and spins out. He’s slammed by another car and flips high into the air, triggering a massive pile-up. And straight through the smoke and chaos of the pileup – a third driver makes his move and takes the lead. “It’s anybody’s race.”

The 30-second spot for ESPN (see it here), promoting the NASCAR Nationwide series, was created by advertising agency Wieden+Kennedy New York and Culver City, California’s Zoic Studios. The commercial is significant because, despite its unique and stylized black-and-white look, it appears to have been shot in live action. In fact, it’s entirely CG.

Zoic co-founder Loni Peristere, who directed the spot, talks about why the commercial was created digitally, and how Zoic was able to create the illusion of perfect realism.

“The question from Wieden+Kennedy was, ‘we have a project, two scripts, which take place on the track, and would require significant action and stunt work. We’re trying to decide whether we should approach this from a live-action standpoint; or should we approach this from an animation standpoint.”

Wieden+Kennedy insisted the final product be photo-realistic; the agency did not want a commercial that looked like a video game.

But Wieden+Kennedy was insistent that the final product must appear perfectly photo-realistic. Peristere says the agency did not want a commercial that looked like a video game. “It was really important to them that it had the energy, grit and testosterone of the track. They were not interested in making a spot that didn’t have the reality of NASCAR.”

The agency was well aware how far CG realism has recently progressed. “Even in the last 12 months it has come a long way,” Peristere says. “With the advent of motion pictures like Avatar or The Curious Case of Benjamin Button, we are seeing the potential for photo-real characters, photo-real environments, and photo-real action. But could we actually achieve that for a commercial, and could we afford it? What would the timeline be?

“We got boards for both spots, and it became readily apparent why they were even asking this question – they had a 40-car pileup in the middle of the first spot, and a pretty significant crash in the second. Now when you looked at the second spot, you thought ‘well, from a production standpoint you could probably pull that off’; in fact we’d done something similar for Budweiser the year before. But the 40-car pileup featured just an enormous amount of damage to an enormous number of vehicles, which from a production standpoint would be very expensive.

“And the ability to control the lighting and the camera and the art direction would be limited in a live action production. You would be fighting against the sun, making you rush through the shots, allowing you limited control over your color palette. And you would have the expense of wrecking an enormous number of vehicles.”

Peristere discussed the project with other principals at Zoic – fellow co-founder Chris Jones, commercial creative director Leslie Ekker, commercial executive producer Erik Press, and CG supervisor Andy Wilkoff. “We thought it would be fun to rise to the challenge,” Peristere says. “We knew the team we had been building over the last several years had the potential to do incredible photo-realistic work. We’d seen large leaps in the realm of photo-real characters. We came back to Wieden+Kennedy and said ‘yes, yes we can.’”

ESPN NASCAR "Dominoes" spot

Deciding to do the spot in CG led to the first question – should the drivers’ faces be represented in the spot? Human characters are the most difficult thing to create realistically in CG. “From a directorial standpoint,” Peristere says, “I felt it was absolutely essential to see the drivers, to understand who they were, and to know what their motivations were so we had a personal connection to the race. I had the ever-present voice of [Buffy the Vampire Slayer and Firefly series creator] Joss Whedon in my head, who says ‘it’s all about the story; it’s all about the people.’

“We enlisted the help of some incredibly talented artists, including Brad Hayes, Brian White, and Michael Cliett.” Hayes and White had worked at Digital Domain on Benjamin Button and more recently on Tron Legacy, and had been a part of the development of a character-based VFX pipeline.

The technique used for “Dominoes” involved projecting the actual NASCAR drivers’ faces onto CG characters, allowing Peristere complete control over movement and lighting while still getting full, photo-realistic facial performances.

“Andy [Wilkoff] and I went to the very last race at Daytona, and after race day we met with the eight stars of our two commercials. We ran them though some technical setups, which involved a three-camera shoot against a greenscreen. I directed them through a series of emotions and actions that related to the story we were telling. We then took those performances back to Zoic, made editorial selects based on those performances, and gave them to Brad and Andy and the smart people to make something cool with.”

Dmitri Gueer, founder and senior editor of Zoic Editorial, was involved in the “Dominoes” spot from the pre-viz stage through the final product. He describes the editorial process as “non-stop,” and uses the facial performances as an example of Editorial’s involvement at each step.

“The pre-viz had the drivers, but we didn’t see their faces,” Gueer explains. “So the drivers were just a placeholder in the cut. When we later got the driver plates, we started picking the selects and placing them in the cut. Since the pre-viz already existed, you needed to find takes that worked for the placeholders.

“When you have the drivers’ faces mapped in the shots, it becomes apparent when we need to give them a little bit more time, or take a little time from them, because something’s not working out; and once you have a set of almost-final shots, the edit takes on a different spin. You need to pick the sweetest spots in the shots; you need to reestablish the pacing; you need to make sure there’s continuity from shot to shot; and that the edit comes together not just as a story, but also that it gels with the music and is captivating to watch.”

“We had the added complexity of a 40-car pileup,” Peristere says, “which involved extensive damage to CG vehicles, but which had to happen organically. That was hand-developed and designed by Brian White, another Digital Domain veteran with an intimate knowledge of physics and kinetics, who was able to use both animation-by-hand and procedural techniques to bring these cars into collision. You’ll see that every vehicle reacts and behaves just as a real car would as it impacts. When we have our big moment where we t-bone the hero car, you actually see it break where it should break, and that’s because Brian White made it so.”

I was looking to invoke the German Expressionist period, so I wanted these incredibly long shadows, with crushed blacks.

The spot also required an enormous smoke simulation. “Whenever these cars spin they generate tons of smoke. We worked closely with Zoic Vancouver, and a number of technical directors up in that office who specialize in smoke; they did the phenomenal nuclear explosion scene in the forthcoming movie The Crazies, for which they developed a lot of the pipeline for this — which involves Maya fluid dynamics, along with some techniques in RF4 Real Flow — so they could generate authentic smoke elements that gave the illusion and sense of a full-scale car accident on a NASCAR track.

nascardominoes3_630x354

“Kevin Struckman, Mike Rhone, and Trevor Adams all put in an incredible number of hours to make these smoke simulations incredibly spectacular, concluding with the hero car penetrating the giant smoke cloud, creating those beautiful little vortices that you see. That’s something that’s pretty tricky in a fluid simulation, and they were able to do a really nice job with that.”

In order for the spot to come together organically, there was an immense amount of compositing. “We brought in real smoke, spark, and pyro elements to underline the CG elements. Also, every single one of the 27 shots in this 30-second spot had upwards of hundreds of passes– lighting, reflections, highlights, lens flares, vignettes, grain – all of this stuff that had to be added as a secondary layer.”

The spot was rendered in full color, but the end product was always intended to be in a highly-stylized black-and-white. “That was a choice we made with Wieden+Kennedy, to create a style, a more graphic look. For me it was heading towards the films Alfred Hitchcock made in the 40s and 50s, and looking back even further to F.W. Murnau and Sunrise, and Fritz Lang and Metropolis. I was looking to invoke the German Expressionist period, so I wanted these incredibly long shadows, with crushed blacks. You’ll see a low sun – I call that the Ridley Scott sun, because Ridley Scott shoots at the magic hour all the time, and we wanted to put that in every shot. You’ll see these incredibly long film-noir shadows with bright brights, and black blacks.

nascarconcept_630x354

“Then we wanted to include the branding of Nationwide; so we applied the Nationwide presence as a design element. We had an illustrator, Eytan Zana, who did a phenomenal job setting the tone and palette.” Zana worked with Wieden+Kennedy, and with Derich Wittliff and Darrin Isono of Zoic’s design department, applying the Nationwide Pantone color to the stickers, the cars, and the track.

Peristere says, “I think overall, this black, white and blue we put together in the compositing really lends an original look to this spot that’s unlike anything we’ve seen before.”

Zoic VFX supervisor Steve Meyer handled the final finish, color grading and color treatment. “We wanted to have sort of a Raging Bull kind of look, high contrast black-and-white. So the compositors left things a little bit more on the flat side to give range; and then I took that, got the style Loni [Peristere] was looking for, and added some of those little nuances like the road rumble, the extra shake when something flies by camera, that kind of overall stuff.

“It’s a stylized look that you could attribute to real photography. I’ve been in the business for a bit, and it blows me away when I see it. Wow, that’s frickin’ all CG? It’s a very impressive spot. I was glad to be a part of it, because I think it’s going to have some legs.”

In the end, it was up to editor Gueer to assemble the finished shots into the final product. “It was a non-stop editorial process, from the beginning when Loni was assembling the story, to the time when we had all the final shots on the Flame. One of the things Steve [Meyer] did was add camera shakes to the shots, which made them look much better; but it changes the nature of what you’re seeing, even the slightest shake. You go well, wouldn’t it be better if we cut a few frames from this, or extended it by a few frames? When we had the final shots on the Flame, we literally did editorial on the Flame, making it better and better and tighter and tighter.”

“With this giant team of 40 some-odd people who worked on this spot, it’s certainly one of Zoic’s finest hours,” Peristere says, “and we’re incredibly proud to have put it together.”

People look at this spot and say “where did you guys shoot this?” Well, we didn’t shoot it!

Press is thankful to Wieden+Kennedy for trusting Zoic with the production of such an innovative and risk-taking spot. “They had faith in us and patience with us, and that was really great, because it really took that to produce this spot. It was a great experience on both sides. They gave us a lot of creative freedom, to really bring out the best in us. We pushed ourselves really hard to the level of realism and level of detail.

“I mean this kind of work, this animation, the quality level, is something very new for broadcast,” he says. “The extent to which we have gone to produce this spot in a visual style, in CG animation, has really never been done before. It’s a full 100% photo-real CG spot.

“NASCAR is very concerned about representing their world accurately, which was a big challenge for all of us, both from an agency side and a production side. Down to the decals on the cars, and the physics of the accidents, what would really get damaged and what wouldn’t, where would skid marks be made on the track… So people look at this spot and say ‘where did you guys shoot this?’ Well, we didn’t shoot it!

“The music was Metallica – my understanding is they’ve never licensed their music for broadcast commercials before. That was exciting from the get go — definitely a driving force creatively, no pun intended, the kind of energy that brings to the spot.”

Press says the spot has exceeded everyone’s expectations. “We’ve seen that response all the way around, from the agency, from our colleagues in the advertising world, and from ourselves as well – it’s really some of our best work. We’ve really set the bar anew; there’s a new target for us now, which is fantastic.”

More info: ESPN NASCAR “Dominoes” on Zoic Studios; Wieden+Kennedy.

Zoic Studios’ ZEUS: A VFX Pipeline for the 21st Century

Originally published by IDesignYourEyes on 1/7/2010.

zeus_v_greenscreen_630x354Actors Christopher Shyer and Morena Baccarin on the greenscreen set of ABC’s V; the virtual set is overlaid.

Visual effects professionals refer to the chain of processes and technologies used to produce an effects shot as a “pipeline,” a term borrowed both from traditional manufacturing and from computer architecture.

In the past year, Zoic Studios has developed a unique pipeline product called ZEUS. The showiest of ZEUS’ capabilities is to allow filmmakers on a greenscreen set to view the real-time rendered virtual set during shooting; but ZEUS does far more than that.

Zoic Studios pipeline supervisor Mike Romey explains that the pipeline that would become ZEUS was originally developed for the ABC science fiction series V. “We realized working on the pilot that we needed to create a huge number of virtual sets. [Read this for a discussion of the program’s VFX and its virtual sets.] That led us to try to find different components we could assemble and bind together, that could give us a pipeline that would let us successfully manage the volume of virtual set work we were doing for V. And, while ZEUS is a pipeline that was built to support virtual sets for V, it also fulfills the needs of our studio at large, for every aspect of production.

“One of its components is the Lightcraft virtual set tracking system, which itself is a pipeline of different components. These include InterSense motion tracking, incorporating various specialized NVIDIA graphics cards for I/O out, as well as custom inertial sensors for rotary data for the camera.

“Out of the box, we liked the Lightcraft product the most. We proceeded to build a pipeline around it that could support it.

“Our studio uses a program called Shotgun, a general-purpose database system geared for project shot management, and we were able to tailor it to support the virtual set tracking technology. By coming up with custom tools, we were able to take the on-set data, use Shotgun as a means to manage it, then lean on Shotgun to retrieve the data for custom tools throughout our pipeline. When an artist needed to set up or lay out a scene, we built tools to query Shotgun for the current plate, the current composite that was done on set, the current asset, and the current tracking data; and align them all to the timecode based on editorial selects. Shotgun was where the data was all stored, but we used Autodesk Maya as the conduit for the 3D data – we were then able to make custom tools that transport all the layout scenes from Maya to The Foundry’s Nuke compositing software.”

By offloading a lot of the 3D production onto 2D, we were able to cut the cost-per-shot.

Romey explains the rationale behind creating 3D scenes in Nuke. “When when you look at these episodic shows, there’s a large volume of shots that are close-up, and a smaller percentage of establishing shots; so we could use Nuke’s compositing application to actually do our 3D rendering. In Maya we would be rendering a traditional raytrace pipeline; but for Nuke we could render a scanline pipeline, which didn’t have same overhead. Also, this would give the compositing team immediate access to the tools they need to composite the shot faster, and it let them be responsible for a lot of the close up shots. Then our 3D team would be responsible for the establishing shots, which we knew didn’t have the quality constraints necessary for a scanline render.

“By offloading a lot of the 3D production onto 2D, we were able to cut the cost-per-shot, because we didn’t have to provide the 3D support necessary. That’s how the ZEUS pipeline evolved, with that premise – how do we meet our client’s costs and exceed their visual expectations, without breaking the bank? Throughout the ZEUS pipeline, with everything that we did, we tried to find methodologies that would shave off time, increase quality, and return a better product to the client.

“One of the avenues we R&Ded to cut costs was the I/O time. We found that we were doing many shots that required multiple plates. A new component we looked at was a product that had just been released, called Ki Pro from AJA.

“When I heard about this product, I immediately contacted AJA and explained our pipeline. We have a lot of on-set data – we the have tracking data being acquired, the greenscreen, a composite, and the potential for the key being acquired. The problem is when we went back to production, the I/O time associated with managing all the different plates became astronomical.

“Instead of running a Panasonic D5 deck to record the footage, we could use the Ki Pro, which is essentially a tapeless deck, on-set to record directly to Apple ProRes codecs. The units were cost effective – they were about $4,000 per unit – so we could set up multiple units on stage, and trigger them to record, sync and build plates that all were the exact same length, which directly corresponded to our tracking data.”

We found methodologies that would shave off time, increase quality, and return a better product to the client.

Previously, the timecode would be lost when Editorial made their selects, and would have to be reestablished. “That became a very problematic process, which would take human intervention to do — there was a lot of possibility for human error. By introducing multiple Ki Pros into the pipeline, we could record each plate, and take that back home, make sure the layout was working, and then wait for the editorial select.” The timecode from the set was preserved.

“The ZEUS pipeline is really about a relationship of image sequence to timecode. Any time that relationship is broken, or becomes more convoluted or complicated to reestablish, it introduces more human error. By relieving the process of human error, we’re able to control our costs. We can offer this pipeline to clients who need the Apple ProRes 442 codec, and at the end of the day we can take the line item of I/O time and costs, and dramatically reduce it.”

Another important component is Python, the general-purpose high-level programming language. “Our pipeline is growing faster than we can train people to use it. The reason we were able to build the ZEUS pipeline the way we have, and build it out within a month’s time, is because we opted to use tools like Python. It has given us the ability to quickly and iteratively develop tools that respond proactively to production.

“One case in point – when we first started working with the tracking data for V, we quickly realized it didn’t meet our needs. We were using open source formats such as COLLADA, which are XML scene files that stored the timecode. We needed custom tools to trim, refine and ingest the COLLADA data into our Shotgun database, into the Maya cameras, into the Nuke preferences and Nuke scenes. Python gave us the ability to do that. It’s the glue that binds our studio.

“While most components in our pipeline are interchangeable, I would argue that Python is the one component that is irreplaceable. The ability to iteratively making changes on the fly during an episode could not have been deployed and developed using other tools. It would not have been as successful, and I think it would have taken a larger development team. We don’t have a year to do production, like Avatar – we have weeks. And we don’t have a team of developers, we have one or two.

While most components in our pipeline are interchangeable, Python is the one component that is irreplaceable.

“We’re kind of new to the pipeline game. We’ve only been doing a large amount of pipeline development for two years. What we’ve done is taken some rigid steps, to carve out our pipeline such a way that when we build a tool, it can be shared across the studio.”

Romey expects great things from ZEUS in the future. “We’re currently working on an entire episodic season using ZEUS. We’re working out the kinks. From time to time there are little issues and hiccups, but that’s traditional for developing and growing a pipeline. What we’ve found is that our studio is tackling more advanced technical topics – we’re doing things like motion capture and HDR on-set tracking. We’re making sure that we have a consistent and precise road map of how everything applies in our pipeline.

“With ZEUS, we’ve come up with new ways that motion capture pipelines can work. In the future we’d like to be able to provide our clients with a way not only to be on set and see what the virtual set looks like, while the director is working — but what if the director could be on set with the virtual set, with the actor in the motion capture suit, and see the actual CG character, all in context, in real-time, on stage? Multiple characters! What if we had background characters that were all creatures, and foreground characters that were people, interacting? Quite honestly, given the technology of Lightcraft and our ability to do strong depth-of-field, we could do CG characters close-to-final on stage. I think that’s where we’d like the ZEUS pipeline to go in the future.

“Similar pipelines have been done for other productions. But in my experience, a lot of times they are one-off pipelines. ZEUS is not a pipeline just for one show; it’s a pipeline for our studio.

“It’s cost effective, and we think can get the price point to meet the needs of all our clients, including clients with smaller budgets, like webisodes. The idea of doing an Avatar-like production for a webisode is a stretch; but if we build our pipeline in such a way that we can support it, we can find new clients, and provide them with a better product.

“Our main goal with ZEUS was to find ways to make that kind of pipeline economical, to make it grow and mature. We’ve treated every single component in the pipeline as a dependency that can be interchanged if it doesn’t meet our needs, and we’re willing to do so until we get the results that we need.”

For more info: Lightcraft Technology; InterSense Inc.; Shotgun Software; AJA Video Systems; IDYE’s coverage of V.