Creating Avengers: Endgame's best scenes meant upgrading all the old Marvel movies

It's fitting that Avengers: Endgame, a storytelling and technological marvel that depicts easily the biggest live-action superhero team-up of all time, was only made possible by an equally heroic team-up of visual effects companies.

"For us, the initial conversations with all the vendors was that 'any point in time, you have to share,'" Endgame executive producer and Marvel EVP of production Victoria Alonso tells SYFY WIRE, explaining how they brought companies such as Weta and ILM together for the film and mandated collaboration. "Because it makes the movie better, which, at the end of the day, that's what we all want. We want the best movie there is, so everybody that works with us has to have that level of wanting to collaborate with each other. We started that a long time ago. And it's gotten better and better and better."

Marvel VFX Producer Jen Underdahl adds that if Marvel hadn't taken this approach, "I don't think you would have gotten Endgame and Infinity War."

Alonso and Underdahl worked alongside Weta Digital VFX Supervisor Matt Aitken, Marvel VFX Supervisor Dan DeLeeuw, ILM VFX Supervisor Russell Earl, and Marvel Additional VFX Supervisor Swen Gillberg on every key sequence of Endgame, including the scenes in which the Avengers travel back in time to key moments of the franchise's history. All of those scenes, Alonso says, were recreated entirely for the film — at times reusing older assets (such as unused footage of Natalie Portman from Thor: The Dark World) but otherwise brand new.

"We stay true to what the design of that place at that time was," Alonso says, "but the implementation of it was a reshoot."

Below, the team goes into detail about what it takes to get several different VFX companies to work together, and what they think the audience may take for granted when watching a Marvel film.

When you read the script and learned about the time travel components of the story, what was your reaction?

Jen Underdahl: For us, it was mostly about assets, because going back to Asgard, for example, that's an old CG environment, so it meant resurrecting that. We had to choose our angles, and we had to figure out what cameras we were going to use and then bring online those assets, and tweak them for the updated pipelines to make sure that we can achieve those shots with the current pipeline.

Russell Earl: [ILM] had done the scenes in the original Avengers. In fact, some of the people on our crew were some of the same people who worked on those prior shots. So one of the things that was fun was that prior to starting work on the shots, we had all been so focused on smart Hulk. And then those shots came around and it was like, okay, now we can resurrect the original Hulk from the first movie.

But there have been so many advances in technology that we wanted to bring some of that to the [first] Avengers Hulk. So we upgraded his whole underlying rigs and musculature, while at the same time keeping the features that he had. A lot of the tools we use for muscle stimulation are proprietary tools that are written in-house — it was stuff that we couldn't do on the prior films just because it was computationally expensive, and the toolset wasn't there. But it has now since gotten to a point where all of that stuff has progressed.

It's always fun to go back to a prior film, seeing it from a different angle. People who were working on it would see the shots and go — did we just bring back the old footage? There's a handful of shots that were from the previous film where we were rotoscoping them out and replacing and adding characters. No, this was new photography that now we're placing our characters into.

Swen Gillberg: For the most part, everything was reshot. Yeah, we use some assets. There are two mostly CG shots...

Earl: Yeah, there was a high angle shot from what was the original one — the big shot on that bridge there.

Gillberg: We generally don't like to reuse photography because you want a different angle, like as if it weren't in the movie. So we go out of our way to reshoot pretty much everything. We rebuild the sets to make them look the same. For example, the Avengers street-level stuff was shot originally in Cleveland, at one particular street, and we did it in downtown Atlanta. So we try not to reuse, because we want different angles and want the same look for the same characters but no reusing.

Earl: It's a bit of an archaeological dig. "Where's that? I know we have that asset somewhere." "I think Dan did that shot." And then you go find Dan.

Credit: Marvel Studios

It seems like having a really good archivist on staff is invaluable.

Victoria Alonso: Well, the good thing about having these long, long relationships with these constant vendors is they are our archives. So we go to the same teams, we go to the same companies. That's very, very helpful.

Earl: It's also helpful now that we're sharing a lot more assets. Most facilities work in different ways. They have different rigging systems, proprietary toolsets that they use. But when it comes to actual characters, a lot of that look development can be pretty complex. So we'll package up basically a road map to how we got to the look without necessarily having to send the exact files because no one's going to take your setup and import it directly. You're getting textures, geometry, and sort of tables and turntables. And then we'll just jump on the phone and say, "Hey, how are you guys using those maps?" — you know, just to try to help.

Alonso: Our movies are so big and we want to have a seamless transition from scene to scene with all the characters. And sometimes ILM cannot do all the smart Hulk shots, or they can't do all the Thanos shots, or they can't do all the traveling from one galaxy to another. So what happens is you have another company say, "yeah, we can do that portion of the fight or that portion of the journey. But since you've already built a lot of those assets, give us what you've got."

It's like you get a coat from your older sister. They take what has been built — they get the coat, they get it tailored to their body, and off they go into the winter.

Matt Aitken: As an industry, we're slowly but surely getting better at the sharing. We're establishing software protocols for more than just the model and the texture, and this is something that in a large part is being driven by Marvel because you know you had these giant, all-encompassing visual-effects-driven films with multiple vendors on them. It's part and parcel of the work on these shows, is this asset sharing and environment sharing.

Credit: Marvel

What do you feel like is an aspect of making these films that the general viewing public doesn't necessarily notice?

Gillberg: The overall scope — massive amounts of people that are needed to make these movies. It's hard to fathom unless you're deep into it.

Earl: Yeah, I think things like the time travel suits — there's a lot of shots where they're just costumed bodies, but it's all CG, the costume. I think most people wouldn't even recognize that. The visual effects touch on so many different pieces of filmmaking that a lot of it isn't visible.

Aitken: People aren't aware that the Avengers compound has never existed as a place. You can't go there. There's a couple of buildings that have been used as facades, but I think they're miles apart from each other. It's always a visual effect.

Alonso: I'm on my 25th Marvel movie now. So I've seen it throughout, I think, not only the sheer size of it, the global-ness of it, but the level of commitment that every visual effects artist brings to the project. It's the culture of Marvel fans and fans that work as filmmakers — you couldn't have the level of consistency [without it]. You couldn't have that if the commitment to striving for the best imagery that we can wasn't there. I think that is how we get the level of quality that we have.

This interview has been condensed and edited for clarity.