WEBVTT
00:00:00.000 --> 00:00:05.049
Casey Schatz is a veteran expert in virtual production techniques for motion pictures and television.
00:00:05.166 --> 00:00:11.006
His credits include Avatar, Game of Thrones, Avengers: Age of Ultron, The Martian and more.
00:00:11.064 --> 00:00:16.234
As Head of Virtual Production at The Third Floor, Schatz promotes the design and use of on-set simulcam.
00:00:16.370 --> 00:00:21.629
As with many of our successful guests, his fascination with technology started at an early age.
00:00:21.675 --> 00:00:27.024
Something clicked, very early on, I mean, maybe nine or 10 years old with a Commodor 64.
00:00:27.085 --> 00:00:32.945
And you could do these particular keystrokes where you could make a Siam magenta yellow, or a red green, or blue cube.
00:00:33.484 --> 00:00:50.591
And I would just sit in front of that thing for hours and at the same time, my dad did a lot of documentary work for the news station he was working at and, this fascination with optics and film and, traditional photography was kind of growing concurrently , with computer graphics and that theme.
00:00:50.741 --> 00:00:52.822
Really has never left and still hasn't frankly,
00:00:53.542 --> 00:01:00.222
Schatz looked to gain education and experience initially as a cinematographer but found himself drawn to other areas of filmmaking.
00:01:00.381 --> 00:01:17.816
I went to Cal arts and I got denied by the cinematography department, but got accepted into the still photography and the animation department, and Cal arts ended up being the perfect place because they let me sort of customize my own degree where I did theater lighting, traditional photography, cinematography.
00:01:17.891 --> 00:01:22.006
And then of course I was always in the F 1 0 5 lab, the computer graphics lab.
00:01:23.090 --> 00:01:39.662
And I had no idea what previz or techvis or virtual production was, at the time, but looking back, I can't think of a better education because it was so multidisciplinary and, fusing, modern, nifty computer graphic stuff with traditional old fashioned, filmmaking principles.
00:01:40.046 --> 00:01:42.305
And it's not that one is better than the other.
00:01:42.436 --> 00:01:44.956
It's where they shake hands that I think is so fascinating.
00:01:45.176 --> 00:01:52.906
Like many other filmmakers in his peer group, Schatz was inspired by a string of blockbuster visual effects films that pushed the technology of the time forward.
00:01:53.483 --> 00:02:00.034
were in high school when jurasic park and teach, who came out, which were okay, tick the box career done.
00:02:00.063 --> 00:02:04.084
Like I was just so enamored with the work and that work still holds up to this day, frankly.
00:02:04.593 --> 00:02:08.913
, so as far as soul searching and career, like, that was almost, never an issue.
00:02:09.270 --> 00:02:16.021
So then the only question afterwards was which field within it, the first school I went to was the academy of art, San Francisco.
00:02:16.361 --> 00:02:22.611
And I was working on modeling, a human face and, you know, putting, specul reflections on the eyes.
00:02:22.632 --> 00:02:26.581
And we were trying to do what at the time was extremely primitive, hair and stuff.
00:02:26.722 --> 00:02:38.181
And I had this kind of moment of like, why am I trying so hard to recreate reality when reality is just right outside the door and so I kind of had this phase actually where I just stopped doing computer graphics and really thought.
00:02:39.026 --> 00:03:03.733
I'm gonna be a national geographic photographer or a traditional cinematographer, and I got a light meter and some spotlights and I really thought maybe I'm going down the career path and the computer graphics bug, bit again, you know, there was no way I was ever gonna give it up, but I think looking back, it was good because now what we see with virtual production is these worlds, shaking hands, , the principles of live action filmmaking that have existed since the Lumyer brothers.
00:03:04.240 --> 00:03:09.610
You have that on one side and then you have, unreal engine and VR headsets, but who cares at the end of the day?
00:03:09.610 --> 00:03:10.569
We're all making stories.
00:03:10.620 --> 00:03:12.330
We're putting images on the screen.
00:03:12.710 --> 00:03:14.659
So what medium, does it really matter?
00:03:14.719 --> 00:03:22.659
And there's so many great between these extremes that, to categorize them is really of secondary importance to just using the tool that's right.
00:03:22.840 --> 00:03:24.180
For the client and for the job.
00:03:24.717 --> 00:03:31.318
Schatz embraced the multidisciplinary nature of his education and, as a result, found his skills in demand right away.
00:03:31.467 --> 00:03:35.343
So every year, , the Kal arts animation department has the end of year producer show.
00:03:35.764 --> 00:03:38.289
And my short film showed at the producer's showcase.
00:03:38.622 --> 00:03:46.566
. And the hallway afterwards, , Tom bear who started image G, which is a motion control company, came up and said, can you start Monday basically?
00:03:46.656 --> 00:03:47.295
I was so excited.
00:03:47.415 --> 00:03:53.436
I hardly knew what MOS control was and I didn't even really know what previ was It was still a very young, art form at the time.
00:03:53.975 --> 00:04:03.808
And when I started there, I started to do previs for car commercial that were being used with this huge, like three ton motion control camera, crane.
00:04:04.269 --> 00:04:08.381
And the way that the offices were set up, , there was a window and then the rig was in the parking lot.
00:04:08.721 --> 00:04:10.260
So literally as I was.
00:04:10.975 --> 00:04:17.026
Animating the real thing that weighed three tons and had to move was right there next to me.
00:04:17.206 --> 00:04:31.185
And so that instilled this sort of discipline to not get too seduced by the tools that the computer had, like no gravity and, velocity was not a, a concern, , so I started prevising with real equipment in mind.
00:04:31.642 --> 00:04:36.353
And the two things have always been embedded to do previs for things that are actually suitable.
00:04:36.572 --> 00:04:37.442
And if if they're not.
00:04:37.987 --> 00:04:38.968
That's fine.
00:04:39.048 --> 00:04:41.810
I mean, let's have a superhero take off and fly to outer space.
00:04:41.920 --> 00:04:52.122
But let's then break it down into its constituent plates and think about how we're gonna photograph this the worst thing you could do in previz is to make pixels move on this green in a certain way.
00:04:52.382 --> 00:04:57.562
And the director says, Yippy, then you get to set and you realize there's no bearing and reality with this.
00:04:57.562 --> 00:04:58.642
We can't accomplish this.
00:04:59.119 --> 00:05:03.238
the, The computer a person didn't have a respect for the physics and the parameters of live action shooting.
00:05:03.749 --> 00:05:12.399
Along with learning about the mechanics of motion control and how it should inform accurate previs, Schatz also took lessons from how to operate in front of clients professionally.
00:05:12.512 --> 00:05:18.127
So I had, been the cinematographer on this, music video and we went to a Tein place in Hollywood.
00:05:18.127 --> 00:05:19.687
It was a really swanky, fancy place.
00:05:20.218 --> 00:05:21.817
what really changed me was seeing.
00:05:22.298 --> 00:05:23.557
The operator work.
00:05:23.901 --> 00:05:31.478
we were just sitting in these, really cushy chairs and talking about the color balances and, and let's add a point of, know, red and,, subtract a point of this and that.
00:05:31.898 --> 00:05:36.401
And the person was just doing it and you could see, the impact in real time.
00:05:37.052 --> 00:05:37.752
And I.
00:05:37.935 --> 00:05:40.350
Thought . why would 3d graphics be any different?
00:05:40.810 --> 00:05:46.750
So , all right, I just need to get fast enough at Maya so that the client can be in the cushy to chair behind me.
00:05:46.970 --> 00:05:47.949
And I'm just like that.
00:05:47.949 --> 00:05:50.990
Tell any guy I'm moving the mouse in the keyboard really fast.
00:05:51.370 --> 00:06:00.237
And the image is just forming sort of like developing in the dark room and if I failed to do something really quickly, While live with a client, I would say, well, that sucked.
00:06:00.237 --> 00:06:04.276
How can I automate that in, you know, Mel and Python, I would write my own little tools.
00:06:04.857 --> 00:06:09.627
Jump cut to now a lot of that's happening with virtual production, because rather than them saying.
00:06:10.341 --> 00:06:10.632
Okay.
00:06:10.641 --> 00:06:12.612
Let's boom, up let's counter tilt down.
00:06:12.682 --> 00:06:18.021
We could actually hand them an iPad or something that's, optically captured and say, well, go for it.
00:06:18.081 --> 00:06:26.595
But also, there are still folks that are shy about the equipment and want to just sit on that couch or whatever, and, sort of dictate what their, vision is.
00:06:26.595 --> 00:06:29.035
And everyone works differently and one's not better than the other.
00:06:29.975 --> 00:06:39.884
Schatz excelled at his work with creatives and was ultimately offered a position on Avatar, a movie that would become known for its pioneering work in motion capture and virtual cinematography.
00:06:40.009 --> 00:06:41.389
I was like, I haven't done that.
00:06:42.089 --> 00:06:47.860
Project yet where I worked with one of the master directors, you know, and I, wanted to see if I could do it, if I was up for it.
00:06:48.110 --> 00:06:51.031
Just to even be in the same building as Jim Cameron was gonna be an honor.
00:06:51.211 --> 00:06:58.161
So talk about trial by fire for virtual production, had Some experience editing mocap, but had never been on a mocap stage.
00:06:58.221 --> 00:07:06.524
So on day one to see horror horses in mocap suits and then the actors also in the mocap suits, riding the horses and they're, they're going full speed down.
00:07:06.524 --> 00:07:09.644
The length of the stage was, a pretty remarkable, education.
00:07:10.035 --> 00:07:13.663
I ended up being like, I think my, credit on the film was SEL cam supervisor.
00:07:13.860 --> 00:07:15.360
So simultaneous camera.
00:07:15.610 --> 00:07:26.790
This would be sort of the art and science of capturing the motion picture camera much like you would, , a person by putting funny dots on their body and , they get wronged and calibrated and everything let's do that same thing to the camera.
00:07:26.850 --> 00:07:31.250
So those, active markers are put onto the camera and that camera is being captured.
00:07:31.737 --> 00:07:38.315
So I gots super duper involved in thes cam and also was also doing a lot of the motion based motion control camera.
00:07:38.980 --> 00:07:40.968
A lot of the tricky, live action setups.
00:07:41.305 --> 00:07:44.605
So yeah, I was on avatar for three and a half or four years.
00:07:44.725 --> 00:07:49.298
I mean, that was a life changing experience, getting to learn from, Jim and all the other, folks on the crew.
00:07:49.798 --> 00:07:52.718
And, I still watched the movie, like, did we do that.
00:07:52.718 --> 00:07:53.718
How did we pull this off?
00:07:53.778 --> 00:07:53.928
You
00:07:54.360 --> 00:07:59.721
The Avatar team invented many techniques which are now considered standard practice in the virtual production world.
00:07:59.949 --> 00:08:12.038
I'll never forget The first really successful moment on thes cam, which is the shot where, net Teri goes up to the shack because the, real Jake is, breathing the Pandora and air and, suffer.
00:08:12.458 --> 00:08:16.437
And so she runs up to the shack, looks through the window, sees him.
00:08:17.088 --> 00:08:22.827
Goes to the stairs, jumps and holds what is now, the live action, same Worthington character.
00:08:23.247 --> 00:08:24.567
And it was very, very tough.
00:08:24.964 --> 00:08:28.584
our virtual production engineer and all around just brilliant dude.
00:08:28.675 --> 00:08:36.424
Brian Champney, he and others, made this, optical volume in a very claustrophobic situation, with optical volumes, you want a lot of space.
00:08:36.445 --> 00:08:46.160
You want the, field of view of the lenses to have a nice spread so that they're encompassing a lot and overlapping area, and that's very tough and a very claustrophobic cuz the shack was real.
00:08:46.581 --> 00:08:48.441
The art department made . A real shack.
00:08:49.039 --> 00:08:58.354
And, we got it working and the live action cameraman, I think it was Dave Ericks, was able to see net teary and compose the shot with a CG character.
00:08:58.677 --> 00:09:00.649
As good as an operator as Dave is.
00:09:00.669 --> 00:09:17.402
And, and many camera operators, are it's hard to perfectly imagine it and do it without seeing the digital element and that was kind of the moment that we really got it and it was, , the culmination of a lot of testing, a lot of hard work , from Brian, from Glen D from day on from just all these remarkable fun.
00:09:17.692 --> 00:09:25.601
Schatz's focus on visualization techniques ultimately led him to join The Third Floor, one of Hollywood's most successful visualization companies.
00:09:25.655 --> 00:09:25.946
Yeah.
00:09:25.946 --> 00:09:28.926
So third floor was, , formed by , the five, , founders.
00:09:28.926 --> 00:09:33.639
So it's, , Chris, Josh, Barry, , Hirosi and, Nick, and, , I had been Mr.
00:09:33.950 --> 00:09:50.256
Solo previz guy , for many years, , after image G and before digital domain, I was just sort of using that, , mentality from the Teles, like I, up with a laptop and set up my monitor and you just start talking and think, you know, I was, I was loving it, but previous was becoming more popular.
00:09:50.716 --> 00:09:56.836
And I had lost a couple of gigs because I was only one guy and it went to, a previous company of sorts.
00:09:57.017 --> 00:10:05.443
And then when I met all the folks from third floor, I felt like we think alike, we all come from some kind of traditional live action, medium and really love.
00:10:05.543 --> 00:10:09.063
Previs a sort of a hybrid of traditional live action and computer graphics.
00:10:09.163 --> 00:10:10.504
So why would I compete with these people?
00:10:10.594 --> 00:10:11.703
Let's just all become friendly.
00:10:11.724 --> 00:10:12.923
And I just joined the third floor
00:10:13.274 --> 00:10:19.403
One of The Third Floor's most successful collaborations has been with the wildly popular movies of the Marvel Cinematic Universe.
00:10:19.456 --> 00:10:20.686
Marvel's a blast to work for.
00:10:20.686 --> 00:10:22.816
We've had a really good synergy with them for a while.
00:10:22.816 --> 00:10:22.995
Now.
00:10:22.995 --> 00:10:26.115
I think our first show with them was iron man, two a while back.
00:10:26.115 --> 00:10:29.365
And, they take their previous very seriously, which we love, of course.
00:10:29.572 --> 00:10:40.101
We're very lucky to just have these amazing previous soups Gerardo Ramirez, Shannon justice sins who have the ability to absorb the script and, break it down kind of into these component pieces.
00:10:40.131 --> 00:10:40.402
So.
00:10:40.658 --> 00:10:41.991
what assets need to be modeled?
00:10:41.991 --> 00:10:46.511
What animation cycles could we potentially reuse or what new ones might we need to make?
00:10:46.955 --> 00:10:49.975
What I've personally seen in terms of the approach, is starting soon.
00:10:50.769 --> 00:10:53.379
And progressing the complexity very carefully.
00:10:53.710 --> 00:11:08.335
So starting with storyboards and editing those into an animatic and those have temp music attached and right away you really get a quick sense of the pacing and the story, . Those might one by one become replaced with stills from Maya.
00:11:08.544 --> 00:11:14.028
This might have a . Version, one of the environment, the characters in the props are whatever state they're in but.
00:11:14.644 --> 00:11:18.292
rendering it through the legit focal length and the legit cameras.
00:11:18.597 --> 00:11:19.707
Then we might start doing.
00:11:20.118 --> 00:11:26.717
Posed a pose sort of animation where maybe they look like they're ice skating and they're going from point a to point B in the correct timing.
00:11:26.967 --> 00:11:29.347
, but these are all these simple iterations.
00:11:29.347 --> 00:11:35.067
The point of all of it is, we want to increase the complexity slowly, such that if any one moment.
00:11:35.577 --> 00:11:40.648
You were told, we've made a big change to the scene or the scene's been replaced by another one.
00:11:40.648 --> 00:11:46.227
Like whatever the situation may be, your heart's not going to be broken . Because you've put all this energy into the scene.
00:11:46.278 --> 00:11:49.168
And, another thing is we try to animate in master scenes.
00:11:49.168 --> 00:11:54.918
So as opposed to just animating , one shot at a time, we might try to have one file that encompasses an action beat.
00:11:55.398 --> 00:11:56.918
So this has a number of benefits.
00:11:56.918 --> 00:12:00.668
It tends to be better for distributing to artists with different skillsets.
00:12:00.668 --> 00:12:09.798
So, , let's pick on me for example, I'm not a particularly strong character animator at all, but I might be able to inherit that scene and then start doing camera and layout kind of stuff.
00:12:09.947 --> 00:12:12.528
Or maybe I would break the scene down into a tech.
00:12:13.258 --> 00:12:17.937
And start to think about, well, when we get to live action, how will we really shoot this on set?
00:12:18.278 --> 00:12:26.312
The importance , of teamwork with this whole workflow, can't be understated, especially on projects of this size.
00:12:26.312 --> 00:12:35.152
Schatz continually refined his approach to previs and virtual production over many Marvel movies but found one project that opened his eyes to new areas of inclusivity with other artists.
00:12:36.082 --> 00:12:38.863
The Avengers age of Tron, was a very, very, very.
00:12:39.462 --> 00:12:47.336
Rich experience an important part of the definition of virtual production for me is opening computer graphics to non-digital people.
00:12:47.426 --> 00:13:00.773
, so actors cinematographers, whatever it is, we will record your talents in the computer so that we can fuse all the years of experience that you have and said artistic, traditional medium inside of the world of computer graphics.
00:13:00.952 --> 00:13:08.293
And so something that really, really demonstrated this to its fullest was the first time that we put James Spader in the mocap suit.
00:13:08.513 --> 00:13:11.163
And so James Spader played, the bad guy Tron.
00:13:11.607 --> 00:13:18.937
And we put him in the XN suit and we had the live feed going into motion builder, and we had, one of the sets.
00:13:18.937 --> 00:13:20.977
It was very fiery and dark and scary.
00:13:21.638 --> 00:13:25.667
And, we had a big, , flat screen TV that was showing the motion builder output.
00:13:25.886 --> 00:13:32.187
And he asked me to arrange the perspective camera so that when he looked at the television, he was looking at himself.
00:13:32.187 --> 00:13:33.346
So it was essentially like a mirror.
00:13:33.647 --> 00:13:43.508
So the camera was about where his eyes were in the CG, in our, , and he just started looking at the TV and pacing and you could just see him inhabiting.
00:13:44.072 --> 00:13:45.802
This character and studying it.
00:13:45.822 --> 00:13:51.812
And he's, , this encyclopedia of knowledge of acting and, Shakespearean references.
00:13:51.852 --> 00:13:56.846
I mean, he's an amazingly smart guy and there were Marvel . Execs in the room but everyone just sort of fell silent.
00:13:57.067 --> 00:14:01.966
You could just see James started pacing , and becoming ultra on.
00:14:01.966 --> 00:14:07.527
And he was studying the movements and how he walked and how he articulated his body.
00:14:08.106 --> 00:14:08.647
And I thought.
00:14:09.447 --> 00:14:10.616
This is so cool.
00:14:11.076 --> 00:14:16.879
And I thought that was a really amazing moment to describe the power of virtual production
00:14:17.059 --> 00:14:24.669
Another project Schatz worked on was Game of Thrones, which became well-known for its seamless blend of gritty realism with fantasy elements like dragons.
00:14:24.716 --> 00:14:37.586
Something really remarkable about that show was this almost obsessive dedication about getting as much in camera as we can and come coming from previz and this sort of marriage of traditional photography and computer graphics.
00:14:37.586 --> 00:14:40.426
I've never been a fan of the fix it and post mentality.
00:14:40.775 --> 00:14:46.436
I always say fix it in pre, let's do our homework ahead of time and then go to set, with a battle plan and get that shot.
00:14:46.456 --> 00:14:52.346
And then computer graphics helps it later on rather than rescuing something that wasn't shot from the correct perp active.
00:14:52.725 --> 00:14:58.816
And so even when it came to,, shooting, , fire, , We were actually shooting real fire elements.
00:14:58.836 --> 00:15:07.326
And so I think the first gig with the game of Thrones was I think it was season four where, one, one, , the big, , giant, , walked past the camera.
00:15:07.426 --> 00:15:12.596
And so this was, , an old fashioned scale trick where we had an actor that was six feet tall.
00:15:13.005 --> 00:15:14.125
I tracked the live action element.
00:15:14.575 --> 00:15:16.945
that was shot with a fella running with a tennis ball.
00:15:17.004 --> 00:15:18.465
So that's how the eye line was done.
00:15:18.875 --> 00:15:24.414
Fella running with a tennis ball, tracked that, and then scaled it down to 50% and then , aligned the perspective.
00:15:24.514 --> 00:15:27.615
So when that monster walked by, he looked like?
00:15:27.735 --> 00:15:32.813
a 12 foot giant again, all old fashioned stuff, just being used in, in a smart way.
00:15:33.118 --> 00:15:38.118
Every trick in the book, whether it's old school or new school, we would pull it out and it was such a blast to do.
00:15:38.298 --> 00:15:43.717
And then, season five te de ended up being used for a lot of stuff in G O T anytime.
00:15:44.176 --> 00:15:46.865
, someone's looks like they're walking right into fire.
00:15:46.975 --> 00:16:04.577
Well, they were photographed and then they went away someplace safe, and then all the fire elements were there, but we kept the camera there and repeated the camera move,, Noah Kadner: The Mandalorian was another the interest in in-camera VFX and LED volume workflows in combination with other virtual production techniques.
00:16:04.650 --> 00:16:13.681
So on Mandalorian, third floor was working on the previ and I was sort of the, motion capture tech vis ambassador between, third floor and the, live action crew.
00:16:14.245 --> 00:16:19.154
is something else about virtual production, which I think is changing in an exciting way, is that.
00:16:19.538 --> 00:16:27.504
It's no longer this, separate discipline that you go down to this room and there's a grid on the floor and everything is gray and there's nothing wrong with mocap volumes.
00:16:27.504 --> 00:16:27.947
They're vital.
00:16:28.327 --> 00:16:34.706
But, , the idea that an animator in their office can say, I'm gonna throw the suit on and just jog back and forth and capture.
00:16:34.736 --> 00:16:37.216
This is also something that I think is really amazing.
00:16:37.317 --> 00:16:41.158
So we have a few of the, XN suits,, which is the inertial mocap system.
00:16:41.379 --> 00:16:48.581
, so really my role , was, getting those suits working in the previous pipeline and doing a lot of the, , , tech fi for when they're on the blurbs.
00:16:48.871 --> 00:16:49.869
, how that motion base.
00:16:50.318 --> 00:16:57.721
Then feeds into the corresponding background on the L E D screens because it has to move the other way to elicit that sense of, for travel.
00:16:58.172 --> 00:17:05.241
Schatz has come full circle for his most recent project and is working with many of the original team on the highly anticipated Avatar sequels.
00:17:05.300 --> 00:17:07.101
There really is a family vibe.
00:17:07.320 --> 00:17:11.141
, , , a lot of us , are still very good friends, from, , working on avatar one together.
00:17:11.317 --> 00:17:14.243
, and, , , it was probably inevitable that I would join it.
00:17:14.263 --> 00:17:15.923
It would be weird not to.
00:17:16.377 --> 00:17:23.817
What I think I love most about my job is being on set and not being , the know itall with the laptop and the fancy computer software.
00:17:23.887 --> 00:17:25.417
It's actually the exact opposite.
00:17:25.667 --> 00:17:33.617
, the way I see it is everyone else are experts at their respective disciplines, construction, cinematography, the gaffer, , the art har.
00:17:34.038 --> 00:17:49.738
So what I love most is when everyone's over my shoulder and everyone's got a different little bit of feedback, like, well, okay, so for stunts, they need the pick point to be over there, but oh, no, that's gonna get in the way of the great off that lighting is gonna set up or whatever it is.
00:17:50.157 --> 00:17:54.367
And we solve these problems digitally right in there.
00:17:55.353 --> 00:17:57.732
Before we really, and truly execute it on set.
00:17:57.833 --> 00:18:02.883
And , that feeling is way more gratifying than being the show off that has all the answers.
00:18:04.772 --> 00:18:11.792
Asked if he has advice for aspiring filmmakers and visual effects artists, Schatz suggests interacting with and observing the real world.
00:18:12.325 --> 00:18:20.234
Turn the computer off, get outside with real optics, with a still camera or a motion picture, camera of sorts, not your phone, not a digital thing.
00:18:20.234 --> 00:18:25.582
Like get something where you have to think about the shutter speed, the F stop, the ISO, how did these three things, , react?
00:18:25.582 --> 00:18:43.804
How do these things overlap and why when you change one thing, , does another have to be changed to compensate,, why, when you're close to something with a wide angle lens, and you're far away with a telephoto lens, you actually have the same field view, but obviously all the objects that there are parent distance from each other, it's a different energy.
00:18:44.241 --> 00:18:46.521
What would be conveyed from a narrative standpoint?
00:18:46.622 --> 00:18:49.295
One might look like, a voyeuristic, like a spy.
00:18:49.634 --> 00:19:06.384
And the other is more of a documentary feel where you're really there in the action, because as amazing as computer graphics is nothing will place the sort of magic of being on a live action set, , working with a crew, collaborating and having ideas and bouncing ideas off of each other.
00:19:06.605 --> 00:19:10.825
That's something that no computer techy thing will ever replace.
00:19:11.435 --> 00:19:25.718
Being on a live action, set, you open computer graphics up to these sort of happy accidents, which you normally wouldn't I think that's something really special and vital about virtual bringing that sort of messiness, so to speak, , from live action into the digital arts realm.
00:19:26.528 --> 00:19:34.057
Finally, as someone who's spent an entire career on the cutting edge of virtual production, Schatz is uniquely qualified to speculate about its future.
00:19:34.550 --> 00:19:42.500
For me the best way to look into the future is to actually look back at the patterns and milestones we've seen in recent history and kind of extrapolate that curve.
00:19:42.770 --> 00:19:50.060
So for instance, after avatar, the most common question was, Hey, that's really great, but what if we don't have that kind of big budget and rightfully?
00:19:50.060 --> 00:19:53.121
So this was seemingly outer space stuff on a big production.
00:19:53.480 --> 00:19:55.191
But the important thing is that the crew.
00:19:55.780 --> 00:19:57.550
Tech footprint got smaller.
00:19:57.851 --> 00:20:02.020
And many of us went on to real steel, which was not a big VFX driven movie.
00:20:02.290 --> 00:20:11.040
And we were able to stand on the shoulders of what we learned during avatar shortly later, , and cam arrived on the scene and that enabled you to, , track without an optical volume.
00:20:11.250 --> 00:20:17.951
So the theme, I guess, is that tools that were seemingly inaccessible only a few years prior are now being utilized on more and more.
00:20:18.890 --> 00:20:21.709
So what we hopefully did for Simon cam on avatar.
00:20:22.449 --> 00:20:25.659
Clearly Mandalorian has done that for led volumes today.
00:20:25.868 --> 00:20:28.588
And so this also goes for, , like individual pieces of tech as well.
00:20:28.588 --> 00:20:31.409
So a big one for me personally, has been LIDAR and photogrammetry.
00:20:31.903 --> 00:20:50.270
Which only a few years ago were really prohibitively expensive, not just financially, but like in terms of time, but we got the Leica BLK and , I've just been obsessively, scanning everything ever since, you know, I can have a mesh in my scene half an hour, and we've been using that here a lot for like technical layout, which then in.
00:20:50.796 --> 00:20:52.205
The simulcasting lineup.
00:20:52.655 --> 00:20:56.836
So there's this really cool feedback loop between reality and digital and inform each other.
00:20:57.165 --> 00:21:01.113
And now even the iPad pros can do LIDAR scanning, which is just amazing.
00:21:01.411 --> 00:21:10.161
And I think also in fairness, I think the XN suits have done a similar thing for motion capture, where you can get, really good capture without an optical volume.
00:21:10.471 --> 00:21:13.800
There's always a need for good mo-cap volumes and we're using them here daily.
00:21:14.010 --> 00:21:19.490
But I think what excites me is being able to do this stuff from home or the office, , or even right there on set.
00:21:19.820 --> 00:21:23.871
And, just opening computer graphics up to more artists from various disciplines.
00:21:24.201 --> 00:21:29.240
So what we're seeing is kind of this blurring of the lines between virtual production, previous and tech.
00:21:29.875 --> 00:21:32.965
Such that maybe what category each fits into.
00:21:32.965 --> 00:21:34.195
It doesn't really matter anymore.
00:21:34.195 --> 00:21:39.056
We just have this amazing toolbox and we are just going to pick what's right for the job.
00:21:39.457 --> 00:21:41.957
You've been listening to the virtual production podcast.
00:21:42.049 --> 00:21:44.839
Special thanks to my guest Casey Schatz for joining us.
00:00:00.000 --> 00:00:05.049
Casey Schatz is a veteran expert in virtual production techniques for motion pictures and television.
00:00:05.166 --> 00:00:11.006
His credits include Avatar, Game of Thrones, Avengers: Age of Ultron, The Martian and more.
00:00:11.064 --> 00:00:16.234
As Head of Virtual Production at The Third Floor, Schatz promotes the design and use of on-set simulcam.
00:00:16.370 --> 00:00:21.629
As with many of our successful guests, his fascination with technology started at an early age.
00:00:21.675 --> 00:00:27.024
Something clicked, very early on, I mean, maybe nine or 10 years old with a Commodor 64.
00:00:27.085 --> 00:00:32.945
And you could do these particular keystrokes where you could make a Siam magenta yellow, or a red green, or blue cube.
00:00:33.484 --> 00:00:50.591
And I would just sit in front of that thing for hours and at the same time, my dad did a lot of documentary work for the news station he was working at and, this fascination with optics and film and, traditional photography was kind of growing concurrently , with computer graphics and that theme.
00:00:50.741 --> 00:00:52.822
Really has never left and still hasn't frankly,
00:00:53.542 --> 00:01:00.222
Schatz looked to gain education and experience initially as a cinematographer but found himself drawn to other areas of filmmaking.
00:01:00.381 --> 00:01:17.816
I went to Cal arts and I got denied by the cinematography department, but got accepted into the still photography and the animation department, and Cal arts ended up being the perfect place because they let me sort of customize my own degree where I did theater lighting, traditional photography, cinematography.
00:01:17.891 --> 00:01:22.006
And then of course I was always in the F 1 0 5 lab, the computer graphics lab.
00:01:23.090 --> 00:01:39.662
And I had no idea what previz or techvis or virtual production was, at the time, but looking back, I can't think of a better education because it was so multidisciplinary and, fusing, modern, nifty computer graphic stuff with traditional old fashioned, filmmaking principles.
00:01:40.046 --> 00:01:42.305
And it's not that one is better than the other.
00:01:42.436 --> 00:01:44.956
It's where they shake hands that I think is so fascinating.
00:01:45.176 --> 00:01:52.906
Like many other filmmakers in his peer group, Schatz was inspired by a string of blockbuster visual effects films that pushed the technology of the time forward.
00:01:53.483 --> 00:02:00.034
were in high school when jurasic park and teach, who came out, which were okay, tick the box career done.
00:02:00.063 --> 00:02:04.084
Like I was just so enamored with the work and that work still holds up to this day, frankly.
00:02:04.593 --> 00:02:08.913
, so as far as soul searching and career, like, that was almost, never an issue.
00:02:09.270 --> 00:02:16.021
So then the only question afterwards was which field within it, the first school I went to was the academy of art, San Francisco.
00:02:16.361 --> 00:02:22.611
And I was working on modeling, a human face and, you know, putting, specul reflections on the eyes.
00:02:22.632 --> 00:02:26.581
And we were trying to do what at the time was extremely primitive, hair and stuff.
00:02:26.722 --> 00:02:38.181
And I had this kind of moment of like, why am I trying so hard to recreate reality when reality is just right outside the door and so I kind of had this phase actually where I just stopped doing computer graphics and really thought.
00:02:39.026 --> 00:03:03.733
I'm gonna be a national geographic photographer or a traditional cinematographer, and I got a light meter and some spotlights and I really thought maybe I'm going down the career path and the computer graphics bug, bit again, you know, there was no way I was ever gonna give it up, but I think looking back, it was good because now what we see with virtual production is these worlds, shaking hands, , the principles of live action filmmaking that have existed since the Lumyer brothers.
00:03:04.240 --> 00:03:09.610
You have that on one side and then you have, unreal engine and VR headsets, but who cares at the end of the day?
00:03:09.610 --> 00:03:10.569
We're all making stories.
00:03:10.620 --> 00:03:12.330
We're putting images on the screen.
00:03:12.710 --> 00:03:14.659
So what medium, does it really matter?
00:03:14.719 --> 00:03:22.659
And there's so many great between these extremes that, to categorize them is really of secondary importance to just using the tool that's right.
00:03:22.840 --> 00:03:24.180
For the client and for the job.
00:03:24.717 --> 00:03:31.318
Schatz embraced the multidisciplinary nature of his education and, as a result, found his skills in demand right away.
00:03:31.467 --> 00:03:35.343
So every year, , the Kal arts animation department has the end of year producer show.
00:03:35.764 --> 00:03:38.289
And my short film showed at the producer's showcase.
00:03:38.622 --> 00:03:46.566
. And the hallway afterwards, , Tom bear who started image G, which is a motion control company, came up and said, can you start Monday basically?
00:03:46.656 --> 00:03:47.295
I was so excited.
00:03:47.415 --> 00:03:53.436
I hardly knew what MOS control was and I didn't even really know what previ was It was still a very young, art form at the time.
00:03:53.975 --> 00:04:03.808
And when I started there, I started to do previs for car commercial that were being used with this huge, like three ton motion control camera, crane.
00:04:04.269 --> 00:04:08.381
And the way that the offices were set up, , there was a window and then the rig was in the parking lot.
00:04:08.721 --> 00:04:10.260
So literally as I was.
00:04:10.975 --> 00:04:17.026
Animating the real thing that weighed three tons and had to move was right there next to me.
00:04:17.206 --> 00:04:31.185
And so that instilled this sort of discipline to not get too seduced by the tools that the computer had, like no gravity and, velocity was not a, a concern, , so I started prevising with real equipment in mind.
00:04:31.642 --> 00:04:36.353
And the two things have always been embedded to do previs for things that are actually suitable.
00:04:36.572 --> 00:04:37.442
And if if they're not.
00:04:37.987 --> 00:04:38.968
That's fine.
00:04:39.048 --> 00:04:41.810
I mean, let's have a superhero take off and fly to outer space.
00:04:41.920 --> 00:04:52.122
But let's then break it down into its constituent plates and think about how we're gonna photograph this the worst thing you could do in previz is to make pixels move on this green in a certain way.
00:04:52.382 --> 00:04:57.562
And the director says, Yippy, then you get to set and you realize there's no bearing and reality with this.
00:04:57.562 --> 00:04:58.642
We can't accomplish this.
00:04:59.119 --> 00:05:03.238
the, The computer a person didn't have a respect for the physics and the parameters of live action shooting.
00:05:03.749 --> 00:05:12.399
Along with learning about the mechanics of motion control and how it should inform accurate previs, Schatz also took lessons from how to operate in front of clients professionally.
00:05:12.512 --> 00:05:18.127
So I had, been the cinematographer on this, music video and we went to a Tein place in Hollywood.
00:05:18.127 --> 00:05:19.687
It was a really swanky, fancy place.
00:05:20.218 --> 00:05:21.817
what really changed me was seeing.
00:05:22.298 --> 00:05:23.557
The operator work.
00:05:23.901 --> 00:05:31.478
we were just sitting in these, really cushy chairs and talking about the color balances and, and let's add a point of, know, red and,, subtract a point of this and that.
00:05:31.898 --> 00:05:36.401
And the person was just doing it and you could see, the impact in real time.
00:05:37.052 --> 00:05:37.752
And I.
00:05:37.935 --> 00:05:40.350
Thought . why would 3d graphics be any different?
00:05:40.810 --> 00:05:46.750
So , all right, I just need to get fast enough at Maya so that the client can be in the cushy to chair behind me.
00:05:46.970 --> 00:05:47.949
And I'm just like that.
00:05:47.949 --> 00:05:50.990
Tell any guy I'm moving the mouse in the keyboard really fast.
00:05:51.370 --> 00:06:00.237
And the image is just forming sort of like developing in the dark room and if I failed to do something really quickly, While live with a client, I would say, well, that sucked.
00:06:00.237 --> 00:06:04.276
How can I automate that in, you know, Mel and Python, I would write my own little tools.
00:06:04.857 --> 00:06:09.627
Jump cut to now a lot of that's happening with virtual production, because rather than them saying.
00:06:10.341 --> 00:06:10.632
Okay.
00:06:10.641 --> 00:06:12.612
Let's boom, up let's counter tilt down.
00:06:12.682 --> 00:06:18.021
We could actually hand them an iPad or something that's, optically captured and say, well, go for it.
00:06:18.081 --> 00:06:26.595
But also, there are still folks that are shy about the equipment and want to just sit on that couch or whatever, and, sort of dictate what their, vision is.
00:06:26.595 --> 00:06:29.035
And everyone works differently and one's not better than the other.
00:06:29.975 --> 00:06:39.884
Schatz excelled at his work with creatives and was ultimately offered a position on Avatar, a movie that would become known for its pioneering work in motion capture and virtual cinematography.
00:06:40.009 --> 00:06:41.389
I was like, I haven't done that.
00:06:42.089 --> 00:06:47.860
Project yet where I worked with one of the master directors, you know, and I, wanted to see if I could do it, if I was up for it.
00:06:48.110 --> 00:06:51.031
Just to even be in the same building as Jim Cameron was gonna be an honor.
00:06:51.211 --> 00:06:58.161
So talk about trial by fire for virtual production, had Some experience editing mocap, but had never been on a mocap stage.
00:06:58.221 --> 00:07:06.524
So on day one to see horror horses in mocap suits and then the actors also in the mocap suits, riding the horses and they're, they're going full speed down.
00:07:06.524 --> 00:07:09.644
The length of the stage was, a pretty remarkable, education.
00:07:10.035 --> 00:07:13.663
I ended up being like, I think my, credit on the film was SEL cam supervisor.
00:07:13.860 --> 00:07:15.360
So simultaneous camera.
00:07:15.610 --> 00:07:26.790
This would be sort of the art and science of capturing the motion picture camera much like you would, , a person by putting funny dots on their body and , they get wronged and calibrated and everything let's do that same thing to the camera.
00:07:26.850 --> 00:07:31.250
So those, active markers are put onto the camera and that camera is being captured.
00:07:31.737 --> 00:07:38.315
So I gots super duper involved in thes cam and also was also doing a lot of the motion based motion control camera.
00:07:38.980 --> 00:07:40.968
A lot of the tricky, live action setups.
00:07:41.305 --> 00:07:44.605
So yeah, I was on avatar for three and a half or four years.
00:07:44.725 --> 00:07:49.298
I mean, that was a life changing experience, getting to learn from, Jim and all the other, folks on the crew.
00:07:49.798 --> 00:07:52.718
And, I still watched the movie, like, did we do that.
00:07:52.718 --> 00:07:53.718
How did we pull this off?
00:07:53.778 --> 00:07:53.928
You
00:07:54.360 --> 00:07:59.721
The Avatar team invented many techniques which are now considered standard practice in the virtual production world.
00:07:59.949 --> 00:08:12.038
I'll never forget The first really successful moment on thes cam, which is the shot where, net Teri goes up to the shack because the, real Jake is, breathing the Pandora and air and, suffer.
00:08:12.458 --> 00:08:16.437
And so she runs up to the shack, looks through the window, sees him.
00:08:17.088 --> 00:08:22.827
Goes to the stairs, jumps and holds what is now, the live action, same Worthington character.
00:08:23.247 --> 00:08:24.567
And it was very, very tough.
00:08:24.964 --> 00:08:28.584
our virtual production engineer and all around just brilliant dude.
00:08:28.675 --> 00:08:36.424
Brian Champney, he and others, made this, optical volume in a very claustrophobic situation, with optical volumes, you want a lot of space.
00:08:36.445 --> 00:08:46.160
You want the, field of view of the lenses to have a nice spread so that they're encompassing a lot and overlapping area, and that's very tough and a very claustrophobic cuz the shack was real.
00:08:46.581 --> 00:08:48.441
The art department made . A real shack.
00:08:49.039 --> 00:08:58.354
And, we got it working and the live action cameraman, I think it was Dave Ericks, was able to see net teary and compose the shot with a CG character.
00:08:58.677 --> 00:09:00.649
As good as an operator as Dave is.
00:09:00.669 --> 00:09:17.402
And, and many camera operators, are it's hard to perfectly imagine it and do it without seeing the digital element and that was kind of the moment that we really got it and it was, , the culmination of a lot of testing, a lot of hard work , from Brian, from Glen D from day on from just all these remarkable fun.
00:09:17.692 --> 00:09:25.601
Schatz's focus on visualization techniques ultimately led him to join The Third Floor, one of Hollywood's most successful visualization companies.
00:09:25.655 --> 00:09:25.946
Yeah.
00:09:25.946 --> 00:09:28.926
So third floor was, , formed by , the five, , founders.
00:09:28.926 --> 00:09:33.639
So it's, , Chris, Josh, Barry, , Hirosi and, Nick, and, , I had been Mr.
00:09:33.950 --> 00:09:50.256
Solo previz guy , for many years, , after image G and before digital domain, I was just sort of using that, , mentality from the Teles, like I, up with a laptop and set up my monitor and you just start talking and think, you know, I was, I was loving it, but previous was becoming more popular.
00:09:50.716 --> 00:09:56.836
And I had lost a couple of gigs because I was only one guy and it went to, a previous company of sorts.
00:09:57.017 --> 00:10:05.443
And then when I met all the folks from third floor, I felt like we think alike, we all come from some kind of traditional live action, medium and really love.
00:10:05.543 --> 00:10:09.063
Previs a sort of a hybrid of traditional live action and computer graphics.
00:10:09.163 --> 00:10:10.504
So why would I compete with these people?
00:10:10.594 --> 00:10:11.703
Let's just all become friendly.
00:10:11.724 --> 00:10:12.923
And I just joined the third floor
00:10:13.274 --> 00:10:19.403
One of The Third Floor's most successful collaborations has been with the wildly popular movies of the Marvel Cinematic Universe.
00:10:19.456 --> 00:10:20.686
Marvel's a blast to work for.
00:10:20.686 --> 00:10:22.816
We've had a really good synergy with them for a while.
00:10:22.816 --> 00:10:22.995
Now.
00:10:22.995 --> 00:10:26.115
I think our first show with them was iron man, two a while back.
00:10:26.115 --> 00:10:29.365
And, they take their previous very seriously, which we love, of course.
00:10:29.572 --> 00:10:40.101
We're very lucky to just have these amazing previous soups Gerardo Ramirez, Shannon justice sins who have the ability to absorb the script and, break it down kind of into these component pieces.
00:10:40.131 --> 00:10:40.402
So.
00:10:40.658 --> 00:10:41.991
what assets need to be modeled?
00:10:41.991 --> 00:10:46.511
What animation cycles could we potentially reuse or what new ones might we need to make?
00:10:46.955 --> 00:10:49.975
What I've personally seen in terms of the approach, is starting soon.
00:10:50.769 --> 00:10:53.379
And progressing the complexity very carefully.
00:10:53.710 --> 00:11:08.335
So starting with storyboards and editing those into an animatic and those have temp music attached and right away you really get a quick sense of the pacing and the story, . Those might one by one become replaced with stills from Maya.
00:11:08.544 --> 00:11:14.028
This might have a . Version, one of the environment, the characters in the props are whatever state they're in but.
00:11:14.644 --> 00:11:18.292
rendering it through the legit focal length and the legit cameras.
00:11:18.597 --> 00:11:19.707
Then we might start doing.
00:11:20.118 --> 00:11:26.717
Posed a pose sort of animation where maybe they look like they're ice skating and they're going from point a to point B in the correct timing.
00:11:26.967 --> 00:11:29.347
, but these are all these simple iterations.
00:11:29.347 --> 00:11:35.067
The point of all of it is, we want to increase the complexity slowly, such that if any one moment.
00:11:35.577 --> 00:11:40.648
You were told, we've made a big change to the scene or the scene's been replaced by another one.
00:11:40.648 --> 00:11:46.227
Like whatever the situation may be, your heart's not going to be broken . Because you've put all this energy into the scene.
00:11:46.278 --> 00:11:49.168
And, another thing is we try to animate in master scenes.
00:11:49.168 --> 00:11:54.918
So as opposed to just animating , one shot at a time, we might try to have one file that encompasses an action beat.
00:11:55.398 --> 00:11:56.918
So this has a number of benefits.
00:11:56.918 --> 00:12:00.668
It tends to be better for distributing to artists with different skillsets.
00:12:00.668 --> 00:12:09.798
So, , let's pick on me for example, I'm not a particularly strong character animator at all, but I might be able to inherit that scene and then start doing camera and layout kind of stuff.
00:12:09.947 --> 00:12:12.528
Or maybe I would break the scene down into a tech.
00:12:13.258 --> 00:12:17.937
And start to think about, well, when we get to live action, how will we really shoot this on set?
00:12:18.278 --> 00:12:26.312
The importance , of teamwork with this whole workflow, can't be understated, especially on projects of this size.
00:12:26.312 --> 00:12:35.152
Schatz continually refined his approach to previs and virtual production over many Marvel movies but found one project that opened his eyes to new areas of inclusivity with other artists.
00:12:36.082 --> 00:12:38.863
The Avengers age of Tron, was a very, very, very.
00:12:39.462 --> 00:12:47.336
Rich experience an important part of the definition of virtual production for me is opening computer graphics to non-digital people.
00:12:47.426 --> 00:13:00.773
, so actors cinematographers, whatever it is, we will record your talents in the computer so that we can fuse all the years of experience that you have and said artistic, traditional medium inside of the world of computer graphics.
00:13:00.952 --> 00:13:08.293
And so something that really, really demonstrated this to its fullest was the first time that we put James Spader in the mocap suit.
00:13:08.513 --> 00:13:11.163
And so James Spader played, the bad guy Tron.
00:13:11.607 --> 00:13:18.937
And we put him in the XN suit and we had the live feed going into motion builder, and we had, one of the sets.
00:13:18.937 --> 00:13:20.977
It was very fiery and dark and scary.
00:13:21.638 --> 00:13:25.667
And, we had a big, , flat screen TV that was showing the motion builder output.
00:13:25.886 --> 00:13:32.187
And he asked me to arrange the perspective camera so that when he looked at the television, he was looking at himself.
00:13:32.187 --> 00:13:33.346
So it was essentially like a mirror.
00:13:33.647 --> 00:13:43.508
So the camera was about where his eyes were in the CG, in our, , and he just started looking at the TV and pacing and you could just see him inhabiting.
00:13:44.072 --> 00:13:45.802
This character and studying it.
00:13:45.822 --> 00:13:51.812
And he's, , this encyclopedia of knowledge of acting and, Shakespearean references.
00:13:51.852 --> 00:13:56.846
I mean, he's an amazingly smart guy and there were Marvel . Execs in the room but everyone just sort of fell silent.
00:13:57.067 --> 00:14:01.966
You could just see James started pacing , and becoming ultra on.
00:14:01.966 --> 00:14:07.527
And he was studying the movements and how he walked and how he articulated his body.
00:14:08.106 --> 00:14:08.647
And I thought.
00:14:09.447 --> 00:14:10.616
This is so cool.
00:14:11.076 --> 00:14:16.879
And I thought that was a really amazing moment to describe the power of virtual production
00:14:17.059 --> 00:14:24.669
Another project Schatz worked on was Game of Thrones, which became well-known for its seamless blend of gritty realism with fantasy elements like dragons.
00:14:24.716 --> 00:14:37.586
Something really remarkable about that show was this almost obsessive dedication about getting as much in camera as we can and come coming from previz and this sort of marriage of traditional photography and computer graphics.
00:14:37.586 --> 00:14:40.426
I've never been a fan of the fix it and post mentality.
00:14:40.775 --> 00:14:46.436
I always say fix it in pre, let's do our homework ahead of time and then go to set, with a battle plan and get that shot.
00:14:46.456 --> 00:14:52.346
And then computer graphics helps it later on rather than rescuing something that wasn't shot from the correct perp active.
00:14:52.725 --> 00:14:58.816
And so even when it came to,, shooting, , fire, , We were actually shooting real fire elements.
00:14:58.836 --> 00:15:07.326
And so I think the first gig with the game of Thrones was I think it was season four where, one, one, , the big, , giant, , walked past the camera.
00:15:07.426 --> 00:15:12.596
And so this was, , an old fashioned scale trick where we had an actor that was six feet tall.
00:15:13.005 --> 00:15:14.125
I tracked the live action element.
00:15:14.575 --> 00:15:16.945
that was shot with a fella running with a tennis ball.
00:15:17.004 --> 00:15:18.465
So that's how the eye line was done.
00:15:18.875 --> 00:15:24.414
Fella running with a tennis ball, tracked that, and then scaled it down to 50% and then , aligned the perspective.
00:15:24.514 --> 00:15:27.615
So when that monster walked by, he looked like?
00:15:27.735 --> 00:15:32.813
a 12 foot giant again, all old fashioned stuff, just being used in, in a smart way.
00:15:33.118 --> 00:15:38.118
Every trick in the book, whether it's old school or new school, we would pull it out and it was such a blast to do.
00:15:38.298 --> 00:15:43.717
And then, season five te de ended up being used for a lot of stuff in G O T anytime.
00:15:44.176 --> 00:15:46.865
, someone's looks like they're walking right into fire.
00:15:46.975 --> 00:16:04.577
Well, they were photographed and then they went away someplace safe, and then all the fire elements were there, but we kept the camera there and repeated the camera move,, Noah Kadner: The Mandalorian was another the interest in in-camera VFX and LED volume workflows in combination with other virtual production techniques.
00:16:04.650 --> 00:16:13.681
So on Mandalorian, third floor was working on the previ and I was sort of the, motion capture tech vis ambassador between, third floor and the, live action crew.
00:16:14.245 --> 00:16:19.154
is something else about virtual production, which I think is changing in an exciting way, is that.
00:16:19.538 --> 00:16:27.504
It's no longer this, separate discipline that you go down to this room and there's a grid on the floor and everything is gray and there's nothing wrong with mocap volumes.
00:16:27.504 --> 00:16:27.947
They're vital.
00:16:28.327 --> 00:16:34.706
But, , the idea that an animator in their office can say, I'm gonna throw the suit on and just jog back and forth and capture.
00:16:34.736 --> 00:16:37.216
This is also something that I think is really amazing.
00:16:37.317 --> 00:16:41.158
So we have a few of the, XN suits,, which is the inertial mocap system.
00:16:41.379 --> 00:16:48.581
, so really my role , was, getting those suits working in the previous pipeline and doing a lot of the, , , tech fi for when they're on the blurbs.
00:16:48.871 --> 00:16:49.869
, how that motion base.
00:16:50.318 --> 00:16:57.721
Then feeds into the corresponding background on the L E D screens because it has to move the other way to elicit that sense of, for travel.
00:16:58.172 --> 00:17:05.241
Schatz has come full circle for his most recent project and is working with many of the original team on the highly anticipated Avatar sequels.
00:17:05.300 --> 00:17:07.101
There really is a family vibe.
00:17:07.320 --> 00:17:11.141
, , , a lot of us , are still very good friends, from, , working on avatar one together.
00:17:11.317 --> 00:17:14.243
, and, , , it was probably inevitable that I would join it.
00:17:14.263 --> 00:17:15.923
It would be weird not to.
00:17:16.377 --> 00:17:23.817
What I think I love most about my job is being on set and not being , the know itall with the laptop and the fancy computer software.
00:17:23.887 --> 00:17:25.417
It's actually the exact opposite.
00:17:25.667 --> 00:17:33.617
, the way I see it is everyone else are experts at their respective disciplines, construction, cinematography, the gaffer, , the art har.
00:17:34.038 --> 00:17:49.738
So what I love most is when everyone's over my shoulder and everyone's got a different little bit of feedback, like, well, okay, so for stunts, they need the pick point to be over there, but oh, no, that's gonna get in the way of the great off that lighting is gonna set up or whatever it is.
00:17:50.157 --> 00:17:54.367
And we solve these problems digitally right in there.
00:17:55.353 --> 00:17:57.732
Before we really, and truly execute it on set.
00:17:57.833 --> 00:18:02.883
And , that feeling is way more gratifying than being the show off that has all the answers.
00:18:04.772 --> 00:18:11.792
Asked if he has advice for aspiring filmmakers and visual effects artists, Schatz suggests interacting with and observing the real world.
00:18:12.325 --> 00:18:20.234
Turn the computer off, get outside with real optics, with a still camera or a motion picture, camera of sorts, not your phone, not a digital thing.
00:18:20.234 --> 00:18:25.582
Like get something where you have to think about the shutter speed, the F stop, the ISO, how did these three things, , react?
00:18:25.582 --> 00:18:43.804
How do these things overlap and why when you change one thing, , does another have to be changed to compensate,, why, when you're close to something with a wide angle lens, and you're far away with a telephoto lens, you actually have the same field view, but obviously all the objects that there are parent distance from each other, it's a different energy.
00:18:44.241 --> 00:18:46.521
What would be conveyed from a narrative standpoint?
00:18:46.622 --> 00:18:49.295
One might look like, a voyeuristic, like a spy.
00:18:49.634 --> 00:19:06.384
And the other is more of a documentary feel where you're really there in the action, because as amazing as computer graphics is nothing will place the sort of magic of being on a live action set, , working with a crew, collaborating and having ideas and bouncing ideas off of each other.
00:19:06.605 --> 00:19:10.825
That's something that no computer techy thing will ever replace.
00:19:11.435 --> 00:19:25.718
Being on a live action, set, you open computer graphics up to these sort of happy accidents, which you normally wouldn't I think that's something really special and vital about virtual bringing that sort of messiness, so to speak, , from live action into the digital arts realm.
00:19:26.528 --> 00:19:34.057
Finally, as someone who's spent an entire career on the cutting edge of virtual production, Schatz is uniquely qualified to speculate about its future.
00:19:34.550 --> 00:19:42.500
For me the best way to look into the future is to actually look back at the patterns and milestones we've seen in recent history and kind of extrapolate that curve.
00:19:42.770 --> 00:19:50.060
So for instance, after avatar, the most common question was, Hey, that's really great, but what if we don't have that kind of big budget and rightfully?
00:19:50.060 --> 00:19:53.121
So this was seemingly outer space stuff on a big production.
00:19:53.480 --> 00:19:55.191
But the important thing is that the crew.
00:19:55.780 --> 00:19:57.550
Tech footprint got smaller.
00:19:57.851 --> 00:20:02.020
And many of us went on to real steel, which was not a big VFX driven movie.
00:20:02.290 --> 00:20:11.040
And we were able to stand on the shoulders of what we learned during avatar shortly later, , and cam arrived on the scene and that enabled you to, , track without an optical volume.
00:20:11.250 --> 00:20:17.951
So the theme, I guess, is that tools that were seemingly inaccessible only a few years prior are now being utilized on more and more.
00:20:18.890 --> 00:20:21.709
So what we hopefully did for Simon cam on avatar.
00:20:22.449 --> 00:20:25.659
Clearly Mandalorian has done that for led volumes today.
00:20:25.868 --> 00:20:28.588
And so this also goes for, , like individual pieces of tech as well.
00:20:28.588 --> 00:20:31.409
So a big one for me personally, has been LIDAR and photogrammetry.
00:20:31.903 --> 00:20:50.270
Which only a few years ago were really prohibitively expensive, not just financially, but like in terms of time, but we got the Leica BLK and , I've just been obsessively, scanning everything ever since, you know, I can have a mesh in my scene half an hour, and we've been using that here a lot for like technical layout, which then in.
00:20:50.796 --> 00:20:52.205
The simulcasting lineup.
00:20:52.655 --> 00:20:56.836
So there's this really cool feedback loop between reality and digital and inform each other.
00:20:57.165 --> 00:21:01.113
And now even the iPad pros can do LIDAR scanning, which is just amazing.
00:21:01.411 --> 00:21:10.161
And I think also in fairness, I think the XN suits have done a similar thing for motion capture, where you can get, really good capture without an optical volume.
00:21:10.471 --> 00:21:13.800
There's always a need for good mo-cap volumes and we're using them here daily.
00:21:14.010 --> 00:21:19.490
But I think what excites me is being able to do this stuff from home or the office, , or even right there on set.
00:21:19.820 --> 00:21:23.871
And, just opening computer graphics up to more artists from various disciplines.
00:21:24.201 --> 00:21:29.240
So what we're seeing is kind of this blurring of the lines between virtual production, previous and tech.
00:21:29.875 --> 00:21:32.965
Such that maybe what category each fits into.
00:21:32.965 --> 00:21:34.195
It doesn't really matter anymore.
00:21:34.195 --> 00:21:39.056
We just have this amazing toolbox and we are just going to pick what's right for the job.
00:21:39.457 --> 00:21:41.957
You've been listening to the virtual production podcast.
00:21:42.049 --> 00:21:44.839
Special thanks to my guest Casey Schatz for joining us.
