WEBVTT
1
00:00:00.100 --> 00:00:10.000
Can you set up Blender MCP? I gave it these photos, and I said, "Can you build the Culver Theater and the surrounding area in Blender using the Blender MCP?" So I'm like, "Okay, go do the thing."
2
00:00:10.160 --> 00:00:25.860
And then here's the thing. Oh, wow. Welcome back to Denoised. Addy, how you doing? We're not, we're not in person anymore this time. I know people enjoyed that we were in person, but sorry, we're back, we're back remote.
3
00:00:26.740 --> 00:00:37.060
It's too hot to drive over to that side of town. [laughs] Your tire- your tires will melt. [laughs] All right, so we got SIGGRAPH coming up next week. Yep. I'm excited about this. This is my first SIGGRAPH.
4
00:00:37.100 --> 00:00:45.760
It's very convenient that it's here in LA this year. You gonna hit it up? Yeah, I'll be there Monday morning. Uh, if any of our viewers are there, love to say hi. Um- Yeah, let us know... just look for me. Yeah.
5
00:00:46.120 --> 00:00:56.440
Let me see. There's the Academy Open Source Day on Sunday, so I'll be a- attending that. And then, yeah, most likely Monday, Tuesday. Okay. For, like- I might join you for the Sunday one, actually.
6
00:00:56.880 --> 00:01:00.880
Yeah, come by the Sunday one. Have you, what, have you been to SIGGRAPH before? Oh, so many of them. Yeah.
7
00:01:01.000 --> 00:01:15.520
I mean, it's such a, it's such a mainstay conference for those in, especially on the creative technology side, the bleeding edge side of media and entertainment, because a lot of the newer research papers get announced there from- Mm-hmm...
8
00:01:15.540 --> 00:01:27.740
universities, uh, research labs, and then the new tools or the upcoming tools from the companies get announced there. Just, it, it's like NAB for the uber nerds, if you will. [laughs] Yeah.
9
00:01:28.260 --> 00:01:31.730
So, like, NAB, NAB has, like, the big releases from, like, the big companies.
10
00:01:31.780 --> 00:01:39.300
Like, but then that, like, one thing they're c- really not sure about, they're really experimenting and betting big on, that's gonna be at SIGGRAPH.
11
00:01:39.560 --> 00:01:53.860
Is it more of a talk kinda conference, or a expo floor conference, or, like, both? Yeah, I think the bulk of the floor space is actually the research, uh, entities presenting their findings.
12
00:01:53.980 --> 00:02:00.820
So, like, you'll have university- Okay... graduate students, and you'll see stuff that you're like, "Oh my God, this is like five years out, 10 years out."
13
00:02:01.040 --> 00:02:11.039
Like, for example, the early days of VR, that was the first place I saw haptic touch stuff- Mm. Okay... was at, like, SIGGRAPH 2016 or 2015. Yeah.
14
00:02:11.840 --> 00:02:22.620
So, like, uh, some stuff will fizzle out and never really make it into, uh, a business, but then some stuff will stick. Um, it, it's, it's a glimpse into the future of media and entertainment, in my opinion. Yeah.
15
00:02:22.700 --> 00:02:23.700
I'm excited to check it out.
16
00:02:23.820 --> 00:02:36.609
Uh, there was, like, an LA SIGGRAPH event a few weeks ago, and, um, uh, Netflix did a teaser of some of the stuff they're gonna present and, uh, some very cool papers and some very cool research that they're doing that- Yeah...
17
00:02:36.620 --> 00:02:42.780
I'm excited to see get out there. I, I, I can imagine Netflix research, uh, Netflix Research announcing new stuff there.
18
00:02:43.260 --> 00:02:53.609
It's also very strategic on their part 'cause, uh, you have a high density of really amazing AI researchers. Mm-hmm. So if you wanna hire them, if you wanna woo them, you know, that's the place to do it.
19
00:02:53.840 --> 00:03:00.400
You, you have recruiters out there at the same time. Um, if you're looking for work, I think that that's a good place to connect.
20
00:03:00.620 --> 00:03:09.040
Um, so all in all, the fact that it's in LA, and it's a primarily media and entertainment-based thing, uh, those two things will play well together. Yeah. Yeah.
21
00:03:09.140 --> 00:03:18.790
Uh, I am excited for it, and then outside of the official conference, there's a bunch of satellite events, one of which, if you are in LA and wanna come by, you don't need a SIGGRAPH pass for this, and it's free.
22
00:03:18.820 --> 00:03:28.760
We are at the... It's not the sexiest name, but the AI Workflow Summit. [laughs] Yeah. Under the hood, there's gonna be a lot of cool stuff here.
23
00:03:28.980 --> 00:03:38.140
Um, it's at, not officially Kinoflo, but it's at their, like, satellite e- uh- Yeah. They're, they have giant warehouse space in Burbank... R&D warehouse. Yeah. It's a cool spot. Yeah.
24
00:03:39.040 --> 00:03:49.899
Um, but there's gonna be some cool stuff. Some presentations on state-of-Gaussian splats from NVIDIA, uh, real-time biometric workflows, and one that I'm very excited and curious about is, uh, Ramiro's gonna be...
25
00:03:49.960 --> 00:03:56.860
He's been doing, like, a lot of work with Comfy UI- Mm-hmm... and agentic Comfy UI workflows, and so he's rolling out
26
00:03:57.840 --> 00:04:12.530
comfycode.ai, which from what he kinda briefly told me, but we'll get a lot more info there, is kind of like a MCP sidecar for Comfy that sounds crazier than the official Comfy MCP. This one could work local. This one
27
00:04:13.460 --> 00:04:25.200
has much more of the, like, agentic back end- Nice... that can just do some wild, uh, Comfy UI workflows and integrate with, uh, SMPTE 2110 and all sorts of other higher level production pipeline stuff.
28
00:04:25.260 --> 00:04:35.920
Yeah, it seems more hardware oriented and more ready for, like, tangible real world use in, in the video production world rather than Comfy being- That's it... like, a software running on the cloud kinda thing.
29
00:04:36.200 --> 00:04:47.080
I've known Ramiro for a few years now, and since the time I met him till now, he's always been on the fringe of the extreme innovation side. [laughs] So I'm excited to see what he's cooked up.
30
00:04:47.140 --> 00:04:53.860
It, uh, he's a brilliant mind. Ramiro's been pretty wild. I, I mean, I see his, like, the posts of, uh... He was, like, an early OpenClaw user. Yeah.
31
00:04:53.870 --> 00:05:01.160
And he's like, "I got OpenClaw spinning," and, like, [laughs] coding, like, five different agents at once, and so I, I'm very excited to see what he's got shipping.
32
00:05:01.420 --> 00:05:09.730
And then at the very end of that, we've got a panel that, uh, I'm moderating. You may be on it. I don't think I'll be on it. May not be. I'm bummed to miss this one, but yeah, yeah- I know. I was-...
33
00:05:09.840 --> 00:05:20.740
half of, half of Denoised representing. Ham said [laughs] that you won't be there, but yeah, we'll be there. Uh, or, uh, yeah, half of us will be there. Um, so yeah, this, uh, we'll post the link. It's a free event.
34
00:05:20.780 --> 00:05:30.960
You don't have to have a SIGGRAPH pass. It is Tuesday evening. There most likely will be food and drinks, but I can't promise that. It's not my event. [laughs] As long as there's something cold to drink.
35
00:05:31.000 --> 00:05:40.880
There should be something there. [laughs] Let's talk about Blender. Okay, so OpenAI released GPT 5.6. There's three different versions.
36
00:05:40.940 --> 00:05:47.940
Sol is, like, the beefiest one, and it's sort of the equivalent of Anthropic's Fable 5. It sped up Codex.
37
00:05:48.040 --> 00:05:58.539
One of the things I kept seeing people do online was, like, I just gave it images, and I'm like, "Hey, go build this thing in 3D. Go build this thing in Blender." Sure. And it did it, and I was like- Wow... "All right.
38
00:05:58.760 --> 00:05:59.740
Let's tr- let's try that."
39
00:06:00.080 --> 00:06:13.352
[laughs] So- Yeah, elaborate on it did it Yeah, [laughs] I mean, this is gonna be the du- the quickest tutorial, 'cause literally all I did was, uh, you download Codex, which is- So Codex is their competitor to Claude Code.
40
00:06:13.412 --> 00:06:19.092
It's like a desktop application Codex is a competitor to Claude Code. It was a separate app, but I'm, like, stopping myself because now I'm looking.
41
00:06:19.132 --> 00:06:29.052
I believe they actually got rid of it and they just rolled it all up into the ChatGPT app. So basically, it's the, a copy of the Claude app, where Claude app has regular Claude side and then it has Claude Code side.
42
00:06:29.772 --> 00:06:39.492
ChatGPT has ChatGPT side for normal chats, and then it has Codex, which is their coding agent. Um, and also it just can do more things on your computer.
43
00:06:40.032 --> 00:06:53.372
So literally all I did was I started a new project, and then I said, "Can you set up Blender MCP?" And then it [laughs] um, found the MCP- Anybody, anybody can type that in [laughs] It's, it's a third party tool.
44
00:06:53.432 --> 00:07:04.052
It found it, it installed it, it set it up. Yes, it did [laughs] Wow. And then for this test, though, I had some pictures from the Culver Theater from, uh, on the lot, and, uh, I gave it some of these photos.
45
00:07:04.072 --> 00:07:09.942
Now, these aren't the best photos 'cause they're one-sided. I didn't get a good s- angle of the corner- Mm-hmm... and the other side. Mm-hmm.
46
00:07:09.952 --> 00:07:21.012
But I gave it these photos and I said, "Can you build the Culver Theater and the surrounding area in Blender using the Blender MCP?" Stop there. And- I just wanna say how difficult that would be for a modeler.
47
00:07:21.332 --> 00:07:31.232
If you go to the picture- Yeah, explain this... like, it's, it's a lot of art deco, intricate architecture. It's not everything is not right angles, right? Um, like if you look at the sp- the- Yeah, it's got curves...
48
00:07:31.272 --> 00:07:40.332
four things holding up the sphere up on top, like that, that's a pretty difficult thing to model, at least for me. Yeah. Mm-hmm. Well, took your heads up with that part.
49
00:07:40.432 --> 00:07:48.622
[laughs] But, but I also didn't give it great images. I would be curious, I, I haven't tried if it could give it a reference video. Even if you couldn't, you could extract the frames- Yeah...
50
00:07:48.652 --> 00:07:54.272
if you had, like, a drone shot- Some screen grabs... and it gave you a bunch of frames. Yeah. Um, yeah, I mean, this is a very pr- preliminary test.
51
00:07:54.392 --> 00:08:03.122
I am curious how far, how much better it would get the more images you get, or if there's, like, a cap of, like, it just doesn't- I'm just curious about the first quality- It's not gonna get better...
52
00:08:03.122 --> 00:08:14.812
based on what you have from your cell phone, looks like So, right. So I gave it to this, I'm like, "Okay, go do the thing," and then here's the thing. Oh, wow. That's almost there. [laughs] It, yeah, it's pretty.
53
00:08:14.982 --> 00:08:25.072
[laughs] Yeah. So because I didn't have a great angle of the corner, it kind of... This is actually a second pass. I gave it a revision, and I said the corner should be more rounded and, uh, it's a corner door.
54
00:08:25.372 --> 00:08:34.512
The first pass was a bit flatter. But overall, it got the lamps, it got the overall shape of the building. It's okay. Yeah, like the theater sign is the biggest- It needs more refinement... egregious error.
55
00:08:34.612 --> 00:08:47.272
I mean, look, it, it's giving you rounded corners, some spheres. It's building it with, like, basic shapes. Um, look at the- Mm... look at the floor. Can you go back to your image? Let's see the floor by the door.
56
00:08:47.352 --> 00:08:58.832
Yeah, like that, that's such a terrible picture for the, the art deco- Yeah... you know, concrete stamping thing on the floor, but it, it did an interpretation of what it thinks it could do.
57
00:08:58.872 --> 00:09:12.012
And then, um, like how many windows are there? One, two, three, four maybe. Yeah, it didn't get all the door windows. Yeah. It got some solid thing, some things out. Um- But man, for a first pass- It, it, it- Jeez Yeah.
58
00:09:12.092 --> 00:09:16.972
This is good. And look, it even, it didn't really have a good s- Yeah... image on this side, but it sort of made up these art deco. Sure.
59
00:09:17.132 --> 00:09:27.852
Like there are movie posters on this side, and it's not really in the image, but it did a pretty good job of just sort of making something up- Yeah... that, like, made sense- Oh my God... for the design.
60
00:09:27.892 --> 00:09:37.232
Look, for a first pass, this was a r- I, I did, this was a refined, it, this is technically the second pass, and I said to, like, change a few things, and it did that.
61
00:09:37.492 --> 00:09:45.892
But look, overall, I mean- This is such a great starting point... it's pretty crazy. And look, you have all of your, uh, geometry separated out- And look, it definitely did get your, uh- In the, in the outliner...
62
00:09:46.042 --> 00:09:55.032
it did get your- Like if you- Yeah. This is all, like, ready to modify Yeah. I mean, yeah. [laughs] I'm, I'm thoroughly impressed, Joey. But look, it got your top thing pretty- Yeah... it got your top thing pretty well.
63
00:09:55.061 --> 00:10:02.972
Yeah. I mean, it's not what it actually looks like- No... but it- Not, not even close... I didn't give it a gr- but it got, it got the ball and it got some of the stuff here. Yeah. This is not a great...
64
00:10:03.032 --> 00:10:07.862
If I, uh, you know, if I had a drone and gave it some better angles, maybe it would get more into detail it would get. So that's, that's what I'm thinking, like, um,
65
00:10:09.212 --> 00:10:22.092
a 3D scan company and people that do digital twins of sets and stuff, they can, they have the equipment to then, instead of feeding it into Reality Capture and making a mesh, now they can feed it into Codex.
66
00:10:22.572 --> 00:10:42.112
I'm thinking too, like for me, it's like if a lot of the workflows I am messing with, like I don't need super accurate photo real 3D models, I just need like Graybox get me roughly there so that I have the geometry and we can do some camera moves, and then just use these as base inputs into like video to video workflows.
67
00:10:42.542 --> 00:10:53.922
Mm-hmm. So like, I'm like, if it gets me 70% of the way there of like what it should look like, and has roughly everything in place- Mm-hmm... we can clean all the other stuff up, uh, with like gen AI passes. 100%.
68
00:10:54.212 --> 00:11:02.392
I, I can see that. Also, I'm seeing this for traditional VFX where you have to send a team out to scan, um, put it all back together, right?
69
00:11:02.712 --> 00:11:13.122
Now that is just a bunch of phones really, and maybe some ladders to take [laughs] higher enough pictures. Oh yeah, or it's about drone. I mean, I have, uh, I have the DJI Pocket Drone- Yeah...
70
00:11:13.162 --> 00:11:21.862
which they, like, built it so it's technically under the weight. They have to go through- And then what, uh, how long does it look like it took you, like six minutes in Codex to go through all of this? It's insane, man.
71
00:11:21.862 --> 00:11:33.172
Yeah, it took, once it spun up the server that built this in, yeah, maybe 10 minutes. And then you can also light and modify that. Yeah. Oh yeah, and also I didn't ask for it, but it gave me a video- Like a render...
72
00:11:33.212 --> 00:11:45.782
render movement. Yeah. I don't- And then I said, "Oh, hey, can you relight it so it's like the lamps are illuminated and the building's illuminated, and it's like a moody night scene?" Oh. And then it did this. Yeah.
73
00:11:45.812 --> 00:11:50.912
Obviously, you could tweak it and refine it, but I just said, "Hey, turn it, like these into practical sources and lights and-" Yeah...
74
00:12:06.016 --> 00:12:19.916
B- you know, a year or two ago, I thought that the gen AI revolution would really take us from a 3D world back to a 2D world a little bit. Because why bother with 3D when you can just get to final pixel, right?
75
00:12:20.166 --> 00:12:31.596
And this was my incorrect assumption back then. Since then, I've noticed, and I learned the hard way, is that you still need 3D because gen AI just doesn't give you the control, right?
76
00:12:32.136 --> 00:12:47.896
If you want the camera move, if you want, you know, like that sphere to be held up by four things, like these very nitpicky things, then you gotta do the thing in 3D and then, um, gen AI will hopefully fill that photorealism gap in 2D, which is what we're seeing a lot now.
77
00:12:48.716 --> 00:13:00.756
I think in a year or two from now, my prediction is we're gonna be in a entirely different world that is so deeply in 3D, but without the complexity of traditional 3D. Like,
78
00:13:01.696 --> 00:13:06.296
if you think about- Yeah, like a simplified user interface... yeah, just like all the things you don't have to do.
79
00:13:06.376 --> 00:13:21.136
You don't have to model, you don't have to do shading, you don't have to do lighting, you don't have to do, uh, geometry optimization and like, uh, rigging and skinning and all the things that is associated with the 3D complexity bubble, right?
80
00:13:21.296 --> 00:13:29.116
And you're talking about people's, uh, job to... is to do very specific things, very specific lanes, and now you're gonna have
81
00:13:30.196 --> 00:13:49.216
just an operator, just a, a, you know, AI 3D operator that can do all of these complex 3D functions in Blender, in Unreal hopefully, in Maya, and still have, quote-unquote, "a traditional workflow," but with the simplicity of using ChatGPT or Claude or just like prompting and typing.
82
00:13:49.716 --> 00:13:52.236
Yeah, especially as these things get faster- Yeah... and more real-time.
83
00:13:52.296 --> 00:14:04.376
Like the thing that was also really impressive with this was, uh, it took control of the computer, but it felt much more responsive than when Claude takes control of my computer and it's still like, "Okay, look at the computer.
84
00:14:04.696 --> 00:14:12.576
Okay, let me assess the screen. Let me figure out what I'm looking at. Okay, let me take an action." And it's like still this very slow process where the Codex felt a lot smoother even...
85
00:14:12.616 --> 00:14:14.876
I mean, it was also talking to it via MCP under the hood.
86
00:14:15.476 --> 00:14:28.675
But as this keeps getting faster and faster, and voice models and all that stuff gets better, and then you're just literally in a gray box room or just in a whatever space you want, and you're just talking to the thing, and you're like, "Oh, hey, like, okay, move the camera here.
87
00:14:28.736 --> 00:14:32.365
Okay, let's add a tree over here. Uh, all right, let's, uh, you know, I need to make that wall bigger." Yeah.
88
00:14:32.376 --> 00:14:40.956
And you're literally directing, and it's doing all of the 3D set build stuff under the hood, and it's just responding to the direction- Hundred percent... that, uh, you know, it's giving it.
89
00:14:40.966 --> 00:14:52.636
A- and, and the other two things that are kinda converging and will sort of meet in a really happy place, I think is, uh, the fact that, um, GPUs are getting more powerful, right? That... Nvidia is working on this.
90
00:14:52.656 --> 00:15:03.956
They have, uh, Blackwell now. They're gonna have future GPUs. So on the 3D side, you're gonna be able to run trillions of polys, right? Right now we're in the billions. In the future, hopefully trillions.
91
00:15:04.076 --> 00:15:13.396
Like insane amount of polys, more than you can imagine. And then at the same time, the AI models and inference will get faster or more accurate.
92
00:15:13.656 --> 00:15:25.306
So your interactivity with AI, your responsiveness, your iteration, that's becoming more natural, and then the stuff you're making is becoming more complex and more lifelike. So the end result is- Yeah...
93
00:15:25.306 --> 00:15:33.336
like you being able to recreate reality or recreate a completely new world effortlessly, just effortlessly. And look, also I was just...
94
00:15:33.376 --> 00:15:37.676
This gave it Blender, but y- you know, Blender's the orchestrator or the base level too.
95
00:15:37.756 --> 00:16:01.046
Like imagine connecting this stuff to all of the other models and, and things that we've talked about, image to 3D, text to 3D, uh, Marvel World and World Labs, and using all of those tools at its capacity to bring in or create even more photorealistic things into your scene, and then orchestrating in your 3D scene, and then it's all there and consistent and, uh, controllable, but much more higher quality.
96
00:16:01.056 --> 00:16:07.016
Yeah. Uh, I, I, I mean, it's so hard to predict the future, but like right now the way we're gonna...
97
00:16:07.096 --> 00:16:23.256
we're sort of establishing the workflow ac- And I see this across the board, um, is, uh, uh, you know, stuff on LinkedIn, you know, stuff on Instagram and stuff, is you do a lot of fine creative work in 3D, and then AI is essentially a renderer.
98
00:16:23.596 --> 00:16:33.816
It like takes you to a style or a realism- Mm-hmm... level that you can't- Mm... really achieve easily with 3D. You're saying that in the future, like that will be the workflow.
99
00:16:33.896 --> 00:16:40.156
Like 3D will play the piloting part, and then the finishing part will be AI. Yeah. Yeah. Yeah, yeah. For sure.
100
00:16:40.796 --> 00:16:50.236
Uh, other thing too, I, I saw this and I just wanna like flag it, of just like what even just new formats or new things will be, like be created with these tools.
101
00:16:50.576 --> 00:17:02.096
So Alex, uh, Baraschkov built with Codex, uh, what he's calling, uh, AVAL or AVAL, a new open source format for interactive video on the web. And he has this demo of this sort of this
102
00:17:03.056 --> 00:17:11.276
interactive video with, uh, grass blowing. I know this clip's from a movie. I just don't remember what. Um, and but just,
103
00:17:12.316 --> 00:17:28.016
you know, just he built this whole brand new format for handling video, uh, and interactive video on the web with Codex, and so it just makes me think like what other d- new things, formats are just gonna be open and unlockable for people- Damn, imagine that, like just making your own USD or FBX file format.
104
00:17:28.756 --> 00:17:41.236
You know? Because- Mm-hmm... you can. Exactly. Like you have the ability to have complex software tasks automated. Yeah. Small file size, low CPU overhead, alpha transparency- Wow... web native runtime.
105
00:17:41.496 --> 00:17:49.206
It's gonna be crazy And like the, the future is so hard to imagine because it's literally not built yet. Like we, we can't- Right [laughs]...
106
00:17:49.226 --> 00:17:59.076
we can't draw a straight line 'cause the, the points on the l- beginning of the line aren't even there. I remember you poo-pooed when we talked about that thing a few months ago that was like, uh, what was it?
107
00:17:59.136 --> 00:18:07.832
It was like generative websites. It was like websites sort of built on demand- Yeah, I would, I would poo-poo on something like that... based on your needs[laughs]
108
00:18:07.962 --> 00:18:15.262
But I feel like that is, I feel like that's still gonna kinda come into play more and more where like, i- i- if you have a use- Yeah...
109
00:18:15.272 --> 00:18:27.752
you need software, you need something, these like tools or, or solutions that spin up based on need, um, because it's just so easy to spin something up from scratch, uh, that does exactly what you need to do. Yeah.
110
00:18:28.132 --> 00:18:40.892
Also, it could take us into like the Wild West of software end formats, and things that are, uh, not bu- you know, debuggable, just built by vibe code. It, it, it could be like a total mess, right?
111
00:18:40.952 --> 00:18:51.212
Like one pipeline will look nothing like another pipeline. So if a VFX artist worked on one show, and they go to another show, they're like, "What, what am I doing?" Yeah. [laughs] Yes.
112
00:18:51.272 --> 00:18:56.742
That, that's probably gonna become very messy. Uh, especially if like every, yeah, if every studio builds their own- Yeah... proprietary system.
113
00:18:56.852 --> 00:19:11.212
I think, yeah, maybe the systems interfaces are unique for whoever builds them, but I think open standards are gonna get even more important. Yes. Like, um, USD, Open Timeline, um, what other- Uh...
114
00:19:11.222 --> 00:19:11.792
open standards are there? Yeah.
115
00:19:12.102 --> 00:19:21.262
Uh, yeah, a, a lot of stuff that they're [laughs] talking about this Sunday is gonna just keep getting more and more vital for, so like you could build your own system, but like making sure your system can communicate with everyone else's systems.
116
00:19:21.272 --> 00:19:29.922
Wow, you really just convinced me to go to the Sunday open source thing. [laughs] I'm like, "Yeah, that sounds really relevant. I, I need to go. I need to..." Yeah. [laughs] Yeah. [laughs] Whatever.
117
00:19:29.952 --> 00:19:37.942
I mean, that's what I, that's what I'm thinking. I'm looking at the talks. They're like, yes, all of these things where like I'm messing around with 'cause we need ways to like... If I'm building an app- Yeah...
118
00:19:37.942 --> 00:19:43.072
and I'm building a timeline, I need to make sure that timeline can get sent to, uh, Resolve or Premiere, and- Yeah... that type of timeline. I, I think you're right.
119
00:19:43.172 --> 00:19:52.112
Open source is, open source and open formatting become more important than ever. But that's not gonna stop somebody from coming up with their own USD.
120
00:19:52.232 --> 00:20:01.092
I'm just thinking like just going back to USD, and how much of a revolution that it was, and how long it took. Like Pixar took 10 years to make that from scratch.
121
00:20:01.972 --> 00:20:12.192
And, uh, that was, I don't know, the early 2000s is when the work started. And just now we're seeing, um, all that trickle into all the nooks and crannies of media and entertainment. I mean,
122
00:20:13.332 --> 00:20:21.152
it's still, it's still like, um, not for the amateur because it, you know, you need software to be able to handle it correctly.
123
00:20:21.372 --> 00:20:34.372
But I'm just thinking if, um, there is such a thing as a standard for building your own standard. Uh, yeah. I think it depends what you wanna do and what you need to like be able to transfer- Yeah... in and out.
124
00:20:35.172 --> 00:20:47.192
Um, yeah, I mean, I don't know. Maybe I'm naive, but it's like, yes, the, those like USD and stuff took a long time to adapt because there are only a few big software players that would handle it- Yeah...
125
00:20:47.292 --> 00:20:52.892
and adapt to it. But- So software was expensive, and now software is super cheap, right? So that's the thing that flipped kind of. Yeah.
126
00:20:52.932 --> 00:21:01.062
And it's like, uh, uh, and it's like, oh, if you don't accommodate what I need, I can just build it myself. Yeah. Right.
127
00:21:01.062 --> 00:21:07.962
And that, that's the part where I'm like maybe this is a naive take, but it's only gonna get better and faster and easier- Yeah... to spin up your own software.
128
00:21:07.962 --> 00:21:17.632
And, and there's, there's also all types of decentralized resources that you can tap into. You know, what if, what if there's a format that just runs file stuff?
129
00:21:19.152 --> 00:21:27.842
What if there's a format that just runs NVIDIA's WebGPU stuff, right? And it's not really built for anything local. It can run on a Mac, on a PC, or anything. Yeah.
130
00:21:27.892 --> 00:21:40.272
That hasn't been really imagined as far as I know, right? Like nothing is really 100% optimized for 100% decentralization. No. I mean, unless you're building it yourself. Yeah.
131
00:21:40.312 --> 00:21:53.432
Maybe like GLB- But that's an open question... GLB formats, like stuff that, um... Yeah. Anyway, I don't know enough voodoo science behind those formats. But yeah, like it's an interesting area to watch for sure.
132
00:21:53.532 --> 00:22:08.481
So not only the workflow itself, but the delivery, and the formatting, and the exchange of information. Yeah, yeah. All right. You wanna talk about, [laughs] uh, o- older software now? Yeah. New, new stuff in- Yes...
133
00:22:08.532 --> 00:22:18.032
established software? [laughs] All right. Let's talk about Nuke. Nuke just announced a Smart Roto function along with some updates to Grip Tape. Okay.
134
00:22:18.092 --> 00:22:27.621
Smart Roto, new Nuke plugin that handles basically it's smart roto, smart roto-ing. [laughs] Yeah. I'm, I'm betting it's like AI- Just-... powered under the hood. I would have a- assume so too. Yeah.
135
00:22:27.741 --> 00:22:36.402
I mean, this is stuff that's already been in Resolve with IntelliMask- Yeah... IntelliTrack, and Premiere and After Effects- Yeah... has pretty good, uh, roto Foundry's smart.... plugin.
136
00:22:36.412 --> 00:22:47.492
They're so, uh, sensitive to the use of the word AI or any sh- any sort of artificial intelligence connotations. Although they just acquired Grip Tape, and they're going full I- AI.
137
00:22:47.772 --> 00:22:56.282
But in a way that is, makes sense to like high-end professionals, and that works in professional workflows. Dude, nobody wants to do frame by frame roto. Like I think it's been a, a...
138
00:22:56.282 --> 00:23:04.012
It's a delicate balance, but- Come on, man. Those roto artists- Yeah... don't like to do roto. No, no. I'm talking more about Grip Tape and stuff. Like, you know, the, you were saying they go fully, fully AI.
139
00:23:04.092 --> 00:23:10.872
But like fully AI in a way that still integrates w- with high-powered professional pipelines. Yeah. They're, they're doing it the right way, for sure.
140
00:23:10.952 --> 00:23:20.772
Grip Tape, to me at least, the Grip Tape Enterprise feels like comfy for VFX, for the lack of better terms. Mm. Um, so yeah, I saw this announcement.
141
00:23:20.832 --> 00:23:32.202
It's like, um, OCIO support, which, you know, something you would have to probably build in Comfy. I'm not sure there's a node for it or anything. Um, local GPU inference and stuff- What...
142
00:23:32.202 --> 00:23:42.332
which I thought was super interesting, uh, because studios are super sensitive about- Yeah... using the cloud for unreleased IP- Yeah, yeah... and things like that. Yeah. They're definitely more in the- Yeah...
143
00:23:42.392 --> 00:23:48.212
keep it local. It's all good progress. Um, it- it's really converging with traditional AI nicely.
144
00:23:48.312 --> 00:23:59.752
I think, uh, the Foundry and Grip Tape understand, obviously, given their history with Nuke, they understand that world really, really well. That is their core, uh, audience, their core demographic.
145
00:23:59.892 --> 00:24:08.764
So I expect nothing less from the Foundry, and they, they have, uh, sort of fine-tuned themselves into this world. No pun intended. Yeah.
146
00:24:09.164 --> 00:24:21.944
I think smart roto only gets more important with, uh, AI-generated stuff when you gotta isolate elements- Yeah, and I mentioned on the pod before, like, uh, because of the inherent noise, de-noising of AI generations,
147
00:24:22.844 --> 00:24:34.324
roto stuff and matte stuff doesn't work well with it as far as I- I've heard. I mean, uh, uh, from some of my testing, I- I can't 100% confirm that, but- Oh, right. You mentioned that last time. Yeah.
148
00:24:34.424 --> 00:24:44.113
No, no, I don't think that's true 'cause we've been, we've been masking the crap out of generated stuff to clean it up, and it's- Okay. It's fine. So maybe it's the- Behave- it behaves the same as- as roto video...
149
00:24:44.164 --> 00:24:56.984
I don't think it's roto, but maybe it's tracking, camera tracking. 'Cause, um, camera tracking is essentially trying to figure out a 2D- Mm... space into a 3D camera move. Okay. We haven't done as much- Yeah... of that.
150
00:24:57.084 --> 00:25:08.864
So yeah, maybe. I- I'll... Okay. I'll tr- I'll try to try some shots- Yes... to see if we can track it. All right. [laughs] Rage bait time. We're just doing this for the, for, for the rage bait? Okay. Yeah.
151
00:25:08.884 --> 00:25:18.024
We're doing this to clip this for... AI-generated Odyssey movie costing a few thousand dollars [laughs] unveiled ahead of Nolan's $250 million feature.
152
00:25:18.964 --> 00:25:26.954
Uh, I saw s- one of these things fr- frame it as like going, going head-to-head, and it's like [laughs] So that's what I saw. Who's going- I was like, "Wait, uh-" Who's going head-to-head? "...
153
00:25:26.984 --> 00:25:38.494
Nolan makes like three-hour movies, right?" It's like, wait, somebody made a three-hour Odyssey with AI? I have to see it. And if you look at the length of that video clip, one minute 52 seconds.
154
00:25:39.044 --> 00:25:49.004
Well, this is a trailer for the epic that is coming. Yeah, I think this is it. [laughs] I don't think it's coming. [laughs] You think they, they- I, I think they ran through their credits with this. Yeah.
155
00:25:49.164 --> 00:25:58.714
I mean, you know, I'm looking at the trailer right now. I don't have any sound on it. Yeah. But it's got shots and- I mean, it has- You know- It has some beautiful shots- AI's gonna take your shots... here and there.
156
00:25:58.724 --> 00:26:04.084
Take your shots. Uh, obviously well color graded, um, synthetic humans, I'm guessing.
157
00:26:04.664 --> 00:26:20.164
But like, yeah, some of the shots completely break, uh, especially, yeah, anything with perspective or distance, AI still can't figure out real accurate lensing and glass, and your eyes can easily tell that that is artificial.
158
00:26:21.024 --> 00:26:29.504
Um, so for example, in... Like, you can pause on any frame- Uh-... and I- I'll point it out. That feel- this feels like the... That feels like the least of my issues with this.
159
00:26:29.544 --> 00:26:42.404
There's just more like the camera geography and placement's just off, the performan- pro- performances are, you know, the talking and everything still feels just uncanny valley. Th- That has not been...
160
00:26:43.224 --> 00:26:52.064
Like, that has not been broken from a fully synthetic- Yeah. But-... point of view. Ooh, cyclops. Is it? With two eyes? No. Oh, okay. I think he's coming out of Hades... one of the world's creatures. Yeah.
161
00:26:52.144 --> 00:27:03.453
Ooh, that, that had a nice fisheye- Happy to... or anamorphic-y lens to it. Again- I, I mean, I feel like I've seen five different styles- Yeah... in this one trailer. You know what they should've done?
162
00:27:03.484 --> 00:27:15.224
They, 'cause s- I don't... Well, you're not on X as much, but like when the first costume, um, like Matt Damon appeared, uh- Yeah... images from the film, the history dweebs- That's not period accurate...
163
00:27:15.254 --> 00:27:23.203
were like, "That's not-" Yeah... you know, Greek- Yeah... period accurate. And then the period accurate ones was like these ridiculous like garbage- Yes... chute looking things.
164
00:27:23.304 --> 00:27:31.124
[laughs] So someone, if they wanted to go all in with the Odyssey was to make it a period- Yeah, I think it was like the bronze era-... accurate, uh, version-... or something... of the Odyssey...
165
00:27:31.284 --> 00:27:39.624
and the armor was like way more modern. Yeah. Something like that. Yeah. Something like... I mean, the bottom line is the armor that is in the movie- Yeah... looks cool.
166
00:27:40.004 --> 00:27:49.764
Uh, the armor that was accurate [laughs] did not look cool- Yes... compared to our standards today. So- Yeah... uh, you know, hope that helps. When they go to Hades and then you're like, "What's the accuracy of Hades?"
167
00:27:49.794 --> 00:27:59.644
And it's like, mm. Yeah, I think The Hollywood Reporter is really just struggling for clicks here with, with this headline. Uh, it's, I'm, I'm always sh- more shocked who the hell is their PR rep that gets...
168
00:27:59.744 --> 00:28:07.904
It do- it wasn't just Hollywood Reporter. Yeah. It's like the same Tilly Norwood type, uh, headline, you know? I was gonna say, that was, that came out a few weeks ago, and I've got a strict ban on. Yeah.
169
00:28:08.024 --> 00:28:16.404
We're not talking about that stupid shit. But yeah, that story came out- She got cast in a movie... too that was like, uh, oh y- Yeah. She's gonna be in her first movie. I was like, what the hell is this shit? Yeah.
170
00:28:16.414 --> 00:28:26.534
And that was on everywhere. That even made it to the point that like my completely non-AI wife was like, "There's some like AI actress that's gonna be in a movie or something." She like- I don't know...
171
00:28:26.544 --> 00:28:35.234
aren't you gonna talk about it on the podcast? [laughs] Kudos to whoever the hell is behind that for getting this stupid, the PR press for this thing. That's probably- Yes...
172
00:28:35.244 --> 00:28:45.084
that's been the most amazing feat- Yeah, absolutely... to me out of all this. So I mean, uh, I, I will give credit to the guy, what's his name, Ash, uh, the guy that made this. Ash, uh- Like, it- Ash Kucha...
173
00:28:45.164 --> 00:29:02.424
even though it doesn't, it probably won't work as a cohesive storytelling feature film, the shots are cool by themselves. So hats off to just pure generation. Yeah. Yeah, but we know like, we know shot, we...
174
00:29:02.464 --> 00:29:11.624
The AI's good at making cool shots. Yeah. Like, that's been proven. It's like making a cohesive thing where you're like in the story, it's got consistent characters, consistent shots.
175
00:29:11.804 --> 00:29:18.024
Now we're talking about this, let me bring up one thing that I did see that's- Well, while you're bringing up a quick rant. Uh, yes. We talked about this time and time again.
176
00:29:18.384 --> 00:29:28.294
I'll cover it one more time 'cause we're on this topic. The true unlock with AI is to combine it with physical performance and on location shooting.
177
00:29:29.184 --> 00:29:36.024
If you can just solve for the expensive bits of production, let's say you half your cost, that is the win.
178
00:29:36.664 --> 00:29:45.724
Having said that, there are people trying to figure out fully synthetic everything, and I think they're really limiting themselves by doing that. It's like I'm not gonna put a camera on anything.
179
00:29:46.484 --> 00:29:58.844
I'm not gonna put any human performance in there. Why? Because AI can do it all. I'll just do the whole thing synthetic. I think you've set yourself back like so far back with quality if you do that. Yeah.
180
00:29:58.984 --> 00:30:08.504
I think, yeah, what you're saying the unlock is human performances. I would argue I don't think you need, you know, locations. Um, you know, I think you can get away with a lot if-
181
00:30:09.484 --> 00:30:14.404
It helps, you know, immerse you a little bit- Yeah, you don't- But, uh, i- if you have the wall, the wall looks cool... I, I don't think you need full locations.
182
00:30:14.504 --> 00:30:29.204
I do think you need a, a film, like a cinema camera with real glass and real film back, because the perspective, the distortion, the sort of geometry of the real world, AI just can't get right.
183
00:30:29.264 --> 00:30:41.004
So then if you're just bringing that in as a reference plate, and then have AI fill everything up, I- that'll go so far to make it look realistic. You definitely need props and objects. Yeah.
184
00:30:41.024 --> 00:30:49.964
For anything they touch and interact with, you need that. You need people- Yeah... stuff they touch. But you can do matte painting in the back, and layered- You- Par- ha- have parallax and all that stuff... worlds. Yeah.
185
00:30:49.974 --> 00:30:54.564
Yeah. It's, it's like LED volume in a, in a lot of cinema. Yeah. But I think that is the best balance.
186
00:30:54.664 --> 00:31:03.624
Uh, look, I mean, sure, if you're like st- um, super low budget, and you wanna try to make something that is just completely generated, like, at le- at least the option's there.
187
00:31:03.684 --> 00:31:12.954
I still just, I just don't think the quality is there- Yeah, um, so like-... to make something that feels completely immersive. Yeah. That said- Okay [laughs]... this was the teaser thing.
188
00:31:13.623 --> 00:31:29.444
Uh, ByteDance released a trailer teaser video that was made using CDance 2.5, and this is all from... This is like a, um- Football, you mean?... commercial. I'm in World Cup mode, and, uh, it's all football now.
189
00:31:29.474 --> 00:31:39.064
And then when World Cup ends, it goes back to soccer. Man, that's pretty good. [laughs] This is 2.5? Uh- Yeah, I've been waiting for some 2.5 outputs... this is 2.5. The motion.
190
00:31:39.234 --> 00:31:47.334
It's starring a kid who's kinda like- Yeah... kicking the ball through London. Uh, movie looks good, kid looks great. I mean, the ball contact with the feet- Everything about this feels-...
191
00:31:47.334 --> 00:31:57.594
and yeah, even like that shot there, that should be so hard for AI to do, and it does it well. Yeah. This, um, you know. [laughs] So we might be eating our words. This is- We might be eating our words... pretty good.
192
00:31:57.594 --> 00:32:08.664
And that is okay. This is the whole point of the podcast. You know, follow us for the journey. [laughs] I mean, obviously, uh, the caveat with everything is like, "This is the worst it'll be today." Yeah.
193
00:32:09.204 --> 00:32:18.964
Still no word on when 2.5 is coming out. I keep seeing updates that it's getting de- Yeah... that keeps getting pushed back. But, um- Some, uh- It's coming. I think I did misspoke speak last time.
194
00:32:19.003 --> 00:32:30.764
I think I said 50 references, but I think it's 30. Oh, gosh. Kill me now. Only 30 references. [laughs] Um, are you kidding me? Where am I even gonna find 30 references? I have like three at most, in most generations.
195
00:32:30.784 --> 00:32:41.594
[laughs] Here's my photo. Here's my entire photo library. Just go, go make a movie. [laughs] Oh, you know what? The Culver City Theater could use 30 references. Yeah, that could use more. Yeah, yeah.
196
00:32:41.604 --> 00:32:53.444
That could use- Like all the close-ups and things like that. Yeah. For sure. But you have to see if they'll place all the references in the right s- location spatially. Yeah. You have to give it a wide shot of stuff.
197
00:32:53.904 --> 00:32:58.424
I, I am curious if you, how much instruction you can give it with the references that it'll retain and understand.
198
00:32:58.484 --> 00:33:06.594
If you, like, give it a wide shot, and then close-ups of things, will it understand like, "Okay, this is how everything relates," and then, but like, "The action should be here, and things should happen here"? Yeah.
199
00:33:06.664 --> 00:33:16.304
Um, yeah, it'll be the new Wild West of, like, how do you coordinate and direct this stuff, um, when you have so many reference images, and like what can it understand and stick to?
200
00:33:16.944 --> 00:33:27.264
Okay, so when 2.5 comes out, you and I both have to generate some reference images. [laughs] Okay. All right? We might go broke. We might go broke from that, 'cause, 'cause it might come out- From that generation...
201
00:33:27.284 --> 00:33:34.684
and we also, we don't know how much they're gonna charge, and if they're gonna charge you more based on if you have more reference images. [laughs] Oh, yeah, that, yeah. Okay.
202
00:33:34.944 --> 00:33:45.784
A 4K, uh, 4K- We should start a GoFundMe for that generate. C... [laughs] Right. The 4K CDance output, like one shot is roughly like five bucks. Oh my God, really?
203
00:33:46.304 --> 00:33:57.724
If you max it out to 4K, and like even just, I think, five seconds, it's like roughly like $4 or $5. Yeah. It's steep. It gets pricey. [laughs] Yeah, you can't do iterations that well for $5. Yeah. [laughs] Wow.
204
00:33:57.784 --> 00:34:03.754
Cha-ching. Those are LA prices, man. This is- [laughs] Right. Is it cheaper to run it in Tech Texas?
205
00:34:03.754 --> 00:34:12.534
If they keep those prices up, it'll be like the, uh, other software development companies when they like, were like, "Oh, we're gonna rehire the people because the tokens are too expensive," than- That's happened...
206
00:34:12.554 --> 00:34:16.554
what it costs for a person. I know, it's happening. That's happened. It's gonna be the same with film, where it's like, oh- Yeah...
207
00:34:16.584 --> 00:34:23.614
it's actually cheaper just to like shoot this for real than to, uh, blast CDance with it. [laughs] It, uh, yeah, the costs have got to come down.
208
00:34:23.864 --> 00:34:38.264
Um, I mean, s- I wonder how much it, of it is the actual, like data center GPU cost, and how much of it is just greed? [laughs] Uh, yeah, I don't know. I mean, I think a mix of both.
209
00:34:38.334 --> 00:34:49.244
'Cause like also, depending where you generate, there's, um, for CDance especially, the prices do vary a bit. Quite a bit. Yeah. Okay. So we'll do, we'll do one generate... I can afford like, I don't know, $20.
210
00:34:49.304 --> 00:34:58.364
I'll do four. [laughs] Cl- Yeah. We'll do like four budget. Okay? And that's all, Joey, all right? You just get four takes. Yeah. When 2.5 comes out, we will, we will do that. Okay, uh, last thing, quick thing.
211
00:34:58.404 --> 00:35:10.574
In the open source world, um, Thinky Machines, which was- Mira Murati?... Mira Mu- uh, what's it, Mira Murati? Mira Murati, who left, uh, one of the OpenAI founders, left, started Thinky Machines.
212
00:35:11.374 --> 00:35:21.204
They've been working for a while. They raised a bunch of money. They now have released their first official model called Inking- Yay... and they released it as open source Yay for open source. Open weights. Yeah.
213
00:35:21.284 --> 00:35:33.364
It, it seems like a- Yes... pretty solid model based on this article. Like, it, it does, uh, significantly step up the competition with ChatGPT and Claude. Yeah, I mean, it is definitely not at that level.
214
00:35:33.384 --> 00:35:40.774
But in the world of open source, it's on par with Kimi 2.6. Uh, I mean, definitely this is like a huge leap for- Absolutely, yeah...
215
00:35:40.824 --> 00:35:47.234
an open source American model, 'cause like pretty much all of the open source- Yeah, like DeepSeek and stuff... like decent models have been- Yeah...
216
00:35:47.344 --> 00:36:00.044
uh, from Chinese, yeah, have been from Chinese, Chinese model, uh, uh, AI companies. Uh, so yeah, I mean, this is a big leap for putting out this big model, open sourcing it, um, multimodal.
217
00:36:00.344 --> 00:36:07.824
I'd be curious, once someone, or I think it might be, I think they have a spot on their website called Tanker- Yeah... that you can, like, mess around with it.
218
00:36:07.864 --> 00:36:18.576
And then obviously you could run this open source, or I'm sure a bunch of providers will add, uh, the ability to Work with this model, uh, via third-party tools. Yeah.
219
00:36:18.605 --> 00:36:29.766
Uh, and so going back to Mira Murati, she was the CTO of OpenAI. She left, and then Ilya Sutskever was, I think, the chief research officer at OpenAI. He left. Mm-hmm.
220
00:36:29.836 --> 00:36:41.976
And then, uh, like, those are the two people I'm watching closely. And then the third person I'm watching closely is Yann LeCun, who left Meta and now has a billion dollar in funding for his own company.
221
00:36:42.056 --> 00:36:53.256
He thought LLMs were, like, not the way, right? He still thinks that. I think he said something like, "LLMs have two years left," because, uh, y- I think he's right. Scaling is not working.
222
00:36:53.296 --> 00:37:02.956
It's giving you diminishing returns. Um, and, you know, no matter how many more billions of parameters you train on, intelligence doesn't go up by that much. And what's the alternative?
223
00:37:03.156 --> 00:37:10.486
Just a, a bi- a bigger, like, a world model? It's the JEPA architecture, according to him. We could spend another episode on the JEPA architecture,
224
00:37:11.416 --> 00:37:26.296
but, uh, essentially, you move away from a text-based approach into a more abstract approach where text, images, video, any kind of just become seamless with one another, um, rather than us discreetly building a text model, image model, video model.
225
00:37:27.036 --> 00:37:39.396
Mm-hmm. Mashing those three things together to get reasoning, um, this, uh, is inherently more fluid in nature. Okay. All right. Uh, I'm very curious about that. Me too. [laughs] Um, okay.
226
00:37:39.476 --> 00:37:43.216
So this is out, and then also, I know I just compared it to Kimi, but Kimi
227
00:37:44.516 --> 00:37:59.716
is also teased with a pretty sick launch video that, um, Kimi 3 is about to come out, which is expected to be China's largest AI model with two to three trillion parameters. Jeez. Open weight, um- [laughs]...
228
00:37:59.796 --> 00:38:11.786
on par with, like, Claude Opus- Yeah... 4.8. So- Oh my God. Did you say trillion? Yeah. So, um- Jeez... yeah, that's coming- [laughs]... soon. How big, how big is the file gonna be? I have no idea.
229
00:38:11.836 --> 00:38:21.486
Like, the, the, the weights. Yeah, I got no idea. Like one terabyte? [laughs] I have no idea. [laughs] So, uh, you know, that's, that's coming. [laughs] That's awesome.
230
00:38:22.226 --> 00:38:34.436
Uh, I mean, like, it's not awesome in the sense that the biggest, baddest model ever is coming out of China and not here. Biggest, baddest- But it's awesome... open- Open source... source model. It's crazy.
231
00:38:34.576 --> 00:38:46.636
[laughs] It's crazy. Yeah. I wonder why Thinking Machines launched their thing open source. Um, do you think it's strategically to just get people to use it and tinker with it- I think so... and get their hands on it?
232
00:38:46.796 --> 00:38:54.596
I think, it's like, what do you do when you're starting a new thing and you're already, like, you're not gonna beat ChatGPT or, or Anthropic. So, like- Right... what do you do to make a splash?
233
00:38:54.656 --> 00:39:04.406
And if you've been trained in a pretty good model and you wanna get people on board with it, open source it. You, you start to get users on it- Mm-hmm... build momentum on the flywheel, if you will. Yeah.
234
00:39:04.416 --> 00:39:16.376
I mean, it's what, uh, stable diffusion did so well with SD 1.5, SDXL, right? They gave it away for free very early on, and now it became synonymous with image models, like, a few years later. Yeah.
235
00:39:17.016 --> 00:39:29.196
Uh, I mean, I re- they haven't released it yet, but one of the first things that Thinking Machines teased, like, a few months ago was that conversation chatbot that was much more...
236
00:39:29.236 --> 00:39:47.996
In the demo they did, it was, like, a much more responsive, reactive, like, fluid conversation that they were having, like, where if you interrupted the model, it kind of, like, picked up with the new chat, and it was, uh, uh, something that it seemed like they were focusing on was, like, a conversational aspect, and that part hasn't been released yet of whatever they're working on there.
237
00:39:48.076 --> 00:39:55.856
I did see a really interesting feature that I didn't see with other models, is it can fine-tune itself. Oh, interesting. Based, based on- So, like, it learned. Yeah.
238
00:39:55.996 --> 00:40:11.036
So, like, if you need a m- let's say you wa- you wanna run an agent that runs a dentist office or something. Like, it has to- Mm-hmm... fine-tune on den- dentist stuff, like, you know, orthodontist and things like that.
239
00:40:11.136 --> 00:40:15.546
It could probably fine-tune itself if you just ask it to and give it enough data.
240
00:40:15.916 --> 00:40:23.276
That's cool, 'cause right now it's sort of like a hack where you have the model, and you have a bunch of markdown files, and then the markdown files- Yeah... update themselves.
241
00:40:23.296 --> 00:40:32.275
Right, and then you gotta build a giant context window, and I think the context window here is pretty insane too. I think one million characters or one million tokens, which is a massive context window.
242
00:40:33.816 --> 00:40:42.456
Uh, I think I saw on Tinker at least it was, like, capped at 256, but maybe- Yeah... if you go full blast, you c- it's a million. Yeah.
243
00:40:42.736 --> 00:40:54.476
So, like, it's definitely a modern model by today's standards, which also happens to be open source, and, uh, yeah, I'm, I'm hopeful that this is a, a good alternative to ChatGPT and Claude.
244
00:40:54.576 --> 00:41:02.676
Um, I think competition in this space is what we need, you know, just to get things better and more efficient. I'm just really worried about using all those GPUs.
245
00:41:02.726 --> 00:41:06.716
[laughs] Like, can we get a model- Yeah, and stuff that could run locally too. Yeah. I mean, I think that
246
00:41:07.856 --> 00:41:13.736
as workflows get more established and as it sort of pushes into, like, higher-end film production workflows or other things, like
247
00:41:15.196 --> 00:41:31.656
exploring, uh, stuff that you could run locally is gonna get more interesting- Absolutely... and more needed. Mm-hmm. Especially when you start comparing the costs of API tokens versus spinning it up yourself. Yep.
248
00:41:31.816 --> 00:41:40.436
Absolutely. All right. Good place to wrap it up. This went deeper than I thought, but this was a fun one. Uh, links for everything we talked about at genoispodcast.com.
249
00:41:40.796 --> 00:41:48.786
If you wanna catch Joey and I at Siggraph, we'll be there Sunday at the open source thing and Monday morning as well. And- Yeah... I think, Joey, you're gonna attend- I'll be around a few days longer.
250
00:41:48.876 --> 00:41:57.056
Yeah, you're gonna attend the- Yeah, definitely be there Monday, Tuesday. I don't... The thing, that was a, think it goes until, like, Thursday. That's why I was also curious, like, how... That's, like, longer than NAB.
251
00:41:58.316 --> 00:42:08.516
Yeah. It's a lot. It's a long thing. A, a lot of it- Quite a long thing... a lot of it is, is, like, the big key- keynotes and stuff which go on. Yeah. All right. But Lenny's smooth. Lenny's smooth.
252
00:42:08.636 --> 00:42:12.556
[laughs] Thanks everyone for watching. We will catch you in the next episode.
1
00:00:00.100 --> 00:00:10.000
Can you set up Blender MCP? I gave it these photos, and I said, "Can you build the Culver Theater and the surrounding area in Blender using the Blender MCP?" So I'm like, "Okay, go do the thing."
2
00:00:10.160 --> 00:00:25.860
And then here's the thing. Oh, wow. Welcome back to Denoised. Addy, how you doing? We're not, we're not in person anymore this time. I know people enjoyed that we were in person, but sorry, we're back, we're back remote.
3
00:00:26.740 --> 00:00:37.060
It's too hot to drive over to that side of town. [laughs] Your tire- your tires will melt. [laughs] All right, so we got SIGGRAPH coming up next week. Yep. I'm excited about this. This is my first SIGGRAPH.
4
00:00:37.100 --> 00:00:45.760
It's very convenient that it's here in LA this year. You gonna hit it up? Yeah, I'll be there Monday morning. Uh, if any of our viewers are there, love to say hi. Um- Yeah, let us know... just look for me. Yeah.
5
00:00:46.120 --> 00:00:56.440
Let me see. There's the Academy Open Source Day on Sunday, so I'll be a- attending that. And then, yeah, most likely Monday, Tuesday. Okay. For, like- I might join you for the Sunday one, actually.
6
00:00:56.880 --> 00:01:00.880
Yeah, come by the Sunday one. Have you, what, have you been to SIGGRAPH before? Oh, so many of them. Yeah.
7
00:01:01.000 --> 00:01:15.520
I mean, it's such a, it's such a mainstay conference for those in, especially on the creative technology side, the bleeding edge side of media and entertainment, because a lot of the newer research papers get announced there from- Mm-hmm...
8
00:01:15.540 --> 00:01:27.740
universities, uh, research labs, and then the new tools or the upcoming tools from the companies get announced there. Just, it, it's like NAB for the uber nerds, if you will. [laughs] Yeah.
9
00:01:28.260 --> 00:01:31.730
So, like, NAB, NAB has, like, the big releases from, like, the big companies.
10
00:01:31.780 --> 00:01:39.300
Like, but then that, like, one thing they're c- really not sure about, they're really experimenting and betting big on, that's gonna be at SIGGRAPH.
11
00:01:39.560 --> 00:01:53.860
Is it more of a talk kinda conference, or a expo floor conference, or, like, both? Yeah, I think the bulk of the floor space is actually the research, uh, entities presenting their findings.
12
00:01:53.980 --> 00:02:00.820
So, like, you'll have university- Okay... graduate students, and you'll see stuff that you're like, "Oh my God, this is like five years out, 10 years out."
13
00:02:01.040 --> 00:02:11.039
Like, for example, the early days of VR, that was the first place I saw haptic touch stuff- Mm. Okay... was at, like, SIGGRAPH 2016 or 2015. Yeah.
14
00:02:11.840 --> 00:02:22.620
So, like, uh, some stuff will fizzle out and never really make it into, uh, a business, but then some stuff will stick. Um, it, it's, it's a glimpse into the future of media and entertainment, in my opinion. Yeah.
15
00:02:22.700 --> 00:02:23.700
I'm excited to check it out.
16
00:02:23.820 --> 00:02:36.609
Uh, there was, like, an LA SIGGRAPH event a few weeks ago, and, um, uh, Netflix did a teaser of some of the stuff they're gonna present and, uh, some very cool papers and some very cool research that they're doing that- Yeah...
17
00:02:36.620 --> 00:02:42.780
I'm excited to see get out there. I, I, I can imagine Netflix research, uh, Netflix Research announcing new stuff there.
18
00:02:43.260 --> 00:02:53.609
It's also very strategic on their part 'cause, uh, you have a high density of really amazing AI researchers. Mm-hmm. So if you wanna hire them, if you wanna woo them, you know, that's the place to do it.
19
00:02:53.840 --> 00:03:00.400
You, you have recruiters out there at the same time. Um, if you're looking for work, I think that that's a good place to connect.
20
00:03:00.620 --> 00:03:09.040
Um, so all in all, the fact that it's in LA, and it's a primarily media and entertainment-based thing, uh, those two things will play well together. Yeah. Yeah.
21
00:03:09.140 --> 00:03:18.790
Uh, I am excited for it, and then outside of the official conference, there's a bunch of satellite events, one of which, if you are in LA and wanna come by, you don't need a SIGGRAPH pass for this, and it's free.
22
00:03:18.820 --> 00:03:28.760
We are at the... It's not the sexiest name, but the AI Workflow Summit. [laughs] Yeah. Under the hood, there's gonna be a lot of cool stuff here.
23
00:03:28.980 --> 00:03:38.140
Um, it's at, not officially Kinoflo, but it's at their, like, satellite e- uh- Yeah. They're, they have giant warehouse space in Burbank... R&D warehouse. Yeah. It's a cool spot. Yeah.
24
00:03:39.040 --> 00:03:49.899
Um, but there's gonna be some cool stuff. Some presentations on state-of-Gaussian splats from NVIDIA, uh, real-time biometric workflows, and one that I'm very excited and curious about is, uh, Ramiro's gonna be...
25
00:03:49.960 --> 00:03:56.860
He's been doing, like, a lot of work with Comfy UI- Mm-hmm... and agentic Comfy UI workflows, and so he's rolling out
26
00:03:57.840 --> 00:04:12.530
comfycode.ai, which from what he kinda briefly told me, but we'll get a lot more info there, is kind of like a MCP sidecar for Comfy that sounds crazier than the official Comfy MCP. This one could work local. This one
27
00:04:13.460 --> 00:04:25.200
has much more of the, like, agentic back end- Nice... that can just do some wild, uh, Comfy UI workflows and integrate with, uh, SMPTE 2110 and all sorts of other higher level production pipeline stuff.
28
00:04:25.260 --> 00:04:35.920
Yeah, it seems more hardware oriented and more ready for, like, tangible real world use in, in the video production world rather than Comfy being- That's it... like, a software running on the cloud kinda thing.
29
00:04:36.200 --> 00:04:47.080
I've known Ramiro for a few years now, and since the time I met him till now, he's always been on the fringe of the extreme innovation side. [laughs] So I'm excited to see what he's cooked up.
30
00:04:47.140 --> 00:04:53.860
It, uh, he's a brilliant mind. Ramiro's been pretty wild. I, I mean, I see his, like, the posts of, uh... He was, like, an early OpenClaw user. Yeah.
31
00:04:53.870 --> 00:05:01.160
And he's like, "I got OpenClaw spinning," and, like, [laughs] coding, like, five different agents at once, and so I, I'm very excited to see what he's got shipping.
32
00:05:01.420 --> 00:05:09.730
And then at the very end of that, we've got a panel that, uh, I'm moderating. You may be on it. I don't think I'll be on it. May not be. I'm bummed to miss this one, but yeah, yeah- I know. I was-...
33
00:05:09.840 --> 00:05:20.740
half of, half of Denoised representing. Ham said [laughs] that you won't be there, but yeah, we'll be there. Uh, or, uh, yeah, half of us will be there. Um, so yeah, this, uh, we'll post the link. It's a free event.
34
00:05:20.780 --> 00:05:30.960
You don't have to have a SIGGRAPH pass. It is Tuesday evening. There most likely will be food and drinks, but I can't promise that. It's not my event. [laughs] As long as there's something cold to drink.
35
00:05:31.000 --> 00:05:40.880
There should be something there. [laughs] Let's talk about Blender. Okay, so OpenAI released GPT 5.6. There's three different versions.
36
00:05:40.940 --> 00:05:47.940
Sol is, like, the beefiest one, and it's sort of the equivalent of Anthropic's Fable 5. It sped up Codex.
37
00:05:48.040 --> 00:05:58.539
One of the things I kept seeing people do online was, like, I just gave it images, and I'm like, "Hey, go build this thing in 3D. Go build this thing in Blender." Sure. And it did it, and I was like- Wow... "All right.
38
00:05:58.760 --> 00:05:59.740
Let's tr- let's try that."
39
00:06:00.080 --> 00:06:13.352
[laughs] So- Yeah, elaborate on it did it Yeah, [laughs] I mean, this is gonna be the du- the quickest tutorial, 'cause literally all I did was, uh, you download Codex, which is- So Codex is their competitor to Claude Code.
40
00:06:13.412 --> 00:06:19.092
It's like a desktop application Codex is a competitor to Claude Code. It was a separate app, but I'm, like, stopping myself because now I'm looking.
41
00:06:19.132 --> 00:06:29.052
I believe they actually got rid of it and they just rolled it all up into the ChatGPT app. So basically, it's the, a copy of the Claude app, where Claude app has regular Claude side and then it has Claude Code side.
42
00:06:29.772 --> 00:06:39.492
ChatGPT has ChatGPT side for normal chats, and then it has Codex, which is their coding agent. Um, and also it just can do more things on your computer.
43
00:06:40.032 --> 00:06:53.372
So literally all I did was I started a new project, and then I said, "Can you set up Blender MCP?" And then it [laughs] um, found the MCP- Anybody, anybody can type that in [laughs] It's, it's a third party tool.
44
00:06:53.432 --> 00:07:04.052
It found it, it installed it, it set it up. Yes, it did [laughs] Wow. And then for this test, though, I had some pictures from the Culver Theater from, uh, on the lot, and, uh, I gave it some of these photos.
45
00:07:04.072 --> 00:07:09.942
Now, these aren't the best photos 'cause they're one-sided. I didn't get a good s- angle of the corner- Mm-hmm... and the other side. Mm-hmm.
46
00:07:09.952 --> 00:07:21.012
But I gave it these photos and I said, "Can you build the Culver Theater and the surrounding area in Blender using the Blender MCP?" Stop there. And- I just wanna say how difficult that would be for a modeler.
47
00:07:21.332 --> 00:07:31.232
If you go to the picture- Yeah, explain this... like, it's, it's a lot of art deco, intricate architecture. It's not everything is not right angles, right? Um, like if you look at the sp- the- Yeah, it's got curves...
48
00:07:31.272 --> 00:07:40.332
four things holding up the sphere up on top, like that, that's a pretty difficult thing to model, at least for me. Yeah. Mm-hmm. Well, took your heads up with that part.
49
00:07:40.432 --> 00:07:48.622
[laughs] But, but I also didn't give it great images. I would be curious, I, I haven't tried if it could give it a reference video. Even if you couldn't, you could extract the frames- Yeah...
50
00:07:48.652 --> 00:07:54.272
if you had, like, a drone shot- Some screen grabs... and it gave you a bunch of frames. Yeah. Um, yeah, I mean, this is a very pr- preliminary test.
51
00:07:54.392 --> 00:08:03.122
I am curious how far, how much better it would get the more images you get, or if there's, like, a cap of, like, it just doesn't- I'm just curious about the first quality- It's not gonna get better...
52
00:08:03.122 --> 00:08:14.812
based on what you have from your cell phone, looks like So, right. So I gave it to this, I'm like, "Okay, go do the thing," and then here's the thing. Oh, wow. That's almost there. [laughs] It, yeah, it's pretty.
53
00:08:14.982 --> 00:08:25.072
[laughs] Yeah. So because I didn't have a great angle of the corner, it kind of... This is actually a second pass. I gave it a revision, and I said the corner should be more rounded and, uh, it's a corner door.
54
00:08:25.372 --> 00:08:34.512
The first pass was a bit flatter. But overall, it got the lamps, it got the overall shape of the building. It's okay. Yeah, like the theater sign is the biggest- It needs more refinement... egregious error.
55
00:08:34.612 --> 00:08:47.272
I mean, look, it, it's giving you rounded corners, some spheres. It's building it with, like, basic shapes. Um, look at the- Mm... look at the floor. Can you go back to your image? Let's see the floor by the door.
56
00:08:47.352 --> 00:08:58.832
Yeah, like that, that's such a terrible picture for the, the art deco- Yeah... you know, concrete stamping thing on the floor, but it, it did an interpretation of what it thinks it could do.
57
00:08:58.872 --> 00:09:12.012
And then, um, like how many windows are there? One, two, three, four maybe. Yeah, it didn't get all the door windows. Yeah. It got some solid thing, some things out. Um- But man, for a first pass- It, it, it- Jeez Yeah.
58
00:09:12.092 --> 00:09:16.972
This is good. And look, it even, it didn't really have a good s- Yeah... image on this side, but it sort of made up these art deco. Sure.
59
00:09:17.132 --> 00:09:27.852
Like there are movie posters on this side, and it's not really in the image, but it did a pretty good job of just sort of making something up- Yeah... that, like, made sense- Oh my God... for the design.
60
00:09:27.892 --> 00:09:37.232
Look, for a first pass, this was a r- I, I did, this was a refined, it, this is technically the second pass, and I said to, like, change a few things, and it did that.
61
00:09:37.492 --> 00:09:45.892
But look, overall, I mean- This is such a great starting point... it's pretty crazy. And look, you have all of your, uh, geometry separated out- And look, it definitely did get your, uh- In the, in the outliner...
62
00:09:46.042 --> 00:09:55.032
it did get your- Like if you- Yeah. This is all, like, ready to modify Yeah. I mean, yeah. [laughs] I'm, I'm thoroughly impressed, Joey. But look, it got your top thing pretty- Yeah... it got your top thing pretty well.
63
00:09:55.061 --> 00:10:02.972
Yeah. I mean, it's not what it actually looks like- No... but it- Not, not even close... I didn't give it a gr- but it got, it got the ball and it got some of the stuff here. Yeah. This is not a great...
64
00:10:03.032 --> 00:10:07.862
If I, uh, you know, if I had a drone and gave it some better angles, maybe it would get more into detail it would get. So that's, that's what I'm thinking, like, um,
65
00:10:09.212 --> 00:10:22.092
a 3D scan company and people that do digital twins of sets and stuff, they can, they have the equipment to then, instead of feeding it into Reality Capture and making a mesh, now they can feed it into Codex.
66
00:10:22.572 --> 00:10:42.112
I'm thinking too, like for me, it's like if a lot of the workflows I am messing with, like I don't need super accurate photo real 3D models, I just need like Graybox get me roughly there so that I have the geometry and we can do some camera moves, and then just use these as base inputs into like video to video workflows.
67
00:10:42.542 --> 00:10:53.922
Mm-hmm. So like, I'm like, if it gets me 70% of the way there of like what it should look like, and has roughly everything in place- Mm-hmm... we can clean all the other stuff up, uh, with like gen AI passes. 100%.
68
00:10:54.212 --> 00:11:02.392
I, I can see that. Also, I'm seeing this for traditional VFX where you have to send a team out to scan, um, put it all back together, right?
69
00:11:02.712 --> 00:11:13.122
Now that is just a bunch of phones really, and maybe some ladders to take [laughs] higher enough pictures. Oh yeah, or it's about drone. I mean, I have, uh, I have the DJI Pocket Drone- Yeah...
70
00:11:13.162 --> 00:11:21.862
which they, like, built it so it's technically under the weight. They have to go through- And then what, uh, how long does it look like it took you, like six minutes in Codex to go through all of this? It's insane, man.
71
00:11:21.862 --> 00:11:33.172
Yeah, it took, once it spun up the server that built this in, yeah, maybe 10 minutes. And then you can also light and modify that. Yeah. Oh yeah, and also I didn't ask for it, but it gave me a video- Like a render...
72
00:11:33.212 --> 00:11:45.782
render movement. Yeah. I don't- And then I said, "Oh, hey, can you relight it so it's like the lamps are illuminated and the building's illuminated, and it's like a moody night scene?" Oh. And then it did this. Yeah.
73
00:11:45.812 --> 00:11:50.912
Obviously, you could tweak it and refine it, but I just said, "Hey, turn it, like these into practical sources and lights and-" Yeah...
74
00:12:06.016 --> 00:12:19.916
B- you know, a year or two ago, I thought that the gen AI revolution would really take us from a 3D world back to a 2D world a little bit. Because why bother with 3D when you can just get to final pixel, right?
75
00:12:20.166 --> 00:12:31.596
And this was my incorrect assumption back then. Since then, I've noticed, and I learned the hard way, is that you still need 3D because gen AI just doesn't give you the control, right?
76
00:12:32.136 --> 00:12:47.896
If you want the camera move, if you want, you know, like that sphere to be held up by four things, like these very nitpicky things, then you gotta do the thing in 3D and then, um, gen AI will hopefully fill that photorealism gap in 2D, which is what we're seeing a lot now.
77
00:12:48.716 --> 00:13:00.756
I think in a year or two from now, my prediction is we're gonna be in a entirely different world that is so deeply in 3D, but without the complexity of traditional 3D. Like,
78
00:13:01.696 --> 00:13:06.296
if you think about- Yeah, like a simplified user interface... yeah, just like all the things you don't have to do.
79
00:13:06.376 --> 00:13:21.136
You don't have to model, you don't have to do shading, you don't have to do lighting, you don't have to do, uh, geometry optimization and like, uh, rigging and skinning and all the things that is associated with the 3D complexity bubble, right?
80
00:13:21.296 --> 00:13:29.116
And you're talking about people's, uh, job to... is to do very specific things, very specific lanes, and now you're gonna have
81
00:13:30.196 --> 00:13:49.216
just an operator, just a, a, you know, AI 3D operator that can do all of these complex 3D functions in Blender, in Unreal hopefully, in Maya, and still have, quote-unquote, "a traditional workflow," but with the simplicity of using ChatGPT or Claude or just like prompting and typing.
82
00:13:49.716 --> 00:13:52.236
Yeah, especially as these things get faster- Yeah... and more real-time.
83
00:13:52.296 --> 00:14:04.376
Like the thing that was also really impressive with this was, uh, it took control of the computer, but it felt much more responsive than when Claude takes control of my computer and it's still like, "Okay, look at the computer.
84
00:14:04.696 --> 00:14:12.576
Okay, let me assess the screen. Let me figure out what I'm looking at. Okay, let me take an action." And it's like still this very slow process where the Codex felt a lot smoother even...
85
00:14:12.616 --> 00:14:14.876
I mean, it was also talking to it via MCP under the hood.
86
00:14:15.476 --> 00:14:28.675
But as this keeps getting faster and faster, and voice models and all that stuff gets better, and then you're just literally in a gray box room or just in a whatever space you want, and you're just talking to the thing, and you're like, "Oh, hey, like, okay, move the camera here.
87
00:14:28.736 --> 00:14:32.365
Okay, let's add a tree over here. Uh, all right, let's, uh, you know, I need to make that wall bigger." Yeah.
88
00:14:32.376 --> 00:14:40.956
And you're literally directing, and it's doing all of the 3D set build stuff under the hood, and it's just responding to the direction- Hundred percent... that, uh, you know, it's giving it.
89
00:14:40.966 --> 00:14:52.636
A- and, and the other two things that are kinda converging and will sort of meet in a really happy place, I think is, uh, the fact that, um, GPUs are getting more powerful, right? That... Nvidia is working on this.
90
00:14:52.656 --> 00:15:03.956
They have, uh, Blackwell now. They're gonna have future GPUs. So on the 3D side, you're gonna be able to run trillions of polys, right? Right now we're in the billions. In the future, hopefully trillions.
91
00:15:04.076 --> 00:15:13.396
Like insane amount of polys, more than you can imagine. And then at the same time, the AI models and inference will get faster or more accurate.
92
00:15:13.656 --> 00:15:25.306
So your interactivity with AI, your responsiveness, your iteration, that's becoming more natural, and then the stuff you're making is becoming more complex and more lifelike. So the end result is- Yeah...
93
00:15:25.306 --> 00:15:33.336
like you being able to recreate reality or recreate a completely new world effortlessly, just effortlessly. And look, also I was just...
94
00:15:33.376 --> 00:15:37.676
This gave it Blender, but y- you know, Blender's the orchestrator or the base level too.
95
00:15:37.756 --> 00:16:01.046
Like imagine connecting this stuff to all of the other models and, and things that we've talked about, image to 3D, text to 3D, uh, Marvel World and World Labs, and using all of those tools at its capacity to bring in or create even more photorealistic things into your scene, and then orchestrating in your 3D scene, and then it's all there and consistent and, uh, controllable, but much more higher quality.
96
00:16:01.056 --> 00:16:07.016
Yeah. Uh, I, I, I mean, it's so hard to predict the future, but like right now the way we're gonna...
97
00:16:07.096 --> 00:16:23.256
we're sort of establishing the workflow ac- And I see this across the board, um, is, uh, uh, you know, stuff on LinkedIn, you know, stuff on Instagram and stuff, is you do a lot of fine creative work in 3D, and then AI is essentially a renderer.
98
00:16:23.596 --> 00:16:33.816
It like takes you to a style or a realism- Mm-hmm... level that you can't- Mm... really achieve easily with 3D. You're saying that in the future, like that will be the workflow.
99
00:16:33.896 --> 00:16:40.156
Like 3D will play the piloting part, and then the finishing part will be AI. Yeah. Yeah. Yeah, yeah. For sure.
100
00:16:40.796 --> 00:16:50.236
Uh, other thing too, I, I saw this and I just wanna like flag it, of just like what even just new formats or new things will be, like be created with these tools.
101
00:16:50.576 --> 00:17:02.096
So Alex, uh, Baraschkov built with Codex, uh, what he's calling, uh, AVAL or AVAL, a new open source format for interactive video on the web. And he has this demo of this sort of this
102
00:17:03.056 --> 00:17:11.276
interactive video with, uh, grass blowing. I know this clip's from a movie. I just don't remember what. Um, and but just,
103
00:17:12.316 --> 00:17:28.016
you know, just he built this whole brand new format for handling video, uh, and interactive video on the web with Codex, and so it just makes me think like what other d- new things, formats are just gonna be open and unlockable for people- Damn, imagine that, like just making your own USD or FBX file format.
104
00:17:28.756 --> 00:17:41.236
You know? Because- Mm-hmm... you can. Exactly. Like you have the ability to have complex software tasks automated. Yeah. Small file size, low CPU overhead, alpha transparency- Wow... web native runtime.
105
00:17:41.496 --> 00:17:49.206
It's gonna be crazy And like the, the future is so hard to imagine because it's literally not built yet. Like we, we can't- Right [laughs]...
106
00:17:49.226 --> 00:17:59.076
we can't draw a straight line 'cause the, the points on the l- beginning of the line aren't even there. I remember you poo-pooed when we talked about that thing a few months ago that was like, uh, what was it?
107
00:17:59.136 --> 00:18:07.832
It was like generative websites. It was like websites sort of built on demand- Yeah, I would, I would poo-poo on something like that... based on your needs[laughs]
108
00:18:07.962 --> 00:18:15.262
But I feel like that is, I feel like that's still gonna kinda come into play more and more where like, i- i- if you have a use- Yeah...
109
00:18:15.272 --> 00:18:27.752
you need software, you need something, these like tools or, or solutions that spin up based on need, um, because it's just so easy to spin something up from scratch, uh, that does exactly what you need to do. Yeah.
110
00:18:28.132 --> 00:18:40.892
Also, it could take us into like the Wild West of software end formats, and things that are, uh, not bu- you know, debuggable, just built by vibe code. It, it, it could be like a total mess, right?
111
00:18:40.952 --> 00:18:51.212
Like one pipeline will look nothing like another pipeline. So if a VFX artist worked on one show, and they go to another show, they're like, "What, what am I doing?" Yeah. [laughs] Yes.
112
00:18:51.272 --> 00:18:56.742
That, that's probably gonna become very messy. Uh, especially if like every, yeah, if every studio builds their own- Yeah... proprietary system.
113
00:18:56.852 --> 00:19:11.212
I think, yeah, maybe the systems interfaces are unique for whoever builds them, but I think open standards are gonna get even more important. Yes. Like, um, USD, Open Timeline, um, what other- Uh...
114
00:19:11.222 --> 00:19:11.792
open standards are there? Yeah.
115
00:19:12.102 --> 00:19:21.262
Uh, yeah, a, a lot of stuff that they're [laughs] talking about this Sunday is gonna just keep getting more and more vital for, so like you could build your own system, but like making sure your system can communicate with everyone else's systems.
116
00:19:21.272 --> 00:19:29.922
Wow, you really just convinced me to go to the Sunday open source thing. [laughs] I'm like, "Yeah, that sounds really relevant. I, I need to go. I need to..." Yeah. [laughs] Yeah. [laughs] Whatever.
117
00:19:29.952 --> 00:19:37.942
I mean, that's what I, that's what I'm thinking. I'm looking at the talks. They're like, yes, all of these things where like I'm messing around with 'cause we need ways to like... If I'm building an app- Yeah...
118
00:19:37.942 --> 00:19:43.072
and I'm building a timeline, I need to make sure that timeline can get sent to, uh, Resolve or Premiere, and- Yeah... that type of timeline. I, I think you're right.
119
00:19:43.172 --> 00:19:52.112
Open source is, open source and open formatting become more important than ever. But that's not gonna stop somebody from coming up with their own USD.
120
00:19:52.232 --> 00:20:01.092
I'm just thinking like just going back to USD, and how much of a revolution that it was, and how long it took. Like Pixar took 10 years to make that from scratch.
121
00:20:01.972 --> 00:20:12.192
And, uh, that was, I don't know, the early 2000s is when the work started. And just now we're seeing, um, all that trickle into all the nooks and crannies of media and entertainment. I mean,
122
00:20:13.332 --> 00:20:21.152
it's still, it's still like, um, not for the amateur because it, you know, you need software to be able to handle it correctly.
123
00:20:21.372 --> 00:20:34.372
But I'm just thinking if, um, there is such a thing as a standard for building your own standard. Uh, yeah. I think it depends what you wanna do and what you need to like be able to transfer- Yeah... in and out.
124
00:20:35.172 --> 00:20:47.192
Um, yeah, I mean, I don't know. Maybe I'm naive, but it's like, yes, the, those like USD and stuff took a long time to adapt because there are only a few big software players that would handle it- Yeah...
125
00:20:47.292 --> 00:20:52.892
and adapt to it. But- So software was expensive, and now software is super cheap, right? So that's the thing that flipped kind of. Yeah.
126
00:20:52.932 --> 00:21:01.062
And it's like, uh, uh, and it's like, oh, if you don't accommodate what I need, I can just build it myself. Yeah. Right.
127
00:21:01.062 --> 00:21:07.962
And that, that's the part where I'm like maybe this is a naive take, but it's only gonna get better and faster and easier- Yeah... to spin up your own software.
128
00:21:07.962 --> 00:21:17.632
And, and there's, there's also all types of decentralized resources that you can tap into. You know, what if, what if there's a format that just runs file stuff?
129
00:21:19.152 --> 00:21:27.842
What if there's a format that just runs NVIDIA's WebGPU stuff, right? And it's not really built for anything local. It can run on a Mac, on a PC, or anything. Yeah.
130
00:21:27.892 --> 00:21:40.272
That hasn't been really imagined as far as I know, right? Like nothing is really 100% optimized for 100% decentralization. No. I mean, unless you're building it yourself. Yeah.
131
00:21:40.312 --> 00:21:53.432
Maybe like GLB- But that's an open question... GLB formats, like stuff that, um... Yeah. Anyway, I don't know enough voodoo science behind those formats. But yeah, like it's an interesting area to watch for sure.
132
00:21:53.532 --> 00:22:08.481
So not only the workflow itself, but the delivery, and the formatting, and the exchange of information. Yeah, yeah. All right. You wanna talk about, [laughs] uh, o- older software now? Yeah. New, new stuff in- Yes...
133
00:22:08.532 --> 00:22:18.032
established software? [laughs] All right. Let's talk about Nuke. Nuke just announced a Smart Roto function along with some updates to Grip Tape. Okay.
134
00:22:18.092 --> 00:22:27.621
Smart Roto, new Nuke plugin that handles basically it's smart roto, smart roto-ing. [laughs] Yeah. I'm, I'm betting it's like AI- Just-... powered under the hood. I would have a- assume so too. Yeah.
135
00:22:27.741 --> 00:22:36.402
I mean, this is stuff that's already been in Resolve with IntelliMask- Yeah... IntelliTrack, and Premiere and After Effects- Yeah... has pretty good, uh, roto Foundry's smart.... plugin.
136
00:22:36.412 --> 00:22:47.492
They're so, uh, sensitive to the use of the word AI or any sh- any sort of artificial intelligence connotations. Although they just acquired Grip Tape, and they're going full I- AI.
137
00:22:47.772 --> 00:22:56.282
But in a way that is, makes sense to like high-end professionals, and that works in professional workflows. Dude, nobody wants to do frame by frame roto. Like I think it's been a, a...
138
00:22:56.282 --> 00:23:04.012
It's a delicate balance, but- Come on, man. Those roto artists- Yeah... don't like to do roto. No, no. I'm talking more about Grip Tape and stuff. Like, you know, the, you were saying they go fully, fully AI.
139
00:23:04.092 --> 00:23:10.872
But like fully AI in a way that still integrates w- with high-powered professional pipelines. Yeah. They're, they're doing it the right way, for sure.
140
00:23:10.952 --> 00:23:20.772
Grip Tape, to me at least, the Grip Tape Enterprise feels like comfy for VFX, for the lack of better terms. Mm. Um, so yeah, I saw this announcement.
141
00:23:20.832 --> 00:23:32.202
It's like, um, OCIO support, which, you know, something you would have to probably build in Comfy. I'm not sure there's a node for it or anything. Um, local GPU inference and stuff- What...
142
00:23:32.202 --> 00:23:42.332
which I thought was super interesting, uh, because studios are super sensitive about- Yeah... using the cloud for unreleased IP- Yeah, yeah... and things like that. Yeah. They're definitely more in the- Yeah...
143
00:23:42.392 --> 00:23:48.212
keep it local. It's all good progress. Um, it- it's really converging with traditional AI nicely.
144
00:23:48.312 --> 00:23:59.752
I think, uh, the Foundry and Grip Tape understand, obviously, given their history with Nuke, they understand that world really, really well. That is their core, uh, audience, their core demographic.
145
00:23:59.892 --> 00:24:08.764
So I expect nothing less from the Foundry, and they, they have, uh, sort of fine-tuned themselves into this world. No pun intended. Yeah.
146
00:24:09.164 --> 00:24:21.944
I think smart roto only gets more important with, uh, AI-generated stuff when you gotta isolate elements- Yeah, and I mentioned on the pod before, like, uh, because of the inherent noise, de-noising of AI generations,
147
00:24:22.844 --> 00:24:34.324
roto stuff and matte stuff doesn't work well with it as far as I- I've heard. I mean, uh, uh, from some of my testing, I- I can't 100% confirm that, but- Oh, right. You mentioned that last time. Yeah.
148
00:24:34.424 --> 00:24:44.113
No, no, I don't think that's true 'cause we've been, we've been masking the crap out of generated stuff to clean it up, and it's- Okay. It's fine. So maybe it's the- Behave- it behaves the same as- as roto video...
149
00:24:44.164 --> 00:24:56.984
I don't think it's roto, but maybe it's tracking, camera tracking. 'Cause, um, camera tracking is essentially trying to figure out a 2D- Mm... space into a 3D camera move. Okay. We haven't done as much- Yeah... of that.
150
00:24:57.084 --> 00:25:08.864
So yeah, maybe. I- I'll... Okay. I'll tr- I'll try to try some shots- Yes... to see if we can track it. All right. [laughs] Rage bait time. We're just doing this for the, for, for the rage bait? Okay. Yeah.
151
00:25:08.884 --> 00:25:18.024
We're doing this to clip this for... AI-generated Odyssey movie costing a few thousand dollars [laughs] unveiled ahead of Nolan's $250 million feature.
152
00:25:18.964 --> 00:25:26.954
Uh, I saw s- one of these things fr- frame it as like going, going head-to-head, and it's like [laughs] So that's what I saw. Who's going- I was like, "Wait, uh-" Who's going head-to-head? "...
153
00:25:26.984 --> 00:25:38.494
Nolan makes like three-hour movies, right?" It's like, wait, somebody made a three-hour Odyssey with AI? I have to see it. And if you look at the length of that video clip, one minute 52 seconds.
154
00:25:39.044 --> 00:25:49.004
Well, this is a trailer for the epic that is coming. Yeah, I think this is it. [laughs] I don't think it's coming. [laughs] You think they, they- I, I think they ran through their credits with this. Yeah.
155
00:25:49.164 --> 00:25:58.714
I mean, you know, I'm looking at the trailer right now. I don't have any sound on it. Yeah. But it's got shots and- I mean, it has- You know- It has some beautiful shots- AI's gonna take your shots... here and there.
156
00:25:58.724 --> 00:26:04.084
Take your shots. Uh, obviously well color graded, um, synthetic humans, I'm guessing.
157
00:26:04.664 --> 00:26:20.164
But like, yeah, some of the shots completely break, uh, especially, yeah, anything with perspective or distance, AI still can't figure out real accurate lensing and glass, and your eyes can easily tell that that is artificial.
158
00:26:21.024 --> 00:26:29.504
Um, so for example, in... Like, you can pause on any frame- Uh-... and I- I'll point it out. That feel- this feels like the... That feels like the least of my issues with this.
159
00:26:29.544 --> 00:26:42.404
There's just more like the camera geography and placement's just off, the performan- pro- performances are, you know, the talking and everything still feels just uncanny valley. Th- That has not been...
160
00:26:43.224 --> 00:26:52.064
Like, that has not been broken from a fully synthetic- Yeah. But-... point of view. Ooh, cyclops. Is it? With two eyes? No. Oh, okay. I think he's coming out of Hades... one of the world's creatures. Yeah.
161
00:26:52.144 --> 00:27:03.453
Ooh, that, that had a nice fisheye- Happy to... or anamorphic-y lens to it. Again- I, I mean, I feel like I've seen five different styles- Yeah... in this one trailer. You know what they should've done?
162
00:27:03.484 --> 00:27:15.224
They, 'cause s- I don't... Well, you're not on X as much, but like when the first costume, um, like Matt Damon appeared, uh- Yeah... images from the film, the history dweebs- That's not period accurate...
163
00:27:15.254 --> 00:27:23.203
were like, "That's not-" Yeah... you know, Greek- Yeah... period accurate. And then the period accurate ones was like these ridiculous like garbage- Yes... chute looking things.
164
00:27:23.304 --> 00:27:31.124
[laughs] So someone, if they wanted to go all in with the Odyssey was to make it a period- Yeah, I think it was like the bronze era-... accurate, uh, version-... or something... of the Odyssey...
165
00:27:31.284 --> 00:27:39.624
and the armor was like way more modern. Yeah. Something like that. Yeah. Something like... I mean, the bottom line is the armor that is in the movie- Yeah... looks cool.
166
00:27:40.004 --> 00:27:49.764
Uh, the armor that was accurate [laughs] did not look cool- Yes... compared to our standards today. So- Yeah... uh, you know, hope that helps. When they go to Hades and then you're like, "What's the accuracy of Hades?"
167
00:27:49.794 --> 00:27:59.644
And it's like, mm. Yeah, I think The Hollywood Reporter is really just struggling for clicks here with, with this headline. Uh, it's, I'm, I'm always sh- more shocked who the hell is their PR rep that gets...
168
00:27:59.744 --> 00:28:07.904
It do- it wasn't just Hollywood Reporter. Yeah. It's like the same Tilly Norwood type, uh, headline, you know? I was gonna say, that was, that came out a few weeks ago, and I've got a strict ban on. Yeah.
169
00:28:08.024 --> 00:28:16.404
We're not talking about that stupid shit. But yeah, that story came out- She got cast in a movie... too that was like, uh, oh y- Yeah. She's gonna be in her first movie. I was like, what the hell is this shit? Yeah.
170
00:28:16.414 --> 00:28:26.534
And that was on everywhere. That even made it to the point that like my completely non-AI wife was like, "There's some like AI actress that's gonna be in a movie or something." She like- I don't know...
171
00:28:26.544 --> 00:28:35.234
aren't you gonna talk about it on the podcast? [laughs] Kudos to whoever the hell is behind that for getting this stupid, the PR press for this thing. That's probably- Yes...
172
00:28:35.244 --> 00:28:45.084
that's been the most amazing feat- Yeah, absolutely... to me out of all this. So I mean, uh, I, I will give credit to the guy, what's his name, Ash, uh, the guy that made this. Ash, uh- Like, it- Ash Kucha...
173
00:28:45.164 --> 00:29:02.424
even though it doesn't, it probably won't work as a cohesive storytelling feature film, the shots are cool by themselves. So hats off to just pure generation. Yeah. Yeah, but we know like, we know shot, we...
174
00:29:02.464 --> 00:29:11.624
The AI's good at making cool shots. Yeah. Like, that's been proven. It's like making a cohesive thing where you're like in the story, it's got consistent characters, consistent shots.
175
00:29:11.804 --> 00:29:18.024
Now we're talking about this, let me bring up one thing that I did see that's- Well, while you're bringing up a quick rant. Uh, yes. We talked about this time and time again.
176
00:29:18.384 --> 00:29:28.294
I'll cover it one more time 'cause we're on this topic. The true unlock with AI is to combine it with physical performance and on location shooting.
177
00:29:29.184 --> 00:29:36.024
If you can just solve for the expensive bits of production, let's say you half your cost, that is the win.
178
00:29:36.664 --> 00:29:45.724
Having said that, there are people trying to figure out fully synthetic everything, and I think they're really limiting themselves by doing that. It's like I'm not gonna put a camera on anything.
179
00:29:46.484 --> 00:29:58.844
I'm not gonna put any human performance in there. Why? Because AI can do it all. I'll just do the whole thing synthetic. I think you've set yourself back like so far back with quality if you do that. Yeah.
180
00:29:58.984 --> 00:30:08.504
I think, yeah, what you're saying the unlock is human performances. I would argue I don't think you need, you know, locations. Um, you know, I think you can get away with a lot if-
181
00:30:09.484 --> 00:30:14.404
It helps, you know, immerse you a little bit- Yeah, you don't- But, uh, i- if you have the wall, the wall looks cool... I, I don't think you need full locations.
182
00:30:14.504 --> 00:30:29.204
I do think you need a, a film, like a cinema camera with real glass and real film back, because the perspective, the distortion, the sort of geometry of the real world, AI just can't get right.
183
00:30:29.264 --> 00:30:41.004
So then if you're just bringing that in as a reference plate, and then have AI fill everything up, I- that'll go so far to make it look realistic. You definitely need props and objects. Yeah.
184
00:30:41.024 --> 00:30:49.964
For anything they touch and interact with, you need that. You need people- Yeah... stuff they touch. But you can do matte painting in the back, and layered- You- Par- ha- have parallax and all that stuff... worlds. Yeah.
185
00:30:49.974 --> 00:30:54.564
Yeah. It's, it's like LED volume in a, in a lot of cinema. Yeah. But I think that is the best balance.
186
00:30:54.664 --> 00:31:03.624
Uh, look, I mean, sure, if you're like st- um, super low budget, and you wanna try to make something that is just completely generated, like, at le- at least the option's there.
187
00:31:03.684 --> 00:31:12.954
I still just, I just don't think the quality is there- Yeah, um, so like-... to make something that feels completely immersive. Yeah. That said- Okay [laughs]... this was the teaser thing.
188
00:31:13.623 --> 00:31:29.444
Uh, ByteDance released a trailer teaser video that was made using CDance 2.5, and this is all from... This is like a, um- Football, you mean?... commercial. I'm in World Cup mode, and, uh, it's all football now.
189
00:31:29.474 --> 00:31:39.064
And then when World Cup ends, it goes back to soccer. Man, that's pretty good. [laughs] This is 2.5? Uh- Yeah, I've been waiting for some 2.5 outputs... this is 2.5. The motion.
190
00:31:39.234 --> 00:31:47.334
It's starring a kid who's kinda like- Yeah... kicking the ball through London. Uh, movie looks good, kid looks great. I mean, the ball contact with the feet- Everything about this feels-...
191
00:31:47.334 --> 00:31:57.594
and yeah, even like that shot there, that should be so hard for AI to do, and it does it well. Yeah. This, um, you know. [laughs] So we might be eating our words. This is- We might be eating our words... pretty good.
192
00:31:57.594 --> 00:32:08.664
And that is okay. This is the whole point of the podcast. You know, follow us for the journey. [laughs] I mean, obviously, uh, the caveat with everything is like, "This is the worst it'll be today." Yeah.
193
00:32:09.204 --> 00:32:18.964
Still no word on when 2.5 is coming out. I keep seeing updates that it's getting de- Yeah... that keeps getting pushed back. But, um- Some, uh- It's coming. I think I did misspoke speak last time.
194
00:32:19.003 --> 00:32:30.764
I think I said 50 references, but I think it's 30. Oh, gosh. Kill me now. Only 30 references. [laughs] Um, are you kidding me? Where am I even gonna find 30 references? I have like three at most, in most generations.
195
00:32:30.784 --> 00:32:41.594
[laughs] Here's my photo. Here's my entire photo library. Just go, go make a movie. [laughs] Oh, you know what? The Culver City Theater could use 30 references. Yeah, that could use more. Yeah, yeah.
196
00:32:41.604 --> 00:32:53.444
That could use- Like all the close-ups and things like that. Yeah. For sure. But you have to see if they'll place all the references in the right s- location spatially. Yeah. You have to give it a wide shot of stuff.
197
00:32:53.904 --> 00:32:58.424
I, I am curious if you, how much instruction you can give it with the references that it'll retain and understand.
198
00:32:58.484 --> 00:33:06.594
If you, like, give it a wide shot, and then close-ups of things, will it understand like, "Okay, this is how everything relates," and then, but like, "The action should be here, and things should happen here"? Yeah.
199
00:33:06.664 --> 00:33:16.304
Um, yeah, it'll be the new Wild West of, like, how do you coordinate and direct this stuff, um, when you have so many reference images, and like what can it understand and stick to?
200
00:33:16.944 --> 00:33:27.264
Okay, so when 2.5 comes out, you and I both have to generate some reference images. [laughs] Okay. All right? We might go broke. We might go broke from that, 'cause, 'cause it might come out- From that generation...
201
00:33:27.284 --> 00:33:34.684
and we also, we don't know how much they're gonna charge, and if they're gonna charge you more based on if you have more reference images. [laughs] Oh, yeah, that, yeah. Okay.
202
00:33:34.944 --> 00:33:45.784
A 4K, uh, 4K- We should start a GoFundMe for that generate. C... [laughs] Right. The 4K CDance output, like one shot is roughly like five bucks. Oh my God, really?
203
00:33:46.304 --> 00:33:57.724
If you max it out to 4K, and like even just, I think, five seconds, it's like roughly like $4 or $5. Yeah. It's steep. It gets pricey. [laughs] Yeah, you can't do iterations that well for $5. Yeah. [laughs] Wow.
204
00:33:57.784 --> 00:34:03.754
Cha-ching. Those are LA prices, man. This is- [laughs] Right. Is it cheaper to run it in Tech Texas?
205
00:34:03.754 --> 00:34:12.534
If they keep those prices up, it'll be like the, uh, other software development companies when they like, were like, "Oh, we're gonna rehire the people because the tokens are too expensive," than- That's happened...
206
00:34:12.554 --> 00:34:16.554
what it costs for a person. I know, it's happening. That's happened. It's gonna be the same with film, where it's like, oh- Yeah...
207
00:34:16.584 --> 00:34:23.614
it's actually cheaper just to like shoot this for real than to, uh, blast CDance with it. [laughs] It, uh, yeah, the costs have got to come down.
208
00:34:23.864 --> 00:34:38.264
Um, I mean, s- I wonder how much it, of it is the actual, like data center GPU cost, and how much of it is just greed? [laughs] Uh, yeah, I don't know. I mean, I think a mix of both.
209
00:34:38.334 --> 00:34:49.244
'Cause like also, depending where you generate, there's, um, for CDance especially, the prices do vary a bit. Quite a bit. Yeah. Okay. So we'll do, we'll do one generate... I can afford like, I don't know, $20.
210
00:34:49.304 --> 00:34:58.364
I'll do four. [laughs] Cl- Yeah. We'll do like four budget. Okay? And that's all, Joey, all right? You just get four takes. Yeah. When 2.5 comes out, we will, we will do that. Okay, uh, last thing, quick thing.
211
00:34:58.404 --> 00:35:10.574
In the open source world, um, Thinky Machines, which was- Mira Murati?... Mira Mu- uh, what's it, Mira Murati? Mira Murati, who left, uh, one of the OpenAI founders, left, started Thinky Machines.
212
00:35:11.374 --> 00:35:21.204
They've been working for a while. They raised a bunch of money. They now have released their first official model called Inking- Yay... and they released it as open source Yay for open source. Open weights. Yeah.
213
00:35:21.284 --> 00:35:33.364
It, it seems like a- Yes... pretty solid model based on this article. Like, it, it does, uh, significantly step up the competition with ChatGPT and Claude. Yeah, I mean, it is definitely not at that level.
214
00:35:33.384 --> 00:35:40.774
But in the world of open source, it's on par with Kimi 2.6. Uh, I mean, definitely this is like a huge leap for- Absolutely, yeah...
215
00:35:40.824 --> 00:35:47.234
an open source American model, 'cause like pretty much all of the open source- Yeah, like DeepSeek and stuff... like decent models have been- Yeah...
216
00:35:47.344 --> 00:36:00.044
uh, from Chinese, yeah, have been from Chinese, Chinese model, uh, uh, AI companies. Uh, so yeah, I mean, this is a big leap for putting out this big model, open sourcing it, um, multimodal.
217
00:36:00.344 --> 00:36:07.824
I'd be curious, once someone, or I think it might be, I think they have a spot on their website called Tanker- Yeah... that you can, like, mess around with it.
218
00:36:07.864 --> 00:36:18.576
And then obviously you could run this open source, or I'm sure a bunch of providers will add, uh, the ability to Work with this model, uh, via third-party tools. Yeah.
219
00:36:18.605 --> 00:36:29.766
Uh, and so going back to Mira Murati, she was the CTO of OpenAI. She left, and then Ilya Sutskever was, I think, the chief research officer at OpenAI. He left. Mm-hmm.
220
00:36:29.836 --> 00:36:41.976
And then, uh, like, those are the two people I'm watching closely. And then the third person I'm watching closely is Yann LeCun, who left Meta and now has a billion dollar in funding for his own company.
221
00:36:42.056 --> 00:36:53.256
He thought LLMs were, like, not the way, right? He still thinks that. I think he said something like, "LLMs have two years left," because, uh, y- I think he's right. Scaling is not working.
222
00:36:53.296 --> 00:37:02.956
It's giving you diminishing returns. Um, and, you know, no matter how many more billions of parameters you train on, intelligence doesn't go up by that much. And what's the alternative?
223
00:37:03.156 --> 00:37:10.486
Just a, a bi- a bigger, like, a world model? It's the JEPA architecture, according to him. We could spend another episode on the JEPA architecture,
224
00:37:11.416 --> 00:37:26.296
but, uh, essentially, you move away from a text-based approach into a more abstract approach where text, images, video, any kind of just become seamless with one another, um, rather than us discreetly building a text model, image model, video model.
225
00:37:27.036 --> 00:37:39.396
Mm-hmm. Mashing those three things together to get reasoning, um, this, uh, is inherently more fluid in nature. Okay. All right. Uh, I'm very curious about that. Me too. [laughs] Um, okay.
226
00:37:39.476 --> 00:37:43.216
So this is out, and then also, I know I just compared it to Kimi, but Kimi
227
00:37:44.516 --> 00:37:59.716
is also teased with a pretty sick launch video that, um, Kimi 3 is about to come out, which is expected to be China's largest AI model with two to three trillion parameters. Jeez. Open weight, um- [laughs]...
228
00:37:59.796 --> 00:38:11.786
on par with, like, Claude Opus- Yeah... 4.8. So- Oh my God. Did you say trillion? Yeah. So, um- Jeez... yeah, that's coming- [laughs]... soon. How big, how big is the file gonna be? I have no idea.
229
00:38:11.836 --> 00:38:21.486
Like, the, the, the weights. Yeah, I got no idea. Like one terabyte? [laughs] I have no idea. [laughs] So, uh, you know, that's, that's coming. [laughs] That's awesome.
230
00:38:22.226 --> 00:38:34.436
Uh, I mean, like, it's not awesome in the sense that the biggest, baddest model ever is coming out of China and not here. Biggest, baddest- But it's awesome... open- Open source... source model. It's crazy.
231
00:38:34.576 --> 00:38:46.636
[laughs] It's crazy. Yeah. I wonder why Thinking Machines launched their thing open source. Um, do you think it's strategically to just get people to use it and tinker with it- I think so... and get their hands on it?
232
00:38:46.796 --> 00:38:54.596
I think, it's like, what do you do when you're starting a new thing and you're already, like, you're not gonna beat ChatGPT or, or Anthropic. So, like- Right... what do you do to make a splash?
233
00:38:54.656 --> 00:39:04.406
And if you've been trained in a pretty good model and you wanna get people on board with it, open source it. You, you start to get users on it- Mm-hmm... build momentum on the flywheel, if you will. Yeah.
234
00:39:04.416 --> 00:39:16.376
I mean, it's what, uh, stable diffusion did so well with SD 1.5, SDXL, right? They gave it away for free very early on, and now it became synonymous with image models, like, a few years later. Yeah.
235
00:39:17.016 --> 00:39:29.196
Uh, I mean, I re- they haven't released it yet, but one of the first things that Thinking Machines teased, like, a few months ago was that conversation chatbot that was much more...
236
00:39:29.236 --> 00:39:47.996
In the demo they did, it was, like, a much more responsive, reactive, like, fluid conversation that they were having, like, where if you interrupted the model, it kind of, like, picked up with the new chat, and it was, uh, uh, something that it seemed like they were focusing on was, like, a conversational aspect, and that part hasn't been released yet of whatever they're working on there.
237
00:39:48.076 --> 00:39:55.856
I did see a really interesting feature that I didn't see with other models, is it can fine-tune itself. Oh, interesting. Based, based on- So, like, it learned. Yeah.
238
00:39:55.996 --> 00:40:11.036
So, like, if you need a m- let's say you wa- you wanna run an agent that runs a dentist office or something. Like, it has to- Mm-hmm... fine-tune on den- dentist stuff, like, you know, orthodontist and things like that.
239
00:40:11.136 --> 00:40:15.546
It could probably fine-tune itself if you just ask it to and give it enough data.
240
00:40:15.916 --> 00:40:23.276
That's cool, 'cause right now it's sort of like a hack where you have the model, and you have a bunch of markdown files, and then the markdown files- Yeah... update themselves.
241
00:40:23.296 --> 00:40:32.275
Right, and then you gotta build a giant context window, and I think the context window here is pretty insane too. I think one million characters or one million tokens, which is a massive context window.
242
00:40:33.816 --> 00:40:42.456
Uh, I think I saw on Tinker at least it was, like, capped at 256, but maybe- Yeah... if you go full blast, you c- it's a million. Yeah.
243
00:40:42.736 --> 00:40:54.476
So, like, it's definitely a modern model by today's standards, which also happens to be open source, and, uh, yeah, I'm, I'm hopeful that this is a, a good alternative to ChatGPT and Claude.
244
00:40:54.576 --> 00:41:02.676
Um, I think competition in this space is what we need, you know, just to get things better and more efficient. I'm just really worried about using all those GPUs.
245
00:41:02.726 --> 00:41:06.716
[laughs] Like, can we get a model- Yeah, and stuff that could run locally too. Yeah. I mean, I think that
246
00:41:07.856 --> 00:41:13.736
as workflows get more established and as it sort of pushes into, like, higher-end film production workflows or other things, like
247
00:41:15.196 --> 00:41:31.656
exploring, uh, stuff that you could run locally is gonna get more interesting- Absolutely... and more needed. Mm-hmm. Especially when you start comparing the costs of API tokens versus spinning it up yourself. Yep.
248
00:41:31.816 --> 00:41:40.436
Absolutely. All right. Good place to wrap it up. This went deeper than I thought, but this was a fun one. Uh, links for everything we talked about at genoispodcast.com.
249
00:41:40.796 --> 00:41:48.786
If you wanna catch Joey and I at Siggraph, we'll be there Sunday at the open source thing and Monday morning as well. And- Yeah... I think, Joey, you're gonna attend- I'll be around a few days longer.
250
00:41:48.876 --> 00:41:57.056
Yeah, you're gonna attend the- Yeah, definitely be there Monday, Tuesday. I don't... The thing, that was a, think it goes until, like, Thursday. That's why I was also curious, like, how... That's, like, longer than NAB.
251
00:41:58.316 --> 00:42:08.516
Yeah. It's a lot. It's a long thing. A, a lot of it- Quite a long thing... a lot of it is, is, like, the big key- keynotes and stuff which go on. Yeah. All right. But Lenny's smooth. Lenny's smooth.
252
00:42:08.636 --> 00:42:12.556
[laughs] Thanks everyone for watching. We will catch you in the next episode.
