🔬 Synthesis Superintelligence: from Semiconductors to Superconductors — Periodic Labs’ Liam Fedus and Ekin Dogus Cubuk

WEBVTT
1
00:00:00.000 --> 00:00:12.920
[on-hold music] Okay, we're here at Periodic. We're here today with Liam and Dogus from Periodic. Welcome, and thanks for having us at your- Yeah, great to be here, and yeah, thanks for coming in.
2
00:00:13.200 --> 00:00:21.380
[laughs] Um, I wanna start off with one of these, like, quotes that I love from your website. It says, "Intelligence is necessary but not sufficient.
3
00:00:21.740 --> 00:00:28.880
New knowledge is created when ideas are found to be consistent with reality." It seems so straightforward, but why is it non-obvious?
4
00:00:29.540 --> 00:00:38.400
I think that's sort of the, the thesis and premise behind Periodic that Dogus and I thought when we were building this, which is you can't just think your way to a solution.
5
00:00:38.640 --> 00:00:50.040
Um, the universe is so complicated that in order to actually push the frontier of knowledge and to make progress, you need to create these conjectures and then actually see whether or not it holds.
6
00:00:50.620 --> 00:01:05.340
Um, no amount of rereading that textbook or paper and just thinking and, like, disappearing into a room is going to allow you to think through all possible, um, experimental outcomes, um, expected and unexpected, and you really need this, like, iterative process.
7
00:01:05.800 --> 00:01:17.720
And that's our core belief for putting this together, and that's why from the very beginning we're like, "We need to have the AI systems, the simulations of the physical world, but then also to build up the, the physical high throughput experiment."
8
00:01:18.240 --> 00:01:27.040
And we think a different type of intelligence emerges from that. I think it's also notable, like, with OpenAI and, and Google, you don't have the resources in those big labs.
9
00:01:27.080 --> 00:01:36.540
Like, you have to had-- you had to come out and do your own thing, um, because you, you're kind of inventing your own playbook as you go along. Yeah. I mean, a team like this has never existed, I think.
10
00:01:36.680 --> 00:01:53.360
The-- we try to bring together solid state chemists, solid state physicists, experimentalists, theorists, hardware engineers, LLM experts, computer scientists, um, and some of these technologies are very recent, like the high throughput experimentation, the robotic arms have only been tried in the last three years or so.
11
00:01:53.920 --> 00:02:00.480
The force field expertise, some of these force fields are also very recent. Um, but yeah, we felt like having a lab is very important.
12
00:02:00.540 --> 00:02:04.060
Having this focus and bringing all these people together to work together is very important.
13
00:02:04.880 --> 00:02:15.040
Um, and I guess, like, the closest examples from history are places like Bell Labs, where incredible theorists and experimentalists and chemists worked together and achieved incredible things.
14
00:02:15.520 --> 00:02:21.600
So we're trying to do the same here. Yeah. I'd like-- wanna talk about force fields in a bit. Yes. [laughs] But before we get to that [laughs] yeah, like, what are they?
15
00:02:21.960 --> 00:02:32.420
Um, but before we get to that, I think one of the, just for context, one of the biggest differences between, you know, some of our audience are, uh, AI, you know, engineers, and some of the audience is scientists.
16
00:02:32.920 --> 00:02:43.220
So let's go and talk to the engineers real quick and say, like, what is the difference if you've been working at OpenAI as a big lab, what's the difference between what you do there and what is, you know, what is here?
17
00:02:43.360 --> 00:02:48.880
I think you've already hinted at this- Mm... but maybe explicitly, what's, what's the-- how do you have to reframe your thought process?
18
00:02:49.340 --> 00:02:58.160
Well, I mean, I think, like, one interesting thing is now our reinforcement learning environments literally derive from the environment, from our physical labs.
19
00:02:58.700 --> 00:03:04.600
Uh, our data comes from our physical labs, and this is sort of our ultimate truth.
20
00:03:05.020 --> 00:03:14.660
It's not enough just to do optimization against, um, some answers that were known in some papers or textbooks, because we're going beyond that, and I think that's one of the biggest differences.
21
00:03:15.080 --> 00:03:23.780
But along with that, there's issues of, say, decision-making under uncertainty. So when you're doing optimization against math, there's a, a high precision to it.
22
00:03:23.820 --> 00:03:31.860
You're not really dealing with, you know, kinda like variance or uncertainties or, you know, aberrant measurements, whereas that's very key to, to our process.
23
00:03:31.920 --> 00:03:42.780
Like, um, for example, when we are doing materials discovery loop, things don't come off the, out of the furnace labeled, right? Even the labeling process can be stochastic and noisy.
24
00:03:43.320 --> 00:03:47.100
And, um, sometimes you'll have aberrations between different machines.
25
00:03:47.560 --> 00:03:56.080
Uh, maybe we'll have insufficient telemetry, so we thought we ran it at, um, this temperature, but really the temperature was, you know, some delta away.
26
00:03:56.660 --> 00:04:06.600
And, uh, an intelligence that can take in this noisy data and make intelligent decisions like a scientist would, um, is a sort of a different set of reasoning strategies.
27
00:04:07.100 --> 00:04:22.420
So I think there's a huge amount of commonalities, so the standard mid-training, reinforcement learning, the construction of, um, tool-using agents, uh, variance reduction, um, you know, making sure infra doesn't have mismatches between, like, training and inference.
28
00:04:22.900 --> 00:04:36.000
But we have to go beyond that and really think about how do you do accurate, um, work when there's a high amount of uncertainty? How do you make really incredibly efficient use of a limited set of data?
29
00:04:36.400 --> 00:04:46.380
So we're scaling things up significantly, but still, it's, it's very different than a digital environment where you can arbitrarily add more environments or more rollouts. We don't have that same capacity.
30
00:04:46.720 --> 00:04:48.500
So sample efficiency is another key thing.
31
00:04:48.800 --> 00:05:09.600
This seems a little bit abstract to me, and it is, it is a theme which has been on the science pods for a bit, but, uh, I, I think it might be helpful when you talk about experimental uncertainty, um, to, to explain, like, what would a specific process that you were doing experimentally look like, and, like, what would a human-- how would a human approach this problem before we jump into how do you go into the AI side?
32
00:05:10.040 --> 00:05:18.780
Do you have, like, a specific example of a type of material you make and, like, what sort of uncertainty in the measurements you would experience in the process? I think there are many dimensions to this.
33
00:05:18.840 --> 00:05:27.700
So one of them is, I think every scientific experiment has to do some dimensionality reduction, and this is very different than coding or math. So, um, when you're doing math or coding,
34
00:05:28.660 --> 00:05:38.460
all the context could be available to the human or the LLM. It's a bunch of axioms, some corollaries people have derived from them. It's a bunch of, like, functions and API calls.
35
00:05:38.720 --> 00:05:49.360
So everything you need to reason through is available to you, and then you just have to be very smart and, like, figure it out. In physics, as you know, we start with more atoms than we could ever store on a computer.
36
00:05:49.860 --> 00:05:58.240
So clearly we'll have to go from the original number of dimensions and bits that represent a system to the number of bits we can fit in the computer.
37
00:05:58.820 --> 00:06:02.060
And, you know, this was, I guess, the original premise of thermodynamics.
38
00:06:02.480 --> 00:06:11.640
They were originally really confused about how steam engines worked, but then turns out you can find like five or six thermodynamic variables that explain what's going on pretty well, which is incredible.
39
00:06:11.980 --> 00:06:19.568
Uh, that's the beauty of physics. Like, you have a room full of atoms, and there are You know, 10 to the 23 or 10 to the 27th- Exactly...
40
00:06:19.628 --> 00:06:30.168
or something atoms in this room, and yet you need temperature and pressure and, you know, simple variables, and that tells you everything. Well, it tells you mostly what you- A lot... need to know for most cases, right?
41
00:06:30.208 --> 00:06:39.808
Yeah. So, like let's say you make a material and you could do something like look at an X-ray structure, and this thing is a very lossy, um, uh, projection, right?
42
00:06:39.968 --> 00:06:46.548
It's like not actually a structure, or it doesn't tell you the true structure, it tells you some, you said principal components of that.
43
00:06:47.108 --> 00:06:57.948
So, uh, how would a human take these simple principal components and then, um, kind of reason about them, and then how do you extend that to an AI?
44
00:06:58.268 --> 00:07:05.848
So on the theoretical physics side, right, uh, we've decided that energy, the average value of energy, and the fluctuations of that energy is very important.
45
00:07:06.228 --> 00:07:13.028
So a lot of thermodynamics has been derived from this, and then we realized reaction barriers are also very important, the kinetics is very important.
46
00:07:13.568 --> 00:07:21.167
Um, so a human can tries to reason through this extremely complex system by coming up with some, you know, reduced dimensionality descriptors.
47
00:07:21.748 --> 00:07:32.348
Um, people also really like atomistic picture, like you think about what the local neighborhoods look like, whether they're from octahedra, tetrahedra, and that especially chemists really benefit from that.
48
00:07:32.908 --> 00:07:38.668
Um, okay, so that's on the theoretical side. And on the experimental side, you would try to collect as much data as possible.
49
00:07:38.748 --> 00:07:48.048
So you'd have a lab notebook, and you'd write everything you see, et cetera, but with the understanding, of course, that you'll miss a lot. Uh, so like as Liam mentioned, there are all these issues that come up.
50
00:07:48.068 --> 00:07:58.368
For example, your furnace will degrade over time because as you're using the furnace, some of the things you're baking will evaporate and cover the heating element.
51
00:07:58.448 --> 00:08:07.148
So every time you use the furnace, it gets worse and worse. Uh, another issue is like you have a furnace, but the temperature isn't perfectly uniform, so where you put in the furnace will affect the result.
52
00:08:07.648 --> 00:08:15.008
Uh, we, we don't have this issue yet, but we will maybe someday. The optical instruments, uh, suffer a lot from vibrations, so like every time somebody walks.
53
00:08:15.268 --> 00:08:26.808
Like I remember when I was doing my PhD, one lab had this big issue because sometimes their results would be very different than other times, and they brought it down to at a certain time in the night, upstairs somebody would be walking- [laughs]...
54
00:08:26.848 --> 00:08:32.948
and that vibration affected the laser setup. Um, so yeah, science is very hard, but that's what makes it special, right?
55
00:08:32.988 --> 00:08:44.808
Like, the other thing Liam and I have been talking about is a lot of the current improvements focus on math and coding and theoretical computer science because it's easier for LLMs, but in real life, most things that require intelligence are actually more like science.
56
00:08:45.248 --> 00:08:53.088
Like there's a lot of uncertainty, there's a lot of noise, um, there's a lot of missing context, but you have to be the intelligent being and figure out what to do next.
57
00:08:53.568 --> 00:09:04.608
And I think like a really interesting point to build on that too is the optimal reasoning strategies to do math, to do theoretical computer science, to do these types of things may not be the optimal reasoning strategies to do science.
58
00:09:04.668 --> 00:09:17.328
Yeah. Mm-hmm. How do you define a RL reward function when, uh, your output is a noisy crystal structure or, um, some number of, I don't know, like collective variables or something like?
59
00:09:17.768 --> 00:09:22.168
Maybe, um, I'll give, uh, one part of the loop, and I'll just kinda dig into that.
60
00:09:22.228 --> 00:09:33.968
So, uh, as part of the materials discovery loop, we have to first figure out, uh, what to make, so that's the stability of the atoms together, um, as well as do we expect it to have the properties we, we want.
61
00:09:34.348 --> 00:09:43.388
So we don't just want, uh, a, a novel configuration of the atoms, we want them when put together to have the property of interest. Mm-hmm. Next is how do you synthesize it? How do you actually make the thing?
62
00:09:43.888 --> 00:09:53.228
Um, that's also non-trivial. So even if you know something will hold together, the process of actually, you know, what are the processing conditions to actually bring that forth is highly non-trivial.
63
00:09:53.688 --> 00:10:02.188
But again, like the thing, once you've done those two steps and you've made something, it doesn't come off labeled, so you actually have to characterize it and figure out what you made. So in that case,
64
00:10:03.448 --> 00:10:15.828
y- rather than thinking about the RL environment of we're just going to kick off an experiment, wait a couple of days, and try to do like a update on like, did you find a room temperature superconductor or not, that's completely infeasible.
65
00:10:15.848 --> 00:10:25.068
Yeah. I was gonna ask that. Yeah, like you can't leave your GPU sitting for a week or something. It's, yeah, it's completely infeasible- Yeah... because you don't have enough agents to reduce the variance sufficiently.
66
00:10:25.388 --> 00:10:35.908
The rollout time is going between the different instruments. You're waiting. There's just some physics limits of the experiments, um, for like the actual like, um, synthesis or furnace time.
67
00:10:36.548 --> 00:10:46.688
Uh, so it's just too, too slow, too noisy. So the way we think about the AI program is basically constructing agents around this set of data that we've produced.
68
00:10:47.128 --> 00:11:01.528
And so in the case of characterization, you're looking for RL environments that are rewarding the identification of the phases actually present. Are you able to, um, given the raw experimental data, fit that effectively?
69
00:11:01.988 --> 00:11:05.368
So the way we do this is we'll shoot X-rays, um, at the material.
70
00:11:05.868 --> 00:11:16.368
Um, X-rays have a s- a frequency or a wavelength roughly measurable or commensurate to the spacing between atoms, so you get nice diffraction patterns and so that kind of give a fingerprint of the crystal structure.
71
00:11:16.948 --> 00:11:27.348
And doing this, um, identification of what actually is present is highly non-trivial, and so some of our early AI work has been on basically producing systems to do this.
72
00:11:27.948 --> 00:11:38.348
And this allows us to move much more quickly too because if you're running so many experiments, you really quickly become bottlenecked on your ability to understand what was produced.
73
00:11:38.888 --> 00:11:52.388
But again, now the reinforcement learning environment is fairly straightforward, which is you're gonna reward the identification of the phases present, um, you're going to penalize spurious phases or things that don't have any like real chemical plausibility.
74
00:11:52.828 --> 00:12:02.128
Uh, so that, that's like a, you know, a little succinct thing where you can kind of do this for the different pieces, and when you stitch this together, that's the end-to-end discovery loop. Okay.
75
00:12:02.208 --> 00:12:03.788
Y- you keep- And maybe, maybe like another- Yeah. Oh, sorry...
76
00:12:03.808 --> 00:12:13.808
another piece would be as we build up, uh, a large basket of experimental data, you can think of another thing of, um, basically like timestamping the state of the world.
77
00:12:14.208 --> 00:12:19.028
So you could say, at this date, this was our experimental evidence up until that point.
78
00:12:19.548 --> 00:12:34.128
And you can construct reinforcement learning environments where you say, "Okay, given that body of experimental evidence and given, you know, the choices, what, what was like the, the next choice that the scientist made, or what was an outcome of that experiment?"
79
00:12:34.688 --> 00:12:55.156
And This is really interesting too, because this allows us to do types of programs that would be harder externally, because if the pre-trained model has already memorized some of this data, and then you try to do reinforcement learning on that, if it already knows the answer and you construct reinforcement learning tasks that depend on that answer, it can fake work.
80
00:12:55.776 --> 00:13:04.956
So, and then, uh, I mean, because it, it's like, oh, well, it already knows the answer, so it doesn't actually have to do the hard physics reasoning. It doesn't have to actually do the, the calculation simulations.
81
00:13:04.996 --> 00:13:15.656
[laughs] And it gets to the right answer, and then you reinforce that policy, and you up-weight those reasoning strategies. Those reasoning strategies will not generalize to novel systems. So it's, it's not useful to us.
82
00:13:16.176 --> 00:13:26.776
But as we build up baskets of experimental data and, like, stitching things together, having the lineage through the whole process, that gives us the ability to construct new types of environments that are just not plausible elsewhere.
83
00:13:27.256 --> 00:13:35.856
I wanna ask my question- There's like-... but I, I'm worried that I'm taking it off the rails. I'm just gonna go for it because I, again, I, I come from the non-scientist but engineering sort of...
84
00:13:35.996 --> 00:13:38.496
I'm familiar with, on the ML side, but not the, not the physics side.
85
00:13:38.736 --> 00:13:47.296
Okay, there's, there's like, there's, there's a few versions of this, but I think the basic question is, um, don't we have, like, most of the laws of physics worked out?
86
00:13:47.936 --> 00:13:55.996
Uh, and like, how come w- we don't have perfect simulators already? [laughs] Oh, that's a great question. Like, like, like this is, like, very basic, like, and he was like, "Ha ha, that's cute."
87
00:13:56.396 --> 00:14:05.416
No, no, no, it's a great question. But like, I'm like, "Dude, like, like, like I have all these physics books. What do you mean, what do you mean we're not done?" [laughs] Yeah, so there- [laughs]
88
00:14:05.476 --> 00:14:11.176
There are a couple of things going on here, right? So- Yeah... I think first of all, we're definitely not done with any of the laws of physics.
89
00:14:11.176 --> 00:14:21.716
[laughs] Like e- even the- Like at quantum scale, okay, but, and like maybe like s- you know, very, very large scale, but like at human scale, at like, you know, material scale, we're, we're, we, like, what's, what's left?
90
00:14:22.036 --> 00:14:27.756
We're, we're not- [laughs] So it's a, it's a really good question. A- and it's very interesting why we're not done yet, right?
91
00:14:27.756 --> 00:14:40.796
[laughs] Um, so there's a very, um, common story people talk about, like when Dirac was figuring out quantum mechanics, like late 1920s, he wrote a textbook about quantum mechanics, and he kind of framed it as it's all done.
92
00:14:41.336 --> 00:14:47.416
And then there's a phrase people like to quote, I, I'm not sure actually how accurate it is, but apparently Dirac said, "The rest is chemistry."
93
00:14:48.116 --> 00:14:51.376
The idea being like he could solve the hydrogen atom- Just compose everything. Mm-hmm. Yeah. Yeah.
94
00:14:51.436 --> 00:15:00.856
He could maybe solve like a chain of 1D hydrogen atoms, but then he cannot currently solve, say, nitrogen interacting with oxygen, but that's okay, that's just chemistry. [laughs] Yeah.
95
00:15:00.916 --> 00:15:08.816
Um- Physics real- physicists really love single, like, atom rep, like- Yeah... really simple systems. Simple systems. And that's all you can solve. Spherical cow. Yeah, yeah. Spherical cow.
96
00:15:08.856 --> 00:15:12.676
But it turns out, I think what we learned in the last hundred years is, first of all, that's not true.
97
00:15:13.076 --> 00:15:20.756
Um, it's not like a simple extension of what it was to do just hydrogen, and it's not just, the rest is not just chemistry. There's actually a lot of physics there.
98
00:15:21.096 --> 00:15:28.656
I mean, I guess one of the things we still haven't figured out is high temperature superconductivity, but there are a lot of other- And high for you is like one seventy-five- Oh... two hundred.
99
00:15:28.696 --> 00:15:37.196
When people say high temperature superconductivity, what they mean there is unconventional superconductivity. So there's the conventional superconductivity that's just mainly driven by electron-phonon coupling. Mm-hmm.
100
00:15:37.456 --> 00:15:47.196
And in that case, you can kind of see the isotope effect, like if you take the same system but just different weight for one of the elements, you can see the superconducting temperature drops at the rate you expect.
101
00:15:47.576 --> 00:15:55.796
So that one is conventional. But then higher temperature superconductors like cuprates, like the ones that are above seventy-seven Kelvin, like ninety-three Kelvin, turns out don't obey that physics.
102
00:15:56.176 --> 00:16:01.336
But we don't know what physics they obey. [laughs] Um, they're just incredible superconductors. Yeah. Um, but it's not just that.
103
00:16:01.376 --> 00:16:07.716
So that definitely is one of these, like, very, uh, popular topics we don't know the theory yet, for yet. But there's so many other things we don't understand.
104
00:16:07.756 --> 00:16:16.836
Like for example, even strong correlation in simple quantum mechanical systems we cannot simulate yet. Um, density functional theory is a very powerful tool.
105
00:16:17.216 --> 00:16:26.056
It's incredibly accurate on some things, but as long it, as soon as there's like some strong electron correlation, it actually fails to capture some of the effects. I mean, there's so much to figure out.
106
00:16:26.096 --> 00:16:32.916
That's why I'm really excited for AI to be applied to this field, because theoretically, computationally and experimentally, there's so much to figure out.
107
00:16:32.976 --> 00:16:41.556
And every improvement we make here should improve human life, because, you know, the better we can understand materials, solid state physics, the better devices we can make for them.
108
00:16:41.996 --> 00:16:57.356
There's this m- uh, famous quote by Phil Anderson, which is, "More is different," which is basically you may understand all the basic laws of something, but when you add many things, they behave qualitatively distinct differently from how the simple, like, physics should tell you.
109
00:16:57.456 --> 00:17:07.356
So understanding how lots of things behave together, it seems like it should be simple. Like, beginning laws are simple, but the collective behavior is so complicated- Yeah... that it's really hard to model.
110
00:17:07.396 --> 00:17:15.156
It's like emergent and universal- Yeah... which is crazy, and we're also seeing this with deep learning models, right? Like we see power laws everywhere in deep learning.
111
00:17:15.196 --> 00:17:24.736
We don't understand, but it, like it's very reminiscent of physics, where there's, uh, emergence and universality. Um- There's been many theory papers in the physics world about emergence in neural networks.
112
00:17:24.736 --> 00:17:34.676
[laughs] Yeah, absolutely, yes. Yeah. Uh, and then my, my other follow-up question was also just on the process of, um, let's call it the, the pro- uh, yeah, process and the end result.
113
00:17:34.716 --> 00:17:41.236
Like let's, let's call it like y- you want to have these properties that you're targeting for materials, and then you have to figure out the process to, to get there.
114
00:17:41.596 --> 00:17:50.776
Is it worth se- separating these things so that you can have, like you just train a model that perfectly reverse engineers any process whatsoever, given, uh, whatever theoretical end state you want?
115
00:17:51.396 --> 00:18:01.016
Uh, is that, uh, meaningful? Maybe going back to the conversation we had about how since there are more than Avogadro's number of atoms, we'll never know the exact context of what happened.
116
00:18:01.496 --> 00:18:12.676
I think there might be a reason why we'll never be able to reverse engineer everything perfectly, but we're just trying to reverse engineer sufficiently to be able to improve the performance of the materials, basically.
117
00:18:13.036 --> 00:18:19.996
Yeah. What you're asking is, is there a forward model where you put in an input and you can predict the output, and you want to underst- Yeah...
118
00:18:20.056 --> 00:18:26.356
you wanna create a, like a surrogate model which can understand- Like the input is, like the, the final state of that, th- this configuration of atoms- Right...
119
00:18:26.396 --> 00:18:35.156
and then the prediction is- Because then you can, you can split up the work, almost. Like you can have one team do the process side, and then the other team do the property side- Yep... and then, you know, just [laughs]
120
00:18:35.216 --> 00:18:44.196
raise them. [laughs] Yeah, you know, I think that there's, uh, a lot of progress we can make in terms of these, like, you know, th- splitting up these agentic workflows to these different areas.
121
00:18:44.636 --> 00:18:54.408
Uh, I think there's a lot of, like, empirical work we can make, uh, on, on synthesis prediction as well. Yeah. Um, whereas like, you know, taking in that, that configuration of atoms, what tools can you use?
122
00:18:54.448 --> 00:19:04.048
How, given, given this experimental evidence, given prior literature, prior papers, how do you actually get to these things? And you ultimately are testing, you know, are you able to actually replicate it or not?
123
00:19:04.668 --> 00:19:23.468
The, the reason I say this maybe also is just thinking about it in terms of a company, um, you know, i- and the way that TSMC is kind of like a fab for semis, but they don't design the semis, um, you know, you could be the TSMC of materials, where, like, people come to you with like, "Well, this here's what I want," and then, like, you can figure out the process and- Yeah, it's like a matter compiler.
124
00:19:23.508 --> 00:19:32.608
Yeah. [laughs] So, like, it's like given these, these requirements- Very Star Trek. Yeah, yeah, yeah. But it's like, given these requirements, is this actually, like, a, a valid set of configurations? Yeah.
125
00:19:32.648 --> 00:19:37.408
Can this, can this exist? Is this a good objective function? I don't know. It's too, too big. [laughs] Yeah.
126
00:19:38.268 --> 00:19:44.208
Well, I mean, we kind of talk about, like, one of our missions as, uh, like synthesis super intelligence, so I think it's in line with this. Yeah.
127
00:19:44.508 --> 00:19:57.988
I, I wanna double-click on something that you said a little, a little while back, which was you talked about phases, and I think this is, you know, going to, um, uh, Doja's comment about, uh, what is, uh, or, like, emergence.
128
00:19:58.028 --> 00:20:01.708
Like, the, the concept of a phase transition I think might be foreign to a lot of the audience.
129
00:20:01.768 --> 00:20:09.708
So can you explain what is a phase transition, and then why is this, like, a helpful signal for something like an RL environment? Phase transitions are fascinating.
130
00:20:10.148 --> 00:20:17.908
Uh, I mean, I think the, maybe the one that people can most easily relate to in their life is, uh, ice melting probably or water boiling.
131
00:20:18.408 --> 00:20:30.908
Like, you know, you increase the temperature of ice, and it still looks like ice, and it behaves like ice, but there's a certain temperature at which it just stops raising its temperature and then turns into liquid.
132
00:20:30.988 --> 00:20:37.608
So it suddenly goes from this solid phase to liquid phase. Um, another one that's very relevant to us, of course, is the Ising model.
133
00:20:38.148 --> 00:20:48.848
Uh, and I think computer science and mathematicians also study this maybe under different names, but basically, uh, you can have spins up and down, and they have different interaction terms if they're both up versus one is up, one is down.
134
00:20:49.388 --> 00:21:00.468
And at high enough temperature, they're usually just randomly up or down. Entropy wins. You lower the temperature, it's still looking the same. But then at some point, uh, suddenly they perfectly start aligning.
135
00:21:00.508 --> 00:21:11.048
Yeah, the phase transition. Phase transitions are really nice because it m-makes physicists' life easier for studying certain things. They tend to have certain, um, spatial correlations that really help us, um, as well.
136
00:21:11.468 --> 00:21:26.908
Okay, so in our lab, uh, phase transitions come about usually because we mix the precursors that are different, um, crystals basically, but then we r-raise the temperature or do something else to encourage them to react, and then atoms start reacting, and they form into this new crystal.
137
00:21:27.568 --> 00:21:35.228
Um, and the, this shows up in the XRD pattern because usually if the geometry of the atoms change drastically, the XRD pattern changes a lot.
138
00:21:35.708 --> 00:21:46.948
But what really happens in, like, a typical practical materials discovery campaign is when you first try something, it doesn't work, and the outcome isn't just, like, a clear yes or no, but it's usually, like, a very mixed phase.
139
00:21:47.488 --> 00:21:54.328
Uh, it will usually have some of the precursors. It might have some amorphous phase, and then it will have, like, a bunch of phases that you maybe didn't try to make.
140
00:21:54.788 --> 00:22:01.708
So that becomes a challenge, and that's why we find it very difficult to bring together simulations and AI to do the characterization.
141
00:22:02.088 --> 00:22:11.308
So the simulations can always say, "Oh, this phase doesn't look like what we predicted, but it's like a slight varia-variation of it. Let me now do force field calculation to see if it's still stable."
142
00:22:11.728 --> 00:22:18.388
Or like the AI can say, "Based on the previous experiment and this one, this is probably not the phase we wanna make, so let's change the conditions."
143
00:22:18.848 --> 00:22:26.788
Yeah, for, for listeners, XRD is X-ray diffraction, which you already described. Yeah, in some cases, there's, like, a, a phase that just hasn't been recorded before.
144
00:22:27.028 --> 00:22:37.048
It doesn't exist in any papers or database, and so the AI has to actually go and sort of put, use these tools to say, "Well, what configuration atoms would actually explain these, these phases?"
145
00:22:37.388 --> 00:22:48.308
But this becomes incredibly important for directing the scientific process because, uh, let's say we have a phase target in mind, we really need to sort of hill climb that to get better measurements on that.
146
00:22:48.548 --> 00:22:56.868
So if it's, uh, one percent and we're like, "Actually, we wanna increase the space period," we want to have a very good measurement on this experimental data to say, you know, what is present and what's not.
147
00:22:57.348 --> 00:23:02.848
So one of the dis-advantages is you have just a distinct, like, variable that you can just observe.
148
00:23:02.968 --> 00:23:13.208
Like, going back to the statement about how do you deal with uncertainty, one of your, it seems like part of your answer is you make your observable signature somewhat unambiguous. There's no interpretation there.
149
00:23:13.268 --> 00:23:19.808
Is that right? Is that, like, the logic behind this, or is this, like, literally you just need to define a new phase, and so this is the thing you care about?
150
00:23:20.208 --> 00:23:28.128
I mean, I think there's still ambiguity in a lot of these cases. Yeah. I mean, like, you know, a, a very dumb thing is, like, replicates. [laughs] Yeah. We run replicates, of course. Yeah, yeah. Yeah, yeah.
151
00:23:28.168 --> 00:23:37.768
But no, I mean, I think there, you know, it's a challenging task because it's, it's not just, uh, fully deterministic and, and there are, like, well, there, there are, like, ambiguities, um, and other things.
152
00:23:38.688 --> 00:23:42.308
Two different, uh, phases can actually be consistent with the same pattern. Yeah.
153
00:23:42.348 --> 00:23:55.288
And so I think that's why it's really important for the system to use some chemical intuition to say, like, "Okay, well, this would be highly unlikely given the synthesis conditions, um, given this, the prior knowledge."
154
00:23:55.788 --> 00:24:07.028
Um, and so that can be used to help disambiguate. So you're, so you're somewhat injecting priors there, um, based on what you expect. Yeah. I mean, I, I think thermodynamics is the biggest prior, right? Right. Ages.
155
00:24:07.048 --> 00:24:13.268
[laughs] Right. And then physics is a big prior. Another one that we really benefit from here is multimodality or materials characterization.
156
00:24:13.628 --> 00:24:21.068
So, like, we can do XRD and yes, some phases might look similar on the XRD pattern, but then we can also measure their electrical properties, magnetic properties.
157
00:24:21.108 --> 00:24:28.528
We can measure their morphology, like some kind of electron microscopy, and then some of the phases that look similar on XRD will look different in some of these dimensions. Yeah.
158
00:24:28.568 --> 00:24:35.348
And that, that kind of multimodality really helps. And again, AI is really helpful here because, like, humans have limited context and computation power as well.
159
00:24:35.448 --> 00:24:46.128
So if you give humans like 10 different modalities at the same time and say, "Analyze these all consistently," it's a bit difficult, but it's actually not super intelligent for an AI to do it. It's great, yeah.
160
00:24:46.328 --> 00:24:49.848
Yeah, it's like the super intelligence is it can just do so much calculations.
161
00:24:49.888 --> 00:25:08.128
It can look at everything, and this is kinda gets to that, um, phrase about, like, deci- better decision-making under uncertainty, where now it's stitching together, um, the signal across many different instruments longitudinally across different experiments, across the replicates, and then kinda getting to this, like, underlying better model.
162
00:25:08.468 --> 00:25:12.268
Yeah. Rather than just over-indexing on one measurement from one instrument.
163
00:25:12.864 --> 00:25:20.704
And, and let me just like, um, so y- you know, you started, uh, I think a, a while ago also saying that there's just more data than you can fit in any, any reasonable computer or a process or anything.
164
00:25:21.104 --> 00:25:28.084
So at some point you have to throw out data, even though you have all these replicates, even though you have like, presumably like 10 different ways to measure the same thing.
165
00:25:28.344 --> 00:25:36.544
Because what if one of your temperature things is wrong? Um, so like when do you throw that out? How do you decide what to do? As a data-hungry ML guy, I just want everything, right?
166
00:25:36.604 --> 00:25:44.564
And then I just, you know- So it's, yeah... model, [laughs] learn everything. I, I don't know what I don't know, and then just, just throw, throw the machine at it. Yeah, so we don't throw in data. Um- [laughs]...
167
00:25:44.703 --> 00:25:53.504
I, I think maybe what I meant was potentially, like i- if God was observing the experiment- Yeah... there is more data in the experiment than humans can fit in a computer.
168
00:25:53.984 --> 00:26:05.484
Unfortunately, us mortals [laughs] cannot observe all the atoms. So even though there's more data than we could store, we actually can't access that data. For example, I cannot trace all the atoms' positions.
169
00:26:06.024 --> 00:26:11.464
Um, so unfortunately we're still in a regime where any data we collect is very precious, and we don't delete any of it. Yeah.
170
00:26:11.504 --> 00:26:18.624
Maybe a way of thinking of it to connect to the AI side is like, um, think about probing, like linear probing of a, of a foundation model, right?
171
00:26:18.984 --> 00:26:30.604
This model has all of this data, so God has this entire picture of what the material is doing, and then the experiment is this little tiny linear probe, which has like eight- Mm... eight bits or something of output.
172
00:26:30.964 --> 00:26:35.544
And so you're taking this humongous thing and coarse graining it down to a little bit of information.
173
00:26:35.584 --> 00:26:45.764
And then it's now, uh, either traditionally human's job or now your agent's job to reconstruct from those eight bits of information- It's, it's been analog, yeah... what is actually happening inside.
174
00:26:45.964 --> 00:26:54.364
Like it's an analog where you can't have full visibility over every parameter way, over every activation for that input. Um, yeah, you're, you're getting like some sort of subset- Yeah...
175
00:26:54.384 --> 00:26:58.884
on it or some like, uh, you know, coarse-grained features across the whole thing. Yeah.
176
00:26:59.204 --> 00:27:09.304
But so you, you'd probably find that, you probably, uh, do find, uh, Maxwell's demon argument very interesting because th- that kind of- Can you, can you re- recite it? Because I, I don't... Uh, he's mentioned it before.
177
00:27:09.324 --> 00:27:21.144
[laughs] I keep forgetting it. Like, so you know, i- in the 1800s, uh, people realized that entropy of a system has to increase or like a, for a, a closed system- Yeah... or stay the same, but it can't go down- Yeah...
178
00:27:21.324 --> 00:27:22.704
as the second law of thermodynamics.
179
00:27:23.164 --> 00:27:42.204
And there was a, a thought experiment that was asking what would happen if there was a very tiny, all-knowing demon that could kind of look at atoms, and then whenever an atom had low energy, kind of open a door and let it into this other room, and keep doing this to reduce entropy of the system, which would break second law of thermodynamics.
180
00:27:42.664 --> 00:28:01.964
And for a long time, I think there wasn't like a very satisfying answer for why that was the case, but then there was a more recent paper in the 1900s, um, more recently to us, uh, Landau came up with the argument that you'd have to spend energy to delete information, and Maxwell's demon would basically have access to so much information by doing this thing he's doing, that he'd have to delete some.
181
00:28:02.344 --> 00:28:07.264
And for, when he's deleting information, he'd have to spend energy, which would then make sure entropy increases.
182
00:28:07.564 --> 00:28:16.624
So even though we're not at the level where we can be Maxwell's demon, I think your, uh, question is like very visionary towards like a future where we store so much data we have to delete some.
183
00:28:17.204 --> 00:28:24.304
Yeah, no, I, I, I wasn't, I wasn't obviously going to that- [laughs]... to that extent. Uh, it does remind me of the air conditioning. That's kind of how air conditioning works.
184
00:28:24.544 --> 00:28:34.104
But, uh, also I think it's just like, uh, a, a question of like, why don't you just buy every sensor in the world and just like, you know, over- ridiculously over-instrument, over-replicate everything. Oh, we do that.
185
00:28:34.144 --> 00:28:40.224
Right? Okay, yeah. Okay, there we go. Yes. Yeah. Okay, anyways. Right. All right. [laughs] Yeah, I think, I think it's a, a matter of like- Confirmed. Yeah, exactly.
186
00:28:40.284 --> 00:28:49.724
Are you getting visibility to individual atoms, which is infeasible? But no, we add a huge amount of telemetry, and that basically reduces the number of like hidden variables, if you will. Yeah.
187
00:28:49.744 --> 00:28:59.804
Um- Yeah, and just, just out of curiosity, it is like, do you worry that you're just in like California? Like, don't you want to do this in like Nepal and like [laughs] Australia? Yeah, we, we could.
188
00:28:59.864 --> 00:29:09.744
I mean, we, we, we have plans to, you know, open more labs. Because obviously gravity and like, you know, where you face and where, like where you are matters. Also, like expertise can change, right?
189
00:29:09.784 --> 00:29:11.924
Like here we have certain kinds of expertise.
190
00:29:11.944 --> 00:29:20.604
For example, Stanford physics department, engineering department has certain kinds of expertise that really helps us because we can hire from there, our researchers can collaborate.
191
00:29:20.644 --> 00:29:29.284
Like, as you said, like geography can matter, but then different cities can have some different sets of expertise within physics, chemistry. Yeah. Oh, oh, I mean, we have Zoom- Yeah... for that. Yeah.
192
00:29:29.304 --> 00:29:37.764
[laughs] But still- It looks like your experiments are gonna be affected by like where you are. [laughs] Yeah. Yeah. I mean, the future of Periodic is, is not a single lab.
193
00:29:38.104 --> 00:29:44.384
I mean, I think- We need to have- I think you gotta hit up like your buddy Jeff- Yeah, a huge-... and like go to space and like- Right. Right... you know. Yeah, I mean, I think it's, it's necessary- [laughs]...
194
00:29:44.524 --> 00:29:53.884
to have this sort of network of labs and, and like, yeah, I think as Der's just kind of pointing out, like there's an optimal place for each of these based on expertise or, you know, materials constraint.
195
00:29:53.924 --> 00:30:02.164
There's many different constraints permitting. Um, okay, so one of them is, uh, so one thing I- I've been curious about is how do you deal with computational tools?
196
00:30:02.264 --> 00:30:12.424
Or like how do you integrate computational tools into a decision-making workflow, um, especially when, let's say, computation and, uh, uh, experiment don't necessarily, you know, meet.
197
00:30:12.804 --> 00:30:22.364
Um, it's also like the- theory in terms of like high level, like understanding and, uh, yeah, how does that sort of work together? Yeah.
198
00:30:22.424 --> 00:30:29.664
So I mean, one thing that's really helpful here is there are certain things that are easier to do in simulation and more accurate in simulation.
199
00:30:29.704 --> 00:30:38.064
There are certain things that are easier to do in experiment and more accurate in experiment. I mean, in general, of course, experiments are more accurate, but certain measurements are very hard to make with experiment.
200
00:30:38.324 --> 00:30:50.504
So you make a bad version of it, so you get inaccurate data. Um, like, let me just give you some examples. So one of them is, um, we use density functional theory to estimate formation enthalpy of, uh, materials.
201
00:30:50.904 --> 00:30:57.244
And- Sorry, quick pause. Uh, can you explain densital func- density functional theory and formation entropy? Yeah, of course. Sorry, enta- enthalpy. Yeah. Yeah.
202
00:30:57.284 --> 00:31:08.384
So, uh, density functional theory is probably the most commonly used simulation method for materials. Um, it kind of comes from Kohn-Hohenberg theorem, and Kohn got a Nobel Prize for this.
203
00:31:08.964 --> 00:31:20.344
He, when he realized that, sure, quantum mechanics is very expensive because you basically have this exponential Hilbert space for every wave function, so you have to solve this, like, very difficult problem, especially as the system size grows.
204
00:31:20.464 --> 00:31:32.364
Um- Kohn and Hohenberg realized that turns out you don't need to deal with that exponential space, all you have to do is think about the charge density, at least for the ground state of quantum mechanical systems.
205
00:31:32.664 --> 00:31:37.884
So this is, it's actually the, the, the theorem is pretty simple. So even, I think, an undergraduate in physics can understand.
206
00:31:38.524 --> 00:31:46.624
Um, but the idea is all the properties of a quantum mechanical system ground state is a functional of the charge density.
207
00:31:46.964 --> 00:31:54.264
Which is amazing because charge density is just this three-dimensional object, whereas the Hilbert space and wave functions are exponential.
208
00:31:54.804 --> 00:32:05.424
Um, so that was a really interesting observation, and then Kohn published another paper, I think with his postdoc, uh, Sham, Kohn-Sham, uh, wave functions, kind of had a practical way to solving
209
00:32:06.524 --> 00:32:13.364
some approximation of quantum mechanical properties of materials. And today we use it very much, both in our lab and externally.
210
00:32:13.944 --> 00:32:25.144
And then formation enthalpy is one of the measurements we can make to understand the energy of the material. Like, basically we, we want to ask, um, what is the energy that this material has?
211
00:32:25.444 --> 00:32:33.004
Because if that energy is higher than other materials these atoms can come into, then it's unlikely that this material will be able to be made.
212
00:32:33.064 --> 00:32:44.724
So stable just means like it's on the convex wall of other materials, uh, and their energies. Um, was, was that clear enough, or should we- I mean, may- maybe a simple, like, high-level point.
213
00:32:44.804 --> 00:32:52.044
Uh, so, so, like, density functional theory is a way of taking something which is exponentially hard, uh, in number of electrons- Right...
214
00:32:52.084 --> 00:33:04.964
or number, you know, in your system size, and then reducing it approximately with some introduced error to, like, maybe s- usually an N cubed approximation. Um, so now you can compute quantities which are, you know...
215
00:33:05.624 --> 00:33:14.644
You can now compute things which are just uncomputable, but at some cost. Right? If you could do it perfectly, you would-- we wouldn't need the lab, but, you know, we can't.
216
00:33:15.024 --> 00:33:27.624
This may be a good call back to our, actually our second episode ever, Heather Kulik had said, um, uh, she famously said, "There is no AlphaFold for materials," not just computationally, but the ground truth just doesn't exist.
217
00:33:27.684 --> 00:33:37.224
Like, we don't really know what material crystal structures look like, and DFT is generally our best route towards it. Yeah, I think so. I mean, maybe two things I would add is
218
00:33:38.184 --> 00:33:43.584
DFT by definition doesn't have to be an approximation. The Kohn-Hohenberg theorem shows that- It doesn't... it can be exact. Yeah. Mm-hmm.
219
00:33:43.604 --> 00:33:51.904
But I think as you're pointing out, the exchange correlation functional, we don't yet have access to. And then for the, uh, charge density based models, we don't know what the kinetic energy functional is.
220
00:33:52.404 --> 00:34:05.784
And then even if we had perfect DFT, I still think we'd need a lab because we-- even if we have perfect DFT, we cannot fit ten to the twenty-three atoms in the computer. Yeah. Um. You're very obsessed with that number.
221
00:34:05.824 --> 00:34:17.444
[laughs] But, but, but, no, okay, okay. Yeah, but, but with exponentially growing compute, you can argue that in N cubed scaling size, eventually you could in principle calculate things. You just wait till- Oh, yeah...
222
00:34:17.484 --> 00:34:24.364
long enough. Massive computers, yeah. But, yeah. But as long... Yeah. Waiting for a while. Yeah, yeah. But as long as N cubed is a good scaling, you can, you can- Yeah.
223
00:34:24.424 --> 00:34:31.284
Uh, as the non-scientist, I, I will make an observation, which I, I don't know if you wanna throw around or, or anything like that. Um, so I, I came from finance.
224
00:34:31.364 --> 00:34:39.684
Uh, we had the Gaussian copula, which was a way of pri- pricing like, uh, credit default swaps, uh, by doing correlations and reducing everything into a single kernel.
225
00:34:40.204 --> 00:34:48.324
Uh, and, and it's very-- it feels also spiritually similar to the VAE, um, in terms of like, like condensing all these things into like a, a single sort of parameter.
226
00:34:48.764 --> 00:34:51.264
Uh, I wonder if just, like, there's, like, the same trick everywhere.
227
00:34:51.544 --> 00:35:01.964
Yours sounds, uh, you know, uh, a bit more high dimensional than mine, uh, [laughs] which, you know, like VAE is just like, you know, one sigma, one sigma thing, but, like, uh, sounds the same. [laughs] Yeah.
228
00:35:02.004 --> 00:35:09.884
Yes, the charge density file is still quite a big file. Yeah. [laughs] Right. You hand wave a bit, but, like, it's good enough. Like, this is, this is the highest order bit. Like, yeah.
229
00:35:09.984 --> 00:35:19.824
[laughs] I, I, I personally don't know of a better method to predict the stability of a new material. It's definitely not perfect, uh, but it's definitely better than other methods I can think of.
230
00:35:20.364 --> 00:35:29.324
Um, and yeah, so going back to your question, the reason we use them is for certain things, it's easier to simulate than experiment, and they kind of complement each other well.
231
00:35:29.884 --> 00:35:40.843
Uh, and then we can, like, do this in a loop. Like, we feel like simulations will never be enough by themselves, but in the loop of simulations, AI, and experiments, I think we can make progress much faster than before.
232
00:35:41.484 --> 00:35:52.104
One thing that simulations let you do is scale up much more quickly. I mean, because now, like, compute is much easier to bring online- Yeah... than bringing in, bringing on new labs. How do you avoid...
233
00:35:52.484 --> 00:36:02.504
Or let's say you scale up a bunch of DFT, how do you avoid in having your models over-index on that and, and, and still be grounded that, like, that the, the real world is ultimately the truth you care about?
234
00:36:03.404 --> 00:36:07.944
If you, you know, how do you... What would a- You scale up the lab as well. Yeah, you scale up the lab. [laughs] Right. Yes.
235
00:36:07.944 --> 00:36:19.024
But, but I mean, you could, I could imagine scaling up DFT to millions, hundreds of millions given modern compute, whereas your lab, I would imagine, you're hundreds, thousands a day? I don't know.
236
00:36:19.064 --> 00:36:27.684
Like, something still, like, fairly... Well, even in a really high throughput case, like, you know, you're still fairly lim- limited by comparison, right? Yeah.
237
00:36:27.744 --> 00:36:39.684
I mean, one thing is the, the humans, but also the LLMs are kind of pretty aware of the limitations of DFT. Mm-hmm. For example, even if we can do, as you said, a hundred million trials with DFT- Yeah...
238
00:36:39.744 --> 00:36:41.844
we could never actually try their microstructure.
239
00:36:42.304 --> 00:36:54.364
And microstructure is this idea that, like, DFT usually simulates the perfect crystal, but crystals usually actually aren't perfect, and their microstructure, which is the structure at, like, a medium order, can really affect the properties.
240
00:36:54.864 --> 00:37:04.684
Other issues, of course, like some properties like superconducting temperature cannot be easily simulated by DFT. Again, like, even if you do a hundred million trials with DFT, it won't be enough.
241
00:37:05.024 --> 00:37:16.784
So we have to rely on heuristics, experiments, et cetera. And so there's a huge filter- Mm-hmm... from all those calculations to actually what gets executed in the lab. Is that human mediated or LM or a mix?
242
00:37:17.284 --> 00:37:26.028
Can be a mix. Yeah. Like, for example, for a long time Materials Project, which was the best open source DFT database, it- Use experimental calibration for DFT.
243
00:37:26.508 --> 00:37:31.148
So they would actually use experimental data to calibrate DFT. Uh, we also do this internally here.
244
00:37:31.208 --> 00:37:38.588
Like, whenever there's a certain chemical system we're interested in, you get experiments, you get simulations, and you can calibrate them. This is another thing.
245
00:37:38.668 --> 00:37:48.528
Um, I've talked to a lot of biologist friends, and they are always very confused when you say materials, you can't just do an XRD structure and actually know what the structure is.
246
00:37:48.868 --> 00:37:58.048
Because if you're in the biology world, you can look at a crystal structure of a protein, and typically more or less with, like, sub-a- well, like, order Angstrom accuracy reconstructed.
247
00:37:58.688 --> 00:38:12.088
What is the difference between the materials XRD, um, you know, X-ray dif- for, uh, X-ray dif- diff- no, um- Diffraction... diffraction, sorry, experimental X- XRD for materials versus, let's say, the biology world.
248
00:38:12.308 --> 00:38:19.568
Like, what information do you lose, and why is this sort of like a lossy projection? It- it- it's hard not to sound, like, crazy when answering this question.
249
00:38:19.568 --> 00:38:33.828
[laughs] But do you not feel like, um, organic chemistry and biology almost stem from a lower VC dimension, like, lower Kolmogorov complexity model, where, like, you can almost represent as a 1D sequence, and it can almost be compiled.
250
00:38:33.868 --> 00:38:44.368
[laughs] So, like, there's a sense in which it's, uh, simpler in the Kolmogorov complexity sense. 1D? I mean, they're like just DNA is, like, just, uh- Is 1D, yeah. [laughs] Yeah.
251
00:38:44.368 --> 00:38:50.888
Um, or RNA or- I think most people would argue it's more 2D, but- Okay. Yeah. [laughs] I'm, I'm not a biologist. Yeah, yeah. [laughs] So, you know, I- I'm sure they're right.
252
00:38:51.208 --> 00:39:06.088
But, uh, with inorganic chemistry, virtually it's crazy. It's clearly not coming from such a low Kolmogorov complexity model because the 3D inorganic crystals can host metals, insulators, superconductors, diamond.
253
00:39:06.128 --> 00:39:13.868
It's, like, really crazy different. Like, people tried, but we could never find a 1D representation, right? It's very hard to find a smile string for 3D org- inorganic crystals.
254
00:39:14.468 --> 00:39:23.968
And the, the way the atoms interact with each other is quite strange, like covalent bonds, ionic bonds, metallic bonds. Um, but yeah, this is, it's very interesting.
255
00:39:24.068 --> 00:39:32.728
Um, I, I, I wonder sometimes, like, you know, have you ever looked into issues with solid electrolytes, like why people cannot replace liquid electrolytes in batteries with solids?
256
00:39:33.448 --> 00:39:34.928
Uh, there are all these dendrites that form.
257
00:39:34.968 --> 00:39:43.948
Like, basically the lithium will form these structures that go into a solid electrolyte and crack it, and you look at that and you think this is so primitive compared to biology.
258
00:39:44.008 --> 00:39:55.728
Like, in biology, these systems can do such complex nano technolog- technological progress, whereas we can't even, like, make two interfaces work with each other because lithium will break it.
259
00:39:56.268 --> 00:40:05.048
Um, yeah, but then I guess inorganic stuff can be very resistant to high temperature. You can make space shuttles, you can make silicon compute Moore's law. So yeah, it's a trade-off.
260
00:40:05.068 --> 00:40:08.648
[laughs] Yeah, we've, we've talked about this with several other guests.
261
00:40:08.748 --> 00:40:19.308
I think one of the, the big differences in my mind is that biology, we have a toolkit that, you know, that you can basically borrow, like, from millions of years of evolution- Yeah...
262
00:40:19.648 --> 00:40:29.768
uh, that has already given us all the tools, and we can just reproduce those. But also, evolution also constrains the, yeah, the VC dimension to be fairly low for, like, proteins and so on.
263
00:40:30.328 --> 00:40:38.328
The, like, emergent beha- behavior of large scale systems like cells can be much more complicated. Yeah. And consciousness. Yeah, yeah, yeah.
264
00:40:38.488 --> 00:40:50.008
Yeah, but maybe even just, like, more specifically for, um, like, if you just look at an X-ray, you had a perfect crystal and you shoot an XRD at it, and you get the, you know, you get the spectrum,
265
00:40:50.908 --> 00:40:59.628
that still doesn't actually tell you uniquely what the crystal structure is, right? No. So, and that's something that- What? It's just average. Yeah. It's just average behavior. Oh, yeah.
266
00:40:59.648 --> 00:41:02.508
But like for example, disproportionation is a very complicated thing, right?
267
00:41:02.548 --> 00:41:13.488
Where it's possible that you're the perfect crystal and on average it looks like it's perfect XRD, but in reality there is some other pattern going on where, like, atoms slightly shift to the right or left, but on average they're in the middle.
268
00:41:14.188 --> 00:41:24.248
Um, yeah, it, it's a difficult life. Yeah. [laughs] Whereas with proteins you basically know there's some sort of underlying protein which follows some rules, so you can, you can combine a forward model with it. Right.
269
00:41:24.288 --> 00:41:28.748
And then you can kind of move, and move towards like, you know, single crystal XRD and that, that can help.
270
00:41:28.948 --> 00:41:35.228
When, when we started The Science Pod, there were all these, like, elements of, of sci- AI in science, AI in math, AI in physics, and all these things.
271
00:41:35.248 --> 00:41:41.748
And we were, there was this, like, kind of pissing contest, like which is harder. And like, I think, like, you can make the scientific argument that material science is the hardest.
272
00:41:41.848 --> 00:41:49.008
[laughs] Well, we didn't pick it necessarily for hardness. [laughs] I mean, I think- [laughs] Um, I mean- I mean, there's actually, like, a lot of areas where it is easier.
273
00:41:49.048 --> 00:42:00.928
I mean, we, we have these, like, great simulators for large classes of materials, uh, whereas in biology it's incredibly difficult to model a cell or an organ or a full organism. Uh, so I think that's- Yeah...
274
00:42:00.948 --> 00:42:01.908
that's a huge advantage. Yeah.
275
00:42:01.948 --> 00:42:13.828
Th- th- th- isn't that interesting that i- at the smallest level it is one-dimensional or two-dimensional, uh, but then, and, and then, like, it just has many orders more scale and structure and like, and that actually introduces the complexity.
276
00:42:14.228 --> 00:42:19.628
It's- that's weird. Like [laughs] whereas, like, maybe, maybe the complexity in materials is more the microstructure.
277
00:42:20.048 --> 00:42:27.228
Yeah, I mean, you still, you still get, like, different complexity at the different, uh, length scales for materials as well. So yeah, microstructure and, and beyond. Yeah. Yeah.
278
00:42:27.808 --> 00:42:37.228
Uh, ju- just sort of like a brief, like, sort of palate cleanser, in terms of science fiction, let's, let's say you can invent what- whatever materials that you, you want, um, what is more valuable?
279
00:42:37.308 --> 00:42:45.468
Um, you know, I have, I have a list here. Uh, there's obviously room temperature super- superconductors, uh, but also, you know, you mentioned batteries. I- I've always thought, like, batteries.
280
00:42:45.848 --> 00:42:50.568
Just, like, do better than lithium ion, you're good. Mm-hmm. Like, you know- Mm-hmm... that's, that's stood around for like 100 years.
281
00:42:51.208 --> 00:42:59.288
Um, and then the last one is carbon nanotubes, uh, mostly for the, uh, space elevators. I, I, I don't know if like, tho- those are, those are three that come to mind.
282
00:42:59.328 --> 00:43:06.688
Are there any that, like, people talk about in your world that, like, are the dream? Definitely superconductors- Yes... um, magnets. Right.
283
00:43:06.748 --> 00:43:14.668
Especially if you can lower the dependence on rare earth or, like, transition metals that are hard to source, like cobalt, reducing cobalt in batteries.
284
00:43:15.228 --> 00:43:26.988
One of the things I think is very fundamentally important is, um, if we can get close to Landauer limit for, uh, compute- Energy efficiency. Can you, uh, explain Landauer limit? Yeah.
285
00:43:27.188 --> 00:43:37.068
So if you look at, um, how much energy we spend per, uh, like, FLOP or compute, it has followed the Moore's law type behavior, but it's gotten exponentially more efficient.
286
00:43:37.148 --> 00:43:44.868
So we've been spending exponentially less energy per FLOP or compute. I mean, I haven't checked recently. This was the case 10 years ago when I last- Yeah, that's great.
287
00:43:44.908 --> 00:43:54.028
I remember when I first learned about it, looked at that, and we were what? Like, 10, 20 orders of magnitude away from it or something. Beyond the limit, yes. Right. And last time I looked, it's we're weirdly close.
288
00:43:54.088 --> 00:44:01.888
Like, we're within a factor of, I don't know, like, three or four orders of magnitude away. Or- Yeah. I think- It's, it's actually, like- I think mainly, like, I guess exponentials rolled out over many years.
289
00:44:01.908 --> 00:44:06.368
I mean, yeah, yeah, exponentials over many years are, uh- Yes, right. Four or five, yeah. Yeah, pretty impressive. No, but, like, in your lifetime, you, you kind of saw that.
290
00:44:06.408 --> 00:44:11.728
Yeah, no, but, like, I mean, yeah, since, just since I discovered this, what? Like, 10 or 15 years ago. [laughs] Yeah, no.
291
00:44:11.788 --> 00:44:20.068
And one of the things we had to do is dissipate heat because, like, as, as these, uh, computations are happening, it's producing heat, which has to happen, but then we have to dissipate it.
292
00:44:20.368 --> 00:44:26.068
So, like, if we can get close to the Landauer limit for how efficiently we do computation, that's amazing for humanity, right?
293
00:44:26.108 --> 00:44:34.088
Because that means now we're doing computation, which is probably one of the most fundamental things we do as humanity, but as efficiently as possible from an energy perspective, which is the real currency in the universe.
294
00:44:34.488 --> 00:44:44.328
That get affected by quantum computing. I'm just gonna throw it out there. Um- Theoretically, massively, very embarrassingly parallel compute. Uh, so it definitely gets
295
00:44:45.268 --> 00:44:48.308
affected by reversible computing, but honestly, I'm not an expert. I don't understand.
296
00:44:48.348 --> 00:44:55.528
So to- if you can do reversible computing now, you're not, you don't actually have to spend energy to do computation, because you're not actually deleting any information. It's reversible.
297
00:44:56.228 --> 00:45:06.168
Uh, but I don't know if that works. I, I don't know. Uh, quantum computing, I assume, has to obey these laws somehow because, uh- [laughs]... they, they are still, like, they have to obey thermodynamics, yeah.
298
00:45:06.168 --> 00:45:15.448
It's, it's unitary, so- Yeah, exactly... like, it's reversible, so there's no- One, one of my most memorable conversations with Elad Gil was, like, he actually was a very skeptical person about quantum computing. I see.
299
00:45:15.508 --> 00:45:23.148
He's like, he's like, "Even if you had it today, there's no applications." I'm like, [laughs] "Whoa." I- Like, it breaks, it breaks sec- it breaks security, I always say. Like, that's it. I agree with that.
300
00:45:23.188 --> 00:45:32.288
Like- [laughs]... um, like, for them, people talk about how if you had a quantum computer today, you could simulate things so much better. And I ask them, "Okay, let's accept that. What would you simulate?"
301
00:45:32.568 --> 00:45:39.208
Like, you're still simulating perfect crystal. Like, you still have the issues that DFT has if DFT was perfect in its prediction ability.
302
00:45:39.628 --> 00:45:44.408
So yeah, I, I do think people are kind of glossing over some things because quantum computing is so exciting.
303
00:45:44.448 --> 00:45:53.148
Like, if we can compute in a quantum logic space instead of classical logic, it's just so exciting that people are glossing over what it would actually do when it's made.
304
00:45:53.468 --> 00:45:58.548
Maybe that's fine, because maybe once it's made, it will do amazing things we can't even imagine. Good. Okay.
305
00:45:58.868 --> 00:46:09.028
Um, I was gonna move to the, uh, automating the lab side, uh, where you've closed the loop on, uh, your experimentation. You can talk, talk about robotic arms, uh, talk about...
306
00:46:09.328 --> 00:46:17.068
Just basically, it's just everything you've done here. My favorite quote from you was that every piece of equipment in your lab is gonna have 140 IQ. Okay, what does that mean?
307
00:46:17.928 --> 00:46:24.228
In the early days, as we were scaling things up, we realized that we had huge bottlenecks imposed just by operating machinery.
308
00:46:24.748 --> 00:46:38.508
So, um, for one set of machinery, we would have technicians and scientists looking for, um, particular morpho- morphology and trying to con- see, okay, what actually were we making, uh, see if this is consistent with our intentions.
309
00:46:39.148 --> 00:46:50.768
And we really quickly, as we scaled up the lab, came into these bottlenecks where it just wasn't keeping up. So we started doing some programmatic approaches to capturing the data on SCM. And
310
00:46:51.788 --> 00:47:01.088
the data that was being surfaced from this, like, very simple program where it sort of takes a field of view, captures, zooms in, captures, was just not all that useful.
311
00:47:01.328 --> 00:47:12.768
Um, it was a little too dumb for what we really wanted to, to get. And so at that point, it was then, uh, really pertinent to build AI systems directly onto the machines to start controlling these things.
312
00:47:13.248 --> 00:47:22.348
And now they have the full context as to what we were trying to achieve. So what was the intent of the experiment? What were we trying to synthesize? What were some other experimental evidence?
313
00:47:22.668 --> 00:47:25.748
And it's actually looking, um, in the machine and capturing that data.
314
00:47:26.228 --> 00:47:41.368
And this is really valuable now because, you know, like, we've been talking about these hidden variables and not being able to capture everything, but if you can do more intelligent data capture at the time of that experiment, your data for future AI systems and future computational predictions is that much better.
315
00:47:41.528 --> 00:47:53.868
You'll, you'll have just a richer set of data. And so that's sort of what we mean by just, you know, every- everything on the, uh, the lab has to be incredibly intelligent to just make the data as useful as possible.
316
00:47:54.148 --> 00:48:03.708
So that was, that was sort of the motivation and inspiration behind that. What's the state-of-the-art there in terms of, like, uh, y- putting intelligence on every device, right? Well, I think, uh,
317
00:48:04.868 --> 00:48:14.088
one interesting aspect, um, is what's the latency- Exactly... of controlling that instrument. 'Cause there's cloud latency, but then there's also device compute, which also- That, that's right. Yeah, [laughs] exactly.
318
00:48:14.128 --> 00:48:32.988
And then there's sort of a, a, a timescale associated with the different, like, physical processes, and if calling out to an API or something is simply too slow, so the amount of reasoning or tokens or tool calls, uh, is just not matched to the latency of that actual process, then that, that's infeasible.
319
00:48:33.448 --> 00:48:36.428
Okay. That, that is fair. I mean, it's, it's almost like, you know, self-driving cars, right?
320
00:48:36.588 --> 00:48:46.508
So you- I, I, real quick, I just, like, am actually surprised to hear that because the latency I'd imagine for reasoning seems small compared to a lot of these things take, like, hours to run, right?
321
00:48:46.608 --> 00:48:54.368
Or maybe, you know... Or maybe, maybe I- maybe your experiments are much faster or high throughput or something. I'm just surprised to hear that, actually. So ultimately, the experiments- Yeah...
322
00:48:54.408 --> 00:49:03.668
do take, uh, you know, many hours- Yeah... days, but, um, there could be particular steps where latency would matter. Yeah, like, you might wanna have finer control.
323
00:49:04.308 --> 00:49:16.248
Uh, another thing is, um, how expensive it is, because, like, the, the same reason some of these models can be really slow in analyzing data also makes them very expensive. And then finally, the human patience.
324
00:49:16.308 --> 00:49:26.248
Like, if a human wants the analysis of a certain XRDI pattern, and they have to wait two hours before they get a good result, th- that's very different, I think, than if they can get it in, like, two minutes. Oh, okay.
325
00:49:26.308 --> 00:49:35.748
Yeah. And then the reasoning, um, it's, you know, you're not doing, like, a single call to that model, right? Like, you're, you're doing, uh, potentially many, many tokens, uh, to kinda get to these patterns.
326
00:49:35.808 --> 00:49:45.828
Like, run simulations in the loop. Exactly. Deep research in- It's not a single tool call, it's not a single inference thread. Sure. So then the latency can blow up quite a bit. I, I see what you mean. Yeah.
327
00:49:45.868 --> 00:49:53.508
Like, how much of your latency is tool calls? And I assume tool calls is, like, largely, like, DfT or computational- Or deals, yeah... and therefore, yeah.
328
00:49:53.908 --> 00:50:02.228
So how much of your latency is derived from those versus, like, actual reasoning? Um- It's really process-dependent. Okay, yeah. Yeah. Yeah.
329
00:50:02.248 --> 00:50:18.548
And then the other thing I would think about is also, I guess, building up from small things to bigger things, um, where I assume the, the, the, the general temptation or the typical development is incremental, where, like, everything is human, uh, operated, and then you're, you find pro- ways in which to automate it, and then you sort of build up from there.
330
00:50:18.808 --> 00:50:26.888
Mm. Um, I worry that sometimes that that is the way that people evolve things, but that's a local minima optimum. Like, short term is an optimization. Right. Exactly. Yeah.
331
00:50:26.908 --> 00:50:37.228
When, like, actually you should get a humanoid and just put them in there. [laughs] We, we- [laughs] I think we're of the opinion that solving humanoids would actually be slower to kind of getting to some of our goals.
332
00:50:37.228 --> 00:50:46.168
[laughs] Just checking. Again, like- Right... a lot of this is just, like, you do this every day. We, we, like- Yes... see this, but we, I don't know the reality of the situation. A lot of people hyping humanoids. Yeah.
333
00:50:46.388 --> 00:51:00.768
I mean, I think, like, maybe one, one process is, um, by having, like, a mix of, uh, humans and automation, you can identify really quickly, like, what are some of the bottlenecks in the experimental process, and you start alleviating those bottlenecks one at a time.
334
00:51:01.248 --> 00:51:12.008
Uh, and so for example, if there's some really tricky dexterity task that humans are excellent at that you'd have to spend months automating machine, maybe don't spend a, a ton of time there.
335
00:51:12.308 --> 00:51:21.888
And maybe, like, a, a more promising thing is something very routine that takes up a huge amount of scientists' and technicians' time and it's easy to automate. So I think s- there's, like, this pragmatism to it.
336
00:51:21.928 --> 00:51:34.628
But I mean, ultimately what we want from the lab is a huge quantity of data, high-quality data, diverse data, um, and that, those are our goals. And full autonomy is a non-goal.
337
00:51:34.848 --> 00:51:44.768
It's sort of in the service that we use automation in the service of achieving the goals on the data. Yeah. Yeah. Um, but also, I mean, I think another aspect to automation is, uh- Mm...
338
00:51:45.068 --> 00:51:59.848
robots will just be, uh, make fewer mistakes potentially in the lab, and it's been really helpful having AI systems with a full view over all of our data because sometimes if there's, like, a permutation in data, uh, then it can identify it.
339
00:51:59.888 --> 00:52:19.588
So for example, one of our steps at one point had, um, a cyclic error because one of the machines was loaded incorrectly and the patterns were inconsistent, so the AI was reading through these things and it says, "Well, given what was run, this is not expected," and it's looking at this basket of data longitudinally.
340
00:52:19.768 --> 00:52:24.048
And then it realized, if I do this, like, cyclic permutation and reverse it, everything is consistent.
341
00:52:24.088 --> 00:52:30.588
And then we were able to go back to the physical infrastructure and understand that a mistake had been made in the loading.
342
00:52:30.988 --> 00:52:42.148
And I think, like, managing data quality is so foundational to doing AI in the physical world and, you know, these are the types of things that we're building in, and so you can improve the operating process.
343
00:52:42.508 --> 00:52:51.248
But at that point you're like, "Okay, this is another great opportunity for automation. How do we make this just so reliable, so durable that we never have those types of mistakes again?"
344
00:52:51.468 --> 00:52:55.248
Yeah, I mean, the reducing the noise flow of experimental data I think is very important.
345
00:52:55.748 --> 00:53:09.788
Most experimental data in literature have such high noise flow that it's actually usually worse than DfT accuracy because, you know, when different people do experiments, different labs, different parts of the world, different times, it really introduces so much fluctuation to the result.
346
00:53:09.848 --> 00:53:15.568
Mm-hmm. So we are hoping that by standardizing these workflows, one of the biggest benefits will be that the noise flow will be lowered.
347
00:53:16.548 --> 00:53:23.788
You've been pretty public about how you use open source models and fine-tune them for, for stuff like this. Is it better? Is it basically...
348
00:53:24.188 --> 00:53:32.788
I can imagine a situation where, uh, you have, like, maybe the dumber models do one task and then you optimize for that task, and then, like, sort of like a generalist frontier model that, like, supervises everything.
349
00:53:32.848 --> 00:53:42.508
Is that a good mental model to have? Are there more stages to this that I can think about? Mm. Um, I mean, we, we basically use a mix of, uh, open source models and closed source models.
350
00:53:43.048 --> 00:53:50.888
You know, we're huge beneficiaries of this from our simulation side, building up the ML stack, the simulation stack. We don't need to push on, on that axis.
351
00:53:51.308 --> 00:53:56.848
But there's a lot of areas where, um, the latency is too high, um, the cost is too high.
352
00:53:57.327 --> 00:54:07.728
But in some cases we can actually, you know, by having access to this data, we can actually push beyond the frontier of what these systems can do under even some, like, the highest reasoning efforts.
353
00:54:08.228 --> 00:54:16.708
And so it's not necessarily just about cost or, um, Pareto efficiency, but in some cases you can push beyond it when you have access to data that no one else has.
354
00:54:17.368 --> 00:54:21.208
One p- um, way to characterize this is in terms of compute efficiencies.
355
00:54:21.568 --> 00:54:33.428
So by having access to this data, you can be that much more compute efficient compared to, um, you know, some of the frontier models, and that's been able, been like a, a really instrumental thing in our program. Yeah.
356
00:54:33.468 --> 00:54:45.228
On, um, on, I'm, I'm so- data quality or data access as a trade-off of compute, like, you know, I think there's some amount of exchange rate of dollars for compute for dol- dollars for data that people...
357
00:54:45.868 --> 00:54:51.108
I feel like the pendulum might be swinging now towards data. I mean, obviously you, you would agree with that, but like- I mean, yeah- Yeah. Yeah.
358
00:54:51.108 --> 00:54:55.668
[laughs] Like, you know, talking about, like, the noise floor of experimental data, like, we need to have high-quality data.
359
00:54:55.728 --> 00:55:02.848
If you throw a huge amount of compute against a bunch of noise, you're, you're not gonna have a good thing emerge. And it's harder for you because you actively try to be sparse.
360
00:55:02.908 --> 00:55:09.948
Like, you actually try to get, like, null results and, like, learn from that and- That's right. Yeah. Yeah, you've ta- yeah, you've talked about null results in, uh, several other venues.
361
00:55:10.048 --> 00:55:20.108
Like, w- what does a null result look like for materials, and how do you use that effectively, especially when I think your overall signal is probably quite sparse usually in terms of success? Mm. Mm.
362
00:55:20.188 --> 00:55:30.768
Um, I mean, and a null result could be we intended to produce some structure and then all of the evidence points to us not producing that structure. You ever intend to not produce a structure? Of course. Yeah. Yeah.
363
00:55:30.808 --> 00:55:40.008
Okay. Okay. Okay, cool. [laughs] Yeah, absolutely. You, you do the negative controls- Absolutely... regularly. Yes. Yes Okay. Okay. Like, there are impurity phases that will kill your property. Uh- Yeah. Okay, okay...
364
00:55:40.048 --> 00:55:43.508
it's gonna be toxic. Yeah, exactly. [laughs] 100%. Okay, okay.
365
00:55:43.588 --> 00:55:54.068
Um, and a negative result, you know, in some cases is actually, you know, we intended to make this, the following thing, and then we're actually able to, uh, identify a new structure that hadn't previously been identified.
366
00:55:54.648 --> 00:56:03.052
Um, so it's- Negative in some respect, like we in- intended to do something else, but something else emerged from, from the data. Yeah.
367
00:56:03.292 --> 00:56:13.572
I mean, also in general, like when you're training a machine learning algorithm, especially like at a very basic level, if it is a classification algorithm, uh, if you don't have negative samples, you can't really train if everything is positive.
368
00:56:13.632 --> 00:56:23.732
Yeah, yeah. And this is a particularly b- bad problem in material science because people usually publish crystals they co-synthesize, but they usually don't publish if they fail to synthesize a crystal.
369
00:56:23.792 --> 00:56:32.392
Sometimes they might. Um, and also, like we never know really if a sy- crystal is not synthesizable ever, right? It could just be a skill issue. Exactly. Yeah, yeah. It could be a skill issue.
370
00:56:32.452 --> 00:56:36.052
Especially with the literature. W- like synthesis method issue, technology. Yeah.
371
00:56:36.412 --> 00:56:46.192
So, um, I think it really helps us when we do our own experiments and get negative results in the context of what we tried, so then we can even train a classification, uh, algorithm.
372
00:56:46.852 --> 00:56:49.612
I think there's also, like, another valid thing too- Yeah...
373
00:56:49.612 --> 00:57:02.452
of when you have this sort of string of, uh, negative results, and then finally, through process iteration, you're able to get to that positive result, it's a really interesting set of, I'll call it, like process engineering type data.
374
00:57:02.952 --> 00:57:14.472
Um, so it's like through iteration, um, how did you actually get to that, that, that correct result? Because so much of, uh, material science has this ambiguity in actually how something was made.
375
00:57:15.012 --> 00:57:22.912
Um, and so, yeah, and, and so there's actually, even if there's a known material, it can be highly non-trivial to replicate that.
376
00:57:23.232 --> 00:57:36.352
Some things are, you know, in, like, you know, a high school textbook, but other things are really at the frontier and, you know, we're building up that know-how as well, and then building up a system that, you know, given this string of negative results, how do you actually get to that, um, positive case?
377
00:57:36.572 --> 00:57:47.412
Yeah, going to the classifier result, I would almost assume that if you are doing everything in-house, most of your results will actually be negative rather than positive, which is kind of ironic, because the literature only gives you positive results.
378
00:57:47.772 --> 00:57:56.012
So if you were sort of cold starting this, it seems like the problem is actually the reverse, that you- Yeah... have mo- uh, an abundance of negative results and not enough positive.
379
00:57:56.112 --> 00:58:02.972
And as you said, like if you were doing per experiment labels, it would be mostly negative. Yeah. But if you were doing per campaign- Mm-hmm...
380
00:58:03.172 --> 00:58:14.492
and assuming campaigns end when we succeed, then it could be a bit more balanced. I see. So your reasoning traces could almost be over entire campaigns- Absolutely... and not... Okay. This is, this is the key thing.
381
00:58:14.852 --> 00:58:23.832
This type of data basically doesn't exist anywhere else, and we spend so much of our time getting the full lineage of the scientific process into the model.
382
00:58:24.292 --> 00:58:35.272
And so, like, tracking all this data, like the conversations, the intuitions, what was executed in the lab, or the computations run, or was the code written, stitching all this together is, is so valuable.
383
00:58:35.732 --> 00:58:43.672
Um, and I, I think, like, you know, the overall goal is rather than training on the final output of science, you're training on the process of doing science.
384
00:58:44.052 --> 00:58:56.432
This feels very much like you, what you would try to do with RSI, where you are having a model train to train better models. Uh, you're now having a model train to make better experiments. Absolutely.
385
00:58:56.472 --> 00:59:05.552
But the difference is, uh, it's not going back into the core model for... So it's- Un- unless the physics models you're developing are improving chips, which then improve AI.
386
00:59:05.552 --> 00:59:10.532
[laughs] So this is the, the wider- The bigger, yeah, the higher level RSI. That's the big unified wheel.
387
00:59:10.732 --> 00:59:27.192
But does that mean that, is it plausible that in, uh, you know, Fable 6 or something, or you know, GPT-8 could just have a, which has been tuned on these sort of higher order, um, reasoning traces, could actually just
388
00:59:28.172 --> 00:59:38.092
do this without any- Mm... logic, because it's sort of the same thinking process, right? Yeah, I mean, I think there's, um, decision-making under uncertainty, obviously in machine learning.
389
00:59:38.132 --> 00:59:46.352
There's, there's noise, uh, in running, like the AI loops. Um, but we do think there's like a different set of challenges when you're actually, like interfacing with the physical world.
390
00:59:46.772 --> 00:59:54.232
But also, I think there's another piece too, which is getting the compression of everything into weights is still really valuable.
391
00:59:54.752 --> 01:00:05.392
If inference time reasoning was sufficient, all of the frontier labs would've stopped training at, like GPT-4 and were like, "Okay, every, from now on out, we're going to get really good at inference time improvements."
392
01:00:05.952 --> 01:00:19.872
Um, and so we think that by, you know, because of the differences between, you know, physical sciences and ma- machine learning, getting that compressed into our own weights will lead to, like different types of systems and different types of capabilities.
393
01:00:20.452 --> 01:00:30.012
But also, we feel like even if, like Fable 7 gets really good at, um, like even better than what it is today, it will still have to run experiments to get results.
394
01:00:30.712 --> 01:00:38.072
Um, and the reason for it is, right, like machine learning is really good at what it's been trained on, but scientific discovery is almost by definition what you haven't been trained on.
395
01:00:38.572 --> 01:00:49.912
And that's why we're building these labs, so that, um, whether open models or closed models can use these labs to tinker with the universe, because we don't feel like you can make a big discovery without trying things.
396
01:00:50.692 --> 01:00:57.272
Yeah, there's- Yeah... there's not gonna be like, no one's gonna like zero shot the room temp superconductor. Yeah, like think, think their way to- That would be pretty cool. Right. Yeah, yeah.
397
01:00:57.392 --> 01:01:05.652
I mean, as, as someone who has worked closely with, you know, wet labs before- Yeah... certainly I, I am more skeptical about zero shooting, like scientific results than some. Yeah.
398
01:01:05.692 --> 01:01:11.592
But it is something that I think, uh, some people might ask, so. Yeah, I mean, I think it's, uh, it's really important to distinguish- Mm-hmm...
399
01:01:12.232 --> 01:01:20.092
uh, the results we see in math and theoretical physics from the physical world. [laughs] Right, like- Yeah, yeah... these are two different things. No, no, it's two very different things. Yes.
400
01:01:20.112 --> 01:01:28.332
And not even theoretical physics, like so far it's been theoretical computer science coding- Yes... and math. Right. Yeah. Yeah, yeah. Maybe theoretical physics is next. It'll, it'll come. Yeah, that'll be next.
401
01:01:28.472 --> 01:01:34.012
Yeah, yeah. Oh, yeah. Can I get a, a sort of mental model of the layers, levels of extraction that you can go?
402
01:01:34.472 --> 01:01:46.632
Uh, so for example, uh, in terms of automation, and I, I, I'm just o- on this theme again where we talked about the campaign, talked about individual as- uh, essays and running tests and how they're, uh, humans are bad at it, so we should stop humans from doing it.
403
01:01:47.012 --> 01:01:58.292
Uh, are there others? Uh, so for example, one level higher than campaign could be, uh, a, a physics theory that you're testing. Mm. Mm. Or one level lower than campaign is what?
404
01:01:58.332 --> 01:02:06.032
Like, uh, [laughs] you know, and, and like, um, I, I just, I like to think about it from th- that point of view and then- Think about it from like, okay, well, this is an API call now.
405
01:02:06.152 --> 01:02:14.832
Like, you, you never have to touch this again. And this one is... Yeah, no, no, this is still, like, 90% human. Mm. You know, and maybe we, we can sort of draw the map of the territory that way.
406
01:02:15.452 --> 01:02:22.652
Is, is there, are there other levels? I can tell you some levels. I feel like you ask a good question. I don't have a very systematic answer, but, like, let's talk about some levels.
407
01:02:22.712 --> 01:02:32.992
So one level is the atomistic structure. This is an abstraction, right? Like, there's no perfect atomistic structure anything we do, but it's one approximation. Another one is the continuum model.
408
01:02:33.432 --> 01:02:43.212
So this one is, like, not atomistic anymore, but it's, uh, the continuous mesh representing the material. Okay. Um, and then in a different dimension, there's the thermodynamics.
409
01:02:43.672 --> 01:02:50.772
Like, assuming, uh, you do this experiment forever, what would be the final state? And then there's another layer of abstraction, which is kinetics.
410
01:02:51.732 --> 01:03:02.572
Acknowledging that we're not doing this experiment forever, so time matters. So how quickly will a reaction happen or not, even if it's lower energy or not. Um, so thermodynamics, kinetics, atomistic, continuum.
411
01:03:02.932 --> 01:03:13.792
What else? Are there other levels of abstraction? Well, I mean, I think, like, your, your point about like, okay, there could be an overarching new theoretical advance that informs multiple campaigns. Right?
412
01:03:13.832 --> 01:03:22.452
Maybe the difference... Kind of like as an investor, I, I wanna go like- Yeah... "Here's the bottleneck, guys, and, like- Mm... when we get this, we get everything else." I don't know. [laughs] Oh. I mean,
413
01:03:23.532 --> 01:03:40.172
from the beginning, our hypothesis has been that one of the big bottlenecks is automated characterization, because it's not that hard to mix powders to get to try stuff, but if you can't characterize and analyze it, and then decide what the next step should be intelligently, you don't really benefit much from mixing powders randomly.
414
01:03:40.632 --> 01:03:51.132
Uh, so we focused a lot on automated characterization, closing the loop with simulations, uh, because that's how you, like, try a lot of things, make an informed decision, and then decide what to do the next day.
415
01:03:51.872 --> 01:04:00.272
May- maybe as a follow-up question, uh, what is the-- how do humans live in this loop? Like, where- [laughs] At what points do you pull, like, drop the human in?
416
01:04:00.892 --> 01:04:08.012
I mean, I- I'm guessing you probably could- You could, you could randomly- Yeah... drop into any part- Of course. Yeah... and, and add value. Yes. But, like, where do you decide to spend your time?
417
01:04:08.152 --> 01:04:09.832
[laughs] I mean, it seems like it might be all levels.
418
01:04:09.892 --> 01:04:19.592
Like, you, you have our-- you have people in the lab who are physically moving materials, and then you also have scientists who are guiding decisions about what campaign, and then you have humans who are...
419
01:04:20.032 --> 01:04:24.132
Well, so I think I'm just asking the question for you. Yeah, yeah. No, yeah. Like, 100%, I think it's always shifting, too. Yeah.
420
01:04:24.172 --> 01:04:33.732
So I mean, I think even just using, like, the guidance of campaigns, in the early days, that was fully human driven. Now, increasingly, it's, it's a mix of AI driven and human driven.
421
01:04:34.232 --> 01:04:38.672
Um, and then Ders was saying early on, we identified that characterization was a big bottleneck.
422
01:04:38.872 --> 01:04:47.252
In the early days, it was human driven, and the scientists were very much overwhelmed, and the balance has shifted significantly towards AI driven.
423
01:04:47.752 --> 01:04:55.412
And this has allowed the scientists, who were previously spending all of their time doing these refinements, to now elevate their work to, to something else.
424
01:04:55.892 --> 01:05:04.432
Um, but I mean, I think we're-- it's, it's really sort of a, a, a function of time. There's-- it's, it's always changing at each of these levels. Yeah.
425
01:05:04.472 --> 01:05:13.912
In terms of just general search, are there things that perform well in, uh, let's say, the physical world that don't perform well in the other worlds or, or vice versa?
426
01:05:14.072 --> 01:05:27.072
Like, like, so, like, evolutionary search is pretty popular in LMS. I don't imagine it works well here. So especially in simulations, people have been using evolutionary search, uh, you know, for a long time.
427
01:05:27.152 --> 01:05:33.032
Um, like, it's pretty good at structure prediction, for example, finding low energy structures. Um- Okay, so it works. Yeah, okay...
428
01:05:33.052 --> 01:05:43.992
and then, uh, like, process engineers in semiconductor industry will do DOE, design of experiments, and they'll often use some kind of zeroth order, uh, like, whether it's Bayesian optimization or evolutionary search or...
429
01:05:44.392 --> 01:05:54.552
I, I don't think they use RL, but similar idea, yeah. You have to have some kind of algorithm- Yeah... to, to figure it out. And all zeroth order optimization behaves the same anyway, right? So... [laughs] Yeah. Okay.
430
01:05:54.612 --> 01:06:06.572
I, I wanna go back to a, a question I had earlier, which was one of scaling. And so I- I'm curious, what is your vision for scaling the lab? Mm. So there's different ways, and coming from the biology world, um,
431
01:06:07.732 --> 01:06:11.692
there's a lot of cool tricks you can do in biology. Uh, you can tag things.
432
01:06:11.912 --> 01:06:22.712
You know, you can use DNA sequencing for all sorts of interesting readouts, and if you can map your complicated assay onto sequencing, you can now scale to millions or hundreds of millions or something.
433
01:06:23.532 --> 01:06:32.352
Can you do tricks like this? Like, ultimately, every assay has its own runtime. Uh, well, I'll-- I have a favorite blog post I would wanna-- I'll quote in the show notes.
434
01:06:32.672 --> 01:06:43.132
But, like, what does the runtimes look like for these labs? Is it-- is your scaling just for mater- materials just fundamentally linear in terms of, you know,
435
01:06:44.032 --> 01:06:52.152
you do more-- you just need more resources to do more things, or can you parallelize things in clever ways and- Mm... combine things, and are, are there tricks there?
436
01:06:52.872 --> 01:07:06.712
Well, I mean, I think, like, one way to think about it is, um, the RL environment is not quite trivially run the experiment, wait for that agent to fully roll out, um, you know, doing the calculations, going through all the instruments to the final outcome.
437
01:07:07.312 --> 01:07:26.452
We basically start producing data across all the different instruments and all these different campaigns, and one way you can begin to expand that for machine learning is now reinforcement learning environments can be constructed based on, again, like, dates for, you know, the experimental campaigns or computational campaigns at that point.
438
01:07:26.972 --> 01:07:41.172
Or you can also, um, start taking, um, subsets of instruments, and you're like, "Okay, we're gonna create reinforcement learning environments on the basis of this instrument alone," or maybe it's, um, across these different instruments to kinda get to some reward state.
439
01:07:41.752 --> 01:07:54.142
Um, so that's a way where- This finite basket of data, when viewed in different ways, um, can then be expanded, um, for training purposes. So it d- it doesn't get at the actual experiment run- Yeah...
440
01:07:54.542 --> 01:08:03.322
but from an ML perspective. Maybe I can give you some analogs for the bio examples you gave. Yeah. So, you know, one thing people have tried is combinatorial sputtering. I don't know if you ever heard of this. No.
441
01:08:03.362 --> 01:08:07.742
Like, you have these sputtering targets, and you create this compositional gradient.
442
01:08:08.302 --> 01:08:20.322
So when you look at the final product, because there is gradient from each precursor, it creates different crystals at different spatial locations. So in one go, you maybe try hundreds, thousands of crystals.
443
01:08:20.902 --> 01:08:26.782
Um, this an exa- this similar, right, to your example. Oh, yeah. That makes sense. And as far as I know, this hasn't worked very well so far- [laughs]...
444
01:08:26.922 --> 01:08:30.622
because it turns out, like, there is diffusivity, so things move around.
445
01:08:30.622 --> 01:08:37.142
[laughs] Another one that we thought about but we haven't done yet, but a lot of people talk about this, is superconductivity measurements are a bottleneck.
446
01:08:37.382 --> 01:08:42.242
Because unlike XD, there's no high throughput superconductivity measurement, and it can take, like, one hour per measurement.
447
01:08:42.602 --> 01:08:52.802
So people [laughs] have thought about taking the different candidates, mixing them all into one sample, and putting it in, and if, uh, it has superconductivity, you know that one of the sources did, and then you can do...
448
01:08:53.042 --> 01:09:00.242
A bit like how people are doing COVID testing to, um- Yeah. Mm-hmm. So there are ideas like this exist. Uh, some of them work well, some of them don't. Yeah. Yeah.
449
01:09:00.282 --> 01:09:06.462
So you, you I think are about to announce a big fundraise. How, how are you thinking about, um, scaling?
450
01:09:06.662 --> 01:09:18.002
Like, we've been doing our design of our labs for a while, and so I think the resources allow us to continue to scale up the labs significantly, but also the compute, both from, like, the AI side and the, the computational side.
451
01:09:18.522 --> 01:09:22.402
Uh, but I think really importantly, too, it's what are the new types of labs we're able to build?
452
01:09:22.502 --> 01:09:33.582
So I think we've been proving out this loop, um, in our initial labs here in Menlo Park, but we'll be continuing to expand to new types of labs and, and repeat the same process.
453
01:09:33.822 --> 01:09:37.762
Yeah, like, the kind of lab we're trying to build I think haven't been built before at this scale or at this approach.
454
01:09:38.102 --> 01:09:44.562
So we have learned a lot from our own build-outs so far, and those lessons guide us for the next version and the next version.
455
01:09:45.062 --> 01:09:52.701
And then we can, as you said, like, increase the scale, the ambition, the quality of instruments. Some instruments are very expensive.
456
01:09:53.122 --> 01:10:01.562
Uh, as we get a really good understand for what we get a lot of return from, we can invest into those more. And it makes- hardware engineering has become so core to it.
457
01:10:01.642 --> 01:10:10.742
So for example, in the early days, just for speed, we'd buy some off-the-shelf instruments. And as we kinda pushed the scale of the lab- You make your own... we have to make our own.
458
01:10:10.822 --> 01:10:18.862
So we realize, okay, for this instrument, um, it's pretty fast. These components are actually pretty quick, but this weighing machine actually becomes the bottleneck.
459
01:10:19.462 --> 01:10:29.562
Or there can be things pertinent to data quality where, well, the resting position of this robotic arm actually is above the plate of previously mixed things, so there's a contamination risk.
460
01:10:29.822 --> 01:10:36.862
So in our hardware design, we're going to redesign it so that the resting position lies away from tho- those things.
461
01:10:36.902 --> 01:10:45.702
Like, these are the kinda subtle details that allow us to kinda push the noise floor down for our experimental campaign. So, um, basically- So-... take these learnings and, and scale it up.
462
01:10:45.962 --> 01:10:56.662
Uh, you know, you ship your org chart. Uh, it- you have many teams. I think the... We were surprised when we looked at your jobs page, and we were like, "Okay, we don't think we fully map what Periodic does."
463
01:10:56.922 --> 01:11:09.742
Maybe some organizing principles, so to kinda make sense of the job page. Uh, so we hire for AI research and infrastructure, computational, uh, experimental roles, and hardware engineering roles, and then product roles.
464
01:11:10.262 --> 01:11:23.462
We want to achieve synthesis superintelligence, right? And we feel like for that we need the right chemistry expertise, the right physics expertise, simulation theory, and thin films, powder. So hardware engineering.
465
01:11:23.522 --> 01:11:30.742
Like, these seem like very diverse roles, which they are, but they're all actually, um, coherent in what they're trying to do, which is a synthesis superintelligence.
466
01:11:30.922 --> 01:11:41.942
And this also applies to the LLM researchers, the infra, even the product engineers, because the product engineers kind of make sure all the research gets made into a product that the experimenters in the lab can use.
467
01:11:42.342 --> 01:11:56.642
Yeah, so in order to do the end-to-end loop with the physical world, it really requires your ability to have agency over the physical world, to build these labs, to run the campaigns effectively, to create AI systems against that, to make them good users of the computational tools.
468
01:11:57.222 --> 01:12:08.042
Uh, that's, like, the difficulty but also the opportunity of Periodic. It's like this group of people has just never been brought together before. It's irreducibly a multidisciplinary problem. Yeah.
469
01:12:08.082 --> 01:12:16.382
And o- one other, like, guiding principle for us that is really important for us is we want the people who are very good at what they do and experienced to do it hands-on.
470
01:12:17.062 --> 01:12:30.822
You know, like, the modern life has gotten us to this place where, especially in academia, when someone is really good at research in some area, we give them so much responsibility for grant writing, teaching, that they stop having the time to do research in that area hands-on.
471
01:12:31.482 --> 01:12:42.702
But if you look back at, like, Bell Labs, IBM, institutions that made really good progress, it was really experienced people doing hands-on work. Like, Bardeen was in the lab every day even though he's a theorist.
472
01:12:43.062 --> 01:12:51.562
Alex Mueller was in the lab doing experiments even though he was the lab lead. So we try to do that here, too. So we have some of the world's leaders in different fields, but they're doing hands-on work.
473
01:12:52.122 --> 01:12:57.462
Do you wanna brag? Maybe just call them out? Like, you know. Oh, I, I would love to, yeah. So, um, you know, it...
474
01:12:57.742 --> 01:13:06.942
When I was doing my PhD, my favorite computational material scientist, who's kind of from my age group, is Muratan Aykol. He did his PhD with one of our advisors, Chris Wolverton.
475
01:13:07.482 --> 01:13:17.682
And, uh, he is kinda like our computational material science expert. He's incredible. Uh, he's sometimes, you know, a real good experimentalist even though he does simulations. Like, he understands systems really well.
476
01:13:18.202 --> 01:13:24.782
Uh, our lab lead is Joe Checkelsky, who is a professor at MIT, but he's on leave to work with us full time.
477
01:13:25.202 --> 01:13:36.322
And Joe is this incredible physicist, understands superconductivity really well, but he also was a professor in Japan for a bit, so, uh, learned synthesis and chemistry really well, especially for a physicist.
478
01:13:36.762 --> 01:13:43.602
We have Daniel Chica, who was a graduate student in Mercury Cannasitis' group, which is maybe the best solid state chemistry group in the world.
479
01:13:44.042 --> 01:13:51.102
And Daniel was one of his, like, synthesis experts, so he's like a magician with his fingers. Um, I mean, there are so many examples like this. No, I know.
480
01:13:51.142 --> 01:14:02.422
I mean, I think, like, um, Dima Bahdanau, so he's been leading a lot of our AI and LLM efforts. He was the inventor of neural network attention. Um- Oh, Bahdanau. The Bahdanau attention. Yeah. Exactly. Yeah.
481
01:14:02.422 --> 01:14:11.762
[laughs] Yeah, yeah. Yeah, yeah. Yeah. Yeah, so very hands-on and sort of getting into, like, you know, the traces, the training distribution, how to connect it to the lab. The chemistry- Chemistry... parts.
482
01:14:11.822 --> 01:14:23.282
I mean, he'll go deep, yeah. He goes very deep. Um, he actually has, like, solid state synthesis books, like, at his lab in, uh, Montreal. Ray Nakano, so he used to tech lead some of the, um, operator work at OpenAI.
483
01:14:23.802 --> 01:14:31.622
Uh, he's in the lab. So when we were doing some lab tours, uh, we were initially, we're like, "Oh, this new, uh, scientist, this new technician really looks like Ray."
484
01:14:32.042 --> 01:14:43.542
And no, it, it was li- literally this was Ray, like, actually in the lab, um, with the scientists automating the machines. And it's like this is sort of the, the DNA and the type of people we need to kinda pull this off.
485
01:14:43.894 --> 01:14:53.974
Uh, so, uh, we also saw that you had four deployed engineering, uh, roles. Um, we are happen to be spinning up a four deployed engineering podcast because- Mm... there's so many engineers that would do that role.
486
01:14:54.414 --> 01:15:01.034
They may not necessarily know that there even is a role for them at Periodic. Mm. What is it, um, and how do they work with customers?
487
01:15:01.534 --> 01:15:08.754
So we've been building our own products, our own tools, our own AI and computational- For yourself. Yeah... for ourselves. So we've been customer zero.
488
01:15:09.374 --> 01:15:15.873
Now what we're doing is we're taking these same tools to different industries, and we're, we have a huge focus right now in the semiconductor industry.
489
01:15:16.554 --> 01:15:25.794
Uh, however, in order to do this, like these are very, uh, um, private companies, we need to be able to operate within some of the most secure environments.
490
01:15:26.174 --> 01:15:38.514
And so our Periodic four deployed engineers and researchers will actually be on site working and actually integrating our AI systems, our computational systems, to help our partners get to their goal states more quickly.
491
01:15:39.014 --> 01:15:46.594
Uh, so basically the tools, the knowhow that we've built in doing our own materials discovery, materials engineering, we're now bringing to industry.
492
01:15:47.154 --> 01:15:58.094
And we think this is the, the most effective way to get partners to their end states and to their goals, rather than just like throwing some technology over the wall and telling them to, to figure it out.
493
01:15:58.474 --> 01:16:05.434
Then these four deployed engineers will also do the inference locally, but then also can, uh, train on the data.
494
01:16:05.554 --> 01:16:24.274
So again, once we take our system that understands these different areas and we deploy, we can make it expert on the customers' or the partners' data so that they can own their own intelligence and have systems that understand this much more effectively than, uh, you know, than just like hitting some API or some untrained model.
495
01:16:24.774 --> 01:16:34.874
And so this is a much expanded, um, four deployment engineering role than typical because this really requires some very deep, uh, machine learning expertise.
496
01:16:34.974 --> 01:16:44.954
Uh, you have to be incredibly precise from like a data training perspective, infrastructure perspective, but also a physics perspective. So we're working with, you know, highly technical industries.
497
01:16:45.254 --> 01:16:52.914
They have to understand, you know, different areas of like chemistry, material science, um, some of the devices. And so that, that, those are the roles we're building out right now.
498
01:16:53.174 --> 01:16:59.014
Pa-part of it is willingness to spend extended periods on site in Taiwan. [laughs] It's like, okay, well I, I know that kind of customer.
499
01:16:59.034 --> 01:17:09.134
[laughs] Uh, it's interesting that like you're monetizing-- that, that, that is the primary way that you're monetizing now, but it might change in the future.
500
01:17:09.214 --> 01:17:15.834
But like, I, you know, and that's, that's like a, a, a, a model that I think people don't really get about APL Research Lab.
501
01:17:15.974 --> 01:17:22.434
Like, 'cause when you compare yourself to like a Bell Labs, that's what people are thinking, right? Like, which is- I think like a really great analog would be software engineering. Right.
502
01:17:22.754 --> 01:17:32.154
So in the early days of software engineering, like we had like, you know, GitHub Copilot, then we had early versions of ChatGPT, people were using these things as copilots to help them get to their solutions.
503
01:17:32.634 --> 01:17:38.614
And as the automation improved, now we have things like Codex and very few of our engineers are writing code the way they used to.
504
01:17:39.194 --> 01:17:48.874
And we think a very similar thing could play out for Periodic as well, where you can have systems to accelerate the researchers, the material scientists, the materials engineers, the process engineers.
505
01:17:49.314 --> 01:18:03.474
But as the autonomy, intelligence, and capability grows, you can begin to, um, price outcomes. So help me get to this kind of goal state. Um, and I think that's a, a really interesting area for us. Yeah.
506
01:18:03.534 --> 01:18:19.154
I mean, we kind of feel like our best contribution to solid state physics and science could be if we made these tools and topics profitable, uh, similar to how ChatGPT made CS and LLM majors way more popular in colleges before and after.
507
01:18:19.594 --> 01:18:30.554
We love to show that solid state physics, material science research can make a big impact commercially, and then that will attract more attention. Young people will want to study physics, uh, which will be, uh, a dream.
508
01:18:31.034 --> 01:18:37.374
Um, and you know, Bell Labs did make a huge commercial impact, right? So they failed to commercialize some of their incredible advances.
509
01:18:37.754 --> 01:18:46.514
They of course, did some research that just couldn't be commercialized, like the cosmic, uh, background, but, but they did benefit a lot from like the vacuum tube connecting- Oh...
510
01:18:46.514 --> 01:18:54.394
uh, East Coast to West Coast by phone lines. Yeah. And I think it's, it's really also interesting to think that technology and capital are incredibly intertwined.
511
01:18:54.874 --> 01:19:05.974
So if you look at what was the progress on chatbots for many years, like imagine recounting, okay, what was the progress from chatbots from, I'll say, 2010 to 2015?
512
01:19:06.594 --> 01:19:17.474
We'd be kind of at a loss to say over that five-year period, how, how much did they improve? Whereas if you look at the period from, uh, 2021 to 2026, you know, it's, it's night and day.
513
01:19:17.854 --> 01:19:31.214
And what happened was ChatGPT and these other systems were able to achieve a product market fit and it changed the capital landscape entirely. This changes the landscape for hiring, um, compute, data, et cetera.
514
01:19:31.294 --> 01:19:41.774
The virtuous cycle. Exactly. And so- Like success breeds success. Exactly. So technology, uh, it's incredibly coupled to the capital and those resources, and we wanna achieve the same thing in the physical world. Yeah.
515
01:19:42.174 --> 01:19:51.614
So one thing I've been seeing is there, there's this move from science being funded from, you know, government grants and so on to, you know, VCs and, uh, private funding.
516
01:19:52.214 --> 01:20:02.854
Do you all plan on making anything you're doing, uh, open source, like either releasing data sets or actual models? Is this something like in the future even... You know, there, there are many models which may be not,
517
01:20:03.874 --> 01:20:09.434
well, your, your state of the art, but which could still be- Mm... you know, useful releases. Absolutely. I mean, so, uh, we have a couple of things.
518
01:20:09.494 --> 01:20:25.594
So we have been contributing to open source to kind of like PyMedGen, the materials project, uh, code base, uh, Custodian, their DFT runner, uh, TorchSim was actually created by one of the researchers here, Abhijit Gangan, and he maintains it still.
519
01:20:25.894 --> 01:20:36.234
JAX-MD, uh, Abhijit maintains it. So we do a lot of, uh, open source contributions- SGLang... including LLMs... Megatron. Yeah. Yeah, yeah. So we've been very active contributors back into open source.
520
01:20:36.554 --> 01:20:47.994
And, and we also have an academic grant program where we give academic gift grants to, uh, academic groups in universities that we feel like are really advancing this direction towards synthesis super intelligence.
521
01:20:48.554 --> 01:20:57.486
Uh, so that's also been great to see. I think the first paper from this funding is about to come out, so it'll be exciting to just get more papers come out like this. Yeah Okay.
522
01:20:57.846 --> 01:21:06.526
Um, let's steel man this say that you've successfully have a hundred percent lab automation, which does everything, characterizes everything correctly. You have all of the fun agents which can do everything.
523
01:21:07.286 --> 01:21:15.686
I, I still am somewhat unclear about how you actually get to superconductivity. There's a lot of hard problems to solve. W- what is the path there when,
524
01:21:16.606 --> 01:21:23.466
like, there is no sort of like theory for model for most of the, like, strongly correlated systems or high temperature systems? Yeah.
525
01:21:23.646 --> 01:21:33.206
So, uh, I mean, our view is that it's not hard to think of chemical spaces that would host superconductivity. What's really hard is to synthesize them.
526
01:21:33.286 --> 01:21:35.886
That's why we're really, like, emphasizing synthesis super intelligence.
527
01:21:36.226 --> 01:21:49.026
So maybe one example to give historically is Alex Müller, when he thought that transition metal oxides might be a good path to superconductivity, he was actually trying nickelates, the nickel oxygen and some, uh, cations, and that didn't work.
528
01:21:49.406 --> 01:21:57.626
And then he tried cuprates, which is cup- copper oxygen and some cations, and that worked, and then he got a Nobel Prize. But then we know that later on, turns out nickelates was a good idea.
529
01:21:57.706 --> 01:22:03.406
I mean, nickel and copper are next to each other on the periodic table. Those are always two, two layers, right? Yeah. Like sodium, yeah. There are so many three-D transition metals. Yeah. Mm-hmm.
530
01:22:03.426 --> 01:22:12.226
And then, uh, Harold Hwang, one of our, um, Stanford professors here who's incredible, he realized he can make nickelates in thin film form and show the superconductivity.
531
01:22:12.786 --> 01:22:25.226
Um, so, you know, if we have a really good synthesis super intelligence in the lab and we can really scale up these things, I think we won't run out of ideas or the LLM won't run out of ideas about what directions to try.
532
01:22:25.866 --> 01:22:34.866
Uh, like three-D transition metals, oxygen, like they're very related ideas here. Like many people have been trying like cobaltates, which is like instead of copper or nickel, you try cobalt.
533
01:22:35.406 --> 01:22:43.706
Um, so I think that's super exciting. Like maybe one historical example to also study is the Japanese group that discovered magnesium diboride.
534
01:22:44.226 --> 01:22:54.306
As you know, that's the highest temperature, ambient pressure conventional superconductor, and they discovered it by just trying a bunch of materials. I think they tried thirty thousand different things.
535
01:22:54.726 --> 01:22:59.066
Thirty of them seemed to host interesting superconductivity, and MgB two was one of them.
536
01:22:59.606 --> 01:23:12.146
But magnesium diboride sat on people's shelves as a precursor for decades before then, and BCS theory had been invented back in nineteen fifty-seven. So the way people found these materials wasn't projected from theory.
537
01:23:12.466 --> 01:23:16.526
It wasn't because they couldn't synthesize until then. It was just like they tried a bunch of things.
538
01:23:16.886 --> 01:23:27.586
So now imagine if we had this perfect automation that you're describing, which sounds like a dream, um, and we tried the thirty thousand the Taketo group tried in over his career in a month.
539
01:23:28.146 --> 01:23:37.186
We just really increased the surface area for luck. Uh, no, but yeah, it's very inspiring what you've done. Congrats on all your success. I feel, I feel like you're creating, yeah, the, the modern, like, playground.
540
01:23:37.366 --> 01:23:45.486
I, I imagine recruiting must be like super easy for you, so I'm just like a little bit jealous, but- [laughs]... uh, you've worked hard for it to get here. So yeah. Yeah. Well, thank you so much.
541
01:23:45.546 --> 01:23:59.526
Yeah, it was great chatting with you both. Yeah, super fun. Yeah. Thank you. [outro jingle]