Trascrizione completa di The Programmable Data Center — Evan Conrad, SF Compute

--> 00:04:38.820 Uh, and, like, I can't pay for this. Yes. And so I will pay you to take my oil. Yes. [laughs] Uh- That, that is, like, the weirdness of physical settlement. Yes. 29 00:04:39.260 --> 00:04:42.000 There's a bunch of weirdnesses like this in other markets as well. 30 00:04:42.320 --> 00:04:54.620 So in the power markets and the grid, um, there's parts of the orders that you fill out, the buyers and sellers, that, um, are there because the boiler takes a while to cool off. 31 00:04:55.180 --> 00:05:03.020 Um, and you can, you know, stop the boiler, and then it will-- you can start it back up again within some period of time, and so that has to get factored into the order book. 32 00:05:03.520 --> 00:05:06.260 Um, this genre thing is the same thing that happens in compute markets. 33 00:05:06.400 --> 00:05:24.120 So we co-designed physical settlement and the order book at the same time, and that meant that we get to actually handle the real constraints of compute, which might include something like, "Oh, I'm okay with being interrupted, but only if there's, like, storage on the thing that I can recover from really quickly." 34 00:05:24.580 --> 00:05:33.980 Um, very similar types of problems. Yeah. So, uh, people can refer to your talk, and you've done a couple other podcasts already. Sure. Um, so I wanna be, like, the sort of 201 level instead of the 101 level. Sure. 35 00:05:34.320 --> 00:05:40.660 Uh, and also because I, I am, I have been a derivatives trader- Yes... so I can give people a bit more detail that you normally don't get into. Okay. 36 00:05:41.380 --> 00:05:48.440 Um, so for example, well, how, how mature or deep or liquid is the compute market today? Uh, not very mature or deep or liquid. Okay. 37 00:05:48.520 --> 00:05:58.560 Um, so what we've been doing is first we had to, we had to go get a bunch of supply online in the first place, which just takes a while, and then we had to build the entire platform in order to do this. 38 00:05:59.000 --> 00:06:10.300 Um, and we just now got that working, and so now we move, um, capacity over onto the order book, um, which makes it liquid. Um, and so 5% of our total capacity is on the order book. Okay. 39 00:06:10.340 --> 00:06:14.300 And we're consistently moving more and more, and then we're bringing on more capacity that's all on the order book. 40 00:06:14.680 --> 00:06:23.480 Um, the nice part is that, um, uh, maybe I should have said the, the volume on the actual order book is small, but the, um, the liquidity on the order book is high. 41 00:06:23.560 --> 00:06:30.640 So for that small amount of capacity that you have, um, you have, um, like, very concentrated liquidity. So Um, [lip smack] 42 00:06:30.640 --> 00:06:42.820 a sort of standard metric, I would say, for someone who is using this is a customer who buys capacity from us, and they spend $100 to buy that capacity, will make back $25 on average from selling it back to somebody else. 43 00:06:43.240 --> 00:06:44.960

Um, and that's the number that we... 44 00:06:45.120 --> 00:06:53.980 Or this is one of the metrics that we try to track because, um, w-we want the liquidity to be useful to the commercial participant, and you can measure liquidity in a bunch of different ways. 45 00:06:54.040 --> 00:07:04.680 You can say like, "Oh, I just want tons of makers or something." Um, and you can measure it in, like, high-minded ways. We try to measure it by, like, is this solving the problem, um, for the customer? Yeah. Got it. 46 00:07:04.960 --> 00:07:15.920 Mm-hmm. And so, uh, last time we talked as well, there was a lot of contr- uh, conversation about standardizing the contracts. Yes. Um, talking with, like, uh, the C- CFTC or whatever. Yeah. 47 00:07:15.920 --> 00:07:28.160 Uh, what's the status on those things? Um, so I think standardizing the contracts in general is still pretty far away. Yeah. Um, and the-- What we were trying to do at the time was, um, [lip smack] 48 00:07:28.220 --> 00:07:34.260 or basically what we learned over the last several years was w-we just needed to operate the whole thing- [laughs]... otherwise you cannot standardize the thing. 49 00:07:34.300 --> 00:07:40.220 And so to get closer to something that can be standardized, you first need to make all the operations of the thing the same. Right. 50 00:07:40.380 --> 00:07:47.400 To give an example of what can g- be different is a single operator can offer two different types of virtual machines. 51 00:07:47.860 --> 00:07:56.020 One type of virtual machine, um, might have a f- 40% performance, um, degradation versus another types of virtual machine, and 40% is a lot. 52 00:07:56.180 --> 00:08:02.580 Um- When you say performance degradation, are you talking, like, MFU type things, or, like, amount of burn-in that you've experienced or what? It will affect... 53 00:08:02.740 --> 00:08:11.320 So in this particular case, I'm talking about GPU kernel launches and how long it takes for that to happen, um, which is a process that happens quite frequently when you're doing sort of either training or inference. 54 00:08:11.440 --> 00:08:22.900 Okay. Um, the um, [lip smack] uh, that will basically-- The end result is that you get a worse MFU, but, um, like- Is the intermediate level- Yeah, it's the intermediate level, basically. Yeah, yeah. 55 00:08:22.940 --> 00:08:30.460 So, so what I'm trying to... Uh, so for anyone who's traded commodities before- Yeah... there's different grades of commodity. Yes. Right? There's like... It's not oil, it's light sweet crude- Yeah... 56 00:08:30.560 --> 00:08:38.519 versus, like, whatever- Exactly... the opposite of light sweet crude is. Or it's like, you know, uh, corn level A or corn B or whatever. Yeah. Uh, and so, like, people are trying to... 57 00:08:38.579 --> 00:08:42.880 Well, well, you have to establish the equivalence of those things. Correct. Yes. And I'm wondering if they, they have been established. 58 00:08:43.080 --> 00:08:51.120 So what we did instead, um, was we created an order book that had multiple different constraints, and you can say, "I wanna have, um, you know, something that's between this and that. 59 00:08:51.280 --> 00:08:55.820 Um, it runs from this time to this time. Um, it's like, uh, X to Y." 60 00:08:56.040 --> 00:09:02.180 It's a bunch of different properties that you can attach to your order, and then we match that against sell orders that have those properties- Mm-hmm... um, in a big matching engine. 61 00:09:02.560 --> 00:09:09.520 Um, and the reason why we did it this way is it allows us to generate liquidity without having to standardize before you know what the technology can do. 62 00:09:09.940 --> 00:09:15.240 Um, and because the technology was shifting so quickly, um, we wanted to basically be able to do that. 63 00:09:15.560 --> 00:09:24.780 The trade-off of this is that you have less concentrated liquidity, and so the prices are, um, going to ha- There's gonna be more spreads between the, the buy and the sell. Yeah. Um- It, it sucks. Yeah. 64 00:09:25.040 --> 00:09:29.720 But on the other hand, it means there's lots of arbitrage to make, um, if you can do that correctly. 65 00:09:29.880 --> 00:09:39.680 So if you have a product that can use lots of different types of capacity, um, then you might really like SF Compute because you will be able to use lots of different types of capacity and you'll make more money. 66 00:09:40.080 --> 00:09:48.600 Um, you can take the, the spread. Okay. What dimensions matter in this grid of things? Um, [exhales] so a bunch of different things. 67 00:09:48.660 --> 00:09:57.700 Um, one is dimensions, categories that I will say is, like, is there a workload running on it right now or is it capacity in the future? So is it capacity- Sure... right now? 68 00:09:58.160 --> 00:10:06.920 Um, [lip smack] capacity right now, in order for you to sell that, um, you have to interrupt your current workload, so you have to think, "Do I care if my current workload is interrupted or not?" 69 00:10:07.280 --> 00:10:14.120 Capacity that's sold that's starting a month from now, um, you can buy and sell that all the time, and so you get, um, liquidity on that really easily. 70 00:10:14.520 --> 00:10:21.880 Um, there is some concern like, "Oh, no, will I have capacity next month?" But it's very different than if you're interrupting a workload right now. Um, the way that the interruption happens matters. 71 00:10:22.080 --> 00:10:31.520 So an example is if you're on a cluster and the cluster doesn't ha- have as amazing storage as you want it to have. Um, so an example is, like, do you have a WEC or a VAST system under the hood? 72 00:10:31.900 --> 00:10:43.360 Um, and do you offer that? And, like, what's the size of that? Um, [lip smack] uh, this changes how you respond to, um, being preempted or removed off the cluster. Um, meanwhile, the location of that storage matters. 73 00:10:43.700 --> 00:10:53.120 So, um, if you're able to move that data

from one cluster to another, um, quickly, then maybe you might be able to spread across multiple different clusters. 74 00:10:53.580 --> 00:11:03.200 Um, and you can be interrupted over here and then pop back up over there. Um, to make that work, you also need, um, like a, a fast transit path between one cluster and another. 75 00:11:03.320 --> 00:11:12.300 Typically, this is implemented with a WAN, but most of these clusters don't necessarily have, um, like- Wi- Wide area network? Yeah, a wide area network. So, um, basically cables that connect the clusters together. 76 00:11:12.460 --> 00:11:23.840 A private internet is another way to think about it. Um, a lot of clusters that are being spun up today are like, uh, they're not attached to, um... [lip smack] Traditional CPU clusters were, um, 77 00:11:24.980 --> 00:11:34.780 next to the customer for latency reasons. In GPUs, they're next to the power, um, wherever the, like, you know... You put the data center next to the power rather than next to the customer. 78 00:11:35.840 --> 00:11:41.660 Um, this means that a lot of stuff is getting put in places that previously they weren't expecting data centers there. 79 00:11:41.960 --> 00:11:48.020 And so you may not have, um, as, um, [lip smack] excellent of a transit path, um, from one to the other. So this, this starts to matter. 80 00:11:48.480 --> 00:12:00.860 Um, other examples are, um, weird shit where we theoretically support but don't actually, um, uh... Like this is a thing that we're thinking about but haven't done basically. 81 00:12:01.360 --> 00:12:06.480 Um, the early GB200s broke, the early H100s broke. 82 00:12:06.960 --> 00:12:16.280 Um, and so if you were buying H100s or you were buying GB200s, you might really want the ones that were deployed after a certain time because you're not getting the first shitty ones off the rack. 83 00:12:16.380 --> 00:12:26.640 And so if you're a super sophisticated customer, you may be willing to pay more, um, because for that- Oh, so it's, it's about, like, what cohort They were, they were produced- Yeah... and they get better over time? 84 00:12:26.960 --> 00:12:31.140 So, like, uh, the first ones off the rack maybe had some fault or some problem. Right. 85 00:12:31.180 --> 00:12:41.260 And so if you were the ones who got the first ones, um, or, you know, first ones from, um, the, the first shipment of them, um, then yeah, they would, they would be broken or, um, in some way. 86 00:12:41.420 --> 00:12:50.100 I assume the sort of process is, like, look, there's defects everywhere. Like, it, it just happens, right? Totally. And so, like, you get a, you get a new shipment off of, from TSMC, whatever- Mm... 87 00:12:50.660 --> 00:12:56.780 and you just kind of run your tests and, like, you, you figure out which ones, which ones are bad, and you throw them out. And isn't that how it works? 88 00:12:56.860 --> 00:13:11.780 I, I, I didn't know about this element of cohorting where, like, let's say Q4 2026,

uh, B- B300s are- Mm... way worse than 2027 B300s. Yeah. So- [laughs]... um, I'll, I'll give an example. Um, you're describing burn-in. 89 00:13:12.060 --> 00:13:20.440 Yes. Um, so when you get a cluster, um, a good cloud- Yeah... what it does is it tries to kill all the GPUs, um, before they ever get in the hands of the customer. 90 00:13:20.960 --> 00:13:27.659 Um, and then, um, the GPUs that are broken or the nodes that are broken, um, you get RMA'd. Yes. You basically go and request a new- You return it, yeah... 91 00:13:27.760 --> 00:13:35.160 you return it, and you get a new one, and there's a lead time to that, but whatever. You'll get- Yeah... you'll get some back. Um, and then the customer never has to experience this. 92 00:13:35.560 --> 00:13:49.080 Um, the, um, thing I'm referring to is that, um, like, in early H100s there were, like, firmware bugs that may not even be related to the specific... Like, they may not be Nvidia. 93 00:13:49.380 --> 00:13:58.660 They might be like Supermicro has a firmware bug in the, um, in the box that you bought, um, and it applies to all of them. [laughs] And it doesn't matter. 94 00:13:58.700 --> 00:14:08.520 Like, um, the solution to that was you need to get them to update their firmware, um, or they need to ship a new thing, um, in every new box. Yeah. Well, con- And so it happens... 95 00:14:08.540 --> 00:14:16.320 cognition for, for, like, the, the B300s that, that, that we got, um- Yeah... and, like, you had to use Devin to go fix it. [laughs] Yes. Right. So, like, different types of problems like this can happen. 96 00:14:16.880 --> 00:14:23.500 Um- Interesting. Okay. So I'm, like... Okay, look, that seems small compared to the overall... Like, I just wanna, like... I want people to understand the, the shape of this market. 97 00:14:23.900 --> 00:14:33.460 Um, you mentioned, like, preemption, you know- Mm-hmm... and, and, you know, we talked about spot instances and how, how that, uh, uh, the analogy is. Yeah. And then there's also, you know, booking w- a month out. Mm. 98 00:14:33.500 --> 00:14:45.680 How far out do those, do these markets go? And is, is real time or sort of preemption more interesting to you versus, uh, far, be- being further, further out? Um, to answer your first question- Yeah... 99 00:14:45.760 --> 00:14:52.360 um, three to five years, um- Wow... we're starting to see- Like, in, like, month by month or quarter by quarter, or how is this? Are you asking how the cash flow works? 100 00:14:52.380 --> 00:15:00.020 Like, the, the, the contracts, like, the fu- the futures contracts, right? Like, a- again, like, I'm a former con- contracts, uh, derivatives trader. In the, in the market itself- Yeah... 101 00:15:00.080 --> 00:15:08.180 um, like as in the broader market- Yeah... um, clusters are financed three to five years. Okay. Um, so the, the, the payments are- Totally get it... over that period of time. Yeah. 102 00:15:08.240 --> 00:15:18.440 Um, there is also, like, blocks of

capacity in, in our order book, um, which can extend any arbitrary time. Yeah. Um, there's some stuff that we have to do, um, for, um, the cash flow on that. 103 00:15:18.700 --> 00:15:25.000 Right now what we do is we just ask people to pay entirely upfront for a block. Um, we don't ask them to do that for a lease for a longer term contract. 104 00:15:25.280 --> 00:15:34.140 This gets into something that I definitely need a whiteboard to explain. [laughs] Um, but, um, you had another question I was about to answer, and I forgot what it was. Uh, do you want real time or do you want- Yeah... 105 00:15:34.180 --> 00:15:45.840 like, do you want more, uh... What's more interesting to you, real time or long- longer out? So we care more about longer out because we're trying to build a forwards curve that has a lot of liquidity on it. Um, the, um, 106 00:15:47.320 --> 00:15:56.740 these are kind of the same thing and kind of not. Um, so what we do is we can say you can buy an hour out. 107 00:15:56.780 --> 00:16:08.860 You can buy right now to the next hour, or you can buy right now to the next three years, and anywhere in between. That's wild. I know. Um, and so the, um, you know, or five years, technically. Um, the... 108 00:16:11.200 --> 00:16:17.820 There's a certain difference that happens that's dependent for each customer. Basically, like, if I have a workload that is, um, 109 00:16:18.740 --> 00:16:26.880 uh, that's batch inference, basically, where I can be interrupted at any point, I don't really care, um, then 110 00:16:28.180 --> 00:16:40.860 my ability to be cut off, um, is probably, like, I need an hour of time to know, um, that I'm gonna be cut off, and then I'm good. Or I can even n- be n- notified in, you know, a minute or two. 111 00:16:41.360 --> 00:16:51.760 Um, but if I have, um, live inference customers who are expecting me to be online, um, then you can interrupt me, but in a month, maybe two months- Mm... maybe three months. Yeah. 112 00:16:51.840 --> 00:16:57.840 Um, and then I can adapt to that, um, capacity over that period of time. Um, but you cannot interrupt me in a minute. 113 00:16:58.320 --> 00:17:13.020 Um, and so those two things are priced differently, and the reason why they're priced differently is because what is the price you would pay, um, in order to interrupt your batch job, it's very different than the price you would pay to interrupt your, um, live job. 114 00:17:13.360 --> 00:17:23.080 If you have a live job, you're, you are probably paying an SLA when you break for your customer. Um, does that make sense? Yeah, I see. So it's, this is actually the market for breaking leases. 115 00:17:23.320 --> 00:17:31.180 These are the market for breaking leases, basically. Oh, my God. And then there's also the market for future leases or, like, for leases that are, um- Oh, I thought this was, like, a subletting situation. 116 00:17:31.360 --> 00:17:41.280 It's, this is like- Oh, the-... a lease break... so one way to think about it, um, is in power markets there's actually two markets. Um, there's, like, the day ahead market and there's the live market. 117 00:17:41.640 --> 00:17:55.000 Compute kind of has the same thing, um, where it has capacity that is coming online, um, or capacity that's about to, um, arrive to you, um, and then capacity that is currently in use, and these have different price dynamics. 118 00:17:56.040 --> 00:18:07.060 Okay. Interesting. Um, backwardation or contango? [laughs] Backwardation or contango? Do you... Uh, so do, uh, is compute further out priced higher than spot or lower than spot? 119 00:18:07.660 --> 00:18:18.740 Um, is compute f- further out priced- Does it, does it go like this or does it go like that over, uh, you know, over, like, one year, one year out, two year out? Do- Oh, this is an ex- both at times... 120 00:18:19.040 --> 00:18:27.616 I, I, I find it fascinating that you don't know. Yeah. I mean, both. It, it depends on who you are. Um- Okay... so, uh, I'll make the case Um, wait, so you're asking is compute like right now? 121 00:18:27.616 --> 00:18:39.316 So the compute, like, the same unit of compute today- Yes... uh, maybe not today, maybe a month out versus one year out versus two year out- Mm... is that, is that curve going up or going down? We see a, like, a hump. 122 00:18:39.596 --> 00:18:48.956 Oh, my God. Um, okay. So- Explain. Um, uh, like I think this will make intuitive sense. It's basically, like, if I have milk and it is about to expire, I will lower the price on it. Yes. 123 00:18:49.076 --> 00:18:54.096 Um, and so that's basically what we see. So it's, it's the Nvidia release cycle, basically, right? 124 00:18:54.456 --> 00:19:03.956 Uh, well, we're more saying, like, if there's a block of compute and it's approaching now, um, it can't be sold in the past, and so it has to drop its price in order to- Yeah... clear at some price. Yeah. 125 00:19:03.976 --> 00:19:13.276 And so it- initially you see this, like, drop where, um, stuff that's sold right now, anything that's on the market right now, um, is something that hasn't been sold, and so therefore the price will drop. Mm-hmm. 126 00:19:13.376 --> 00:19:21.376 Um, and then it goes up, um, to where it wants to be. Mm-hmm. Um, and then maybe compute in the future is worth less, um, because people don't want... 127 00:19:21.416 --> 00:19:28.176 Like, people would like the compute now, and so they're sort of paying for the shorter lead time. Mm. Um, and they pay less for the compute that's off into the future. 128 00:19:28.596 --> 00:19:31.236 This is what I think a normal, healthy curve looks like. Yeah. 129 00:19:32.056 --> 00:19:46.556 The place where I'm like, "Oh, man, no shit," um, is that, um, right now if you took a B300 and you tried to run inference on it, um, you can make $40 an hour or so, um, selling that as inference. Any model. Yeah. 130 00:19:46.616 --> 00:19:54.216 Any model, basically. Um- [laughs] And so that's if you use it at

100%, and maybe if you only have, like, 50% utilization or something, so like $25 an hour. Yeah. 131 00:19:54.576 --> 00:20:02.716 So $25 an hour, meanwhile, you can buy B300s at, like, $5 to $6 an hour. O- one of these numbers has to change [laughs]. Yeah. Um- It's a huge arbitrage. It's a huge arbitrage. 132 00:20:02.716 --> 00:20:05.456 And you, you mentioned this in your, your talks a lot. Yeah. So, um, 133 00:20:06.376 --> 00:20:23.676 a bunch of people are basically looking at this and thinking, um, "If I think prices are going to go up, I might be willing to pay more than what I previously thought in the future, um, because I might think, 'Okay, I can buy a five right now, but if I can't buy any more, then maybe I will be willing to buy even more in the future just to lock it in if it's gonna come online later.'" 134 00:20:24.076 --> 00:20:33.696 So we see the other side as well. Um, kind of depends on who you are. Cool. Uh, so, um, is it... Okay, and, and just, I just really wanna make super clear on this. Of course. 135 00:20:33.736 --> 00:20:40.256 Do you care about the different generations of chips and... or do you just care about overall flops and you spread it among whatever chips you want? 136 00:20:40.656 --> 00:20:46.896 W- uh, we care about different generations of chips because- Yeah, you care. Yeah, yeah. I think, like, I o- only, I guess, do your customers care? Yes. Right? Okay. Oh, wait. 137 00:20:46.956 --> 00:20:53.976 Uh, when you say, uh, different generation chips, do you mean, like- So, like, B300s versus-... B300 versus B200s? Yeah, yeah, exactly. Yeah. Um- Or, or, like, even AMD or whatever, right? 138 00:20:54.036 --> 00:21:04.216 Uh, yes, I think people totally care about the generation of chips. Um, uh, I think part of this is because of things like vRAM, um, and part of this is because of, like, um, uh, larger ND link domains- Yeah... 139 00:21:04.356 --> 00:21:11.496 and things like that. So there's just no way to abstract all this to, like, some generalized unit of compute that you can sort of mush around or whatever you like. 140 00:21:11.756 --> 00:21:19.196 I think, like, when you were saying oil, even oil isn't oil. Yeah. Oil is actually just, like, a bunch of different types of oil. Yeah, like jet fuel or car fuel, whatever. I think it's the same thing. 141 00:21:19.596 --> 00:21:27.336 Um- You're gonna get a bunch of different types. It is, but you have oil refineries- Yeah... that you can s- yeah, effectively are black boxes. You can put oil in and get it, like, whatever out. Uh-huh. 142 00:21:27.396 --> 00:21:37.696 Um, i- if compute it is so, uh, heterogeneous, I, I guess is the- Uh-huh... is the word, is that you just have to expose the underlying complexity to your end customer, which is bad for you- Yes... as a market maker. 143 00:21:38.196 --> 00:21:47.716 [laughs] Maybe. Like, you want to identify the one, like, choke point or, like, the, I, I don't know what the better word for choke point is.

Like a sh- shelling point- Yeah... 144 00:21:47.756 --> 00:21:55.376 where everyone agrees, like, okay, like this is kind of the standard, and we'll- Yeah... we'll all agree to this standard and, uh, you know, it'll be somewhat inefficient. Yes. 145 00:21:55.716 --> 00:22:05.096 Uh, so we've been able to reduce the, um, the, the problems to a certain level, but they still exist. Yeah. Y- y- like your job is, like, dimensionality reduction almost. Yes. 146 00:22:05.396 --> 00:22:12.496 W- we reduce a lot of the dimensions, but nonetheless, there are still dimensions. Yeah. Like, it's still gonna exist. Um- Do, do you know how many SKUs you have? [laughs] So, 147 00:22:14.196 --> 00:22:24.976 um- Like five times a hundred times, like, five thousand. That equals... I don't know. Like, what's... [laughs] W- we actually made SKUs, like, a, a concept, um, that... 148 00:22:25.416 --> 00:22:35.336 And the goal is to make it so that somebody else can make their own SKU. So if I wanna buy Blackwells, I can say, "I'm willing to buy Blackwells at $6 an hour or less." Uh-huh. 149 00:22:35.436 --> 00:22:45.756 "And you can match me against GB300s, or you can match me against B300s, and I don't care which Blackwell, as long as it's less than $6 an hour." Um- But you just said you care about memory and other stuff. Totally. 150 00:22:45.796 --> 00:22:49.876 And CPUs and- But not every customer does. So some customers, like, um, just can work on Blackwells, whatever it is. 151 00:22:50.196 --> 00:22:56.616 Some customers are super hyper-specific, and they need this exact thing, and they're gonna pay more for that because the market's gonna charge them more. Okay. Um, and that's what we do. 152 00:22:56.936 --> 00:23:08.716 Um- So, and so people use your APIs to programmatically define what they want, and then y- you match that with whatever composition of sellers, uh, they might, they might, they might be a- able to fulfill. Yes. 153 00:23:08.796 --> 00:23:19.356 Um, uh, with the caveat that having a named query is the new thing we're doing. Um, but the query system as a concept is how the order book works right now. Okay. Um. Interesting. Um, all right. 154 00:23:19.396 --> 00:23:24.556 So, uh, y- you know, I, I think that's, like, a, a lot of, like, I guess, nitty-gritty. Have you, have you... 155 00:23:24.896 --> 00:23:33.836 Are there anything about, like, I guess, that you wanna educate people on, whether it's financial people or engineers, about the state of the compute trading market as, as you've- Mm... developed it? 156 00:23:33.896 --> 00:23:44.716 Because e- effectively, you, you guys are defining it. Um, mainly just the requirement for physical settlement. Yeah. Um, we think that if you... We s- we spent three years learning why you need physical settlement. 157 00:23:44.716 --> 00:23:56.136 [laughs] Um, and, um, our entire conversation today is, um, that it is, these things are not as fundable as you think. We're gonna try and make them fundable. That's the industry's job to do that. 158 00:23:56.516 --> 00:24:00.596 Um, but without that, you run a lot of risks. 159 00:24:00.876 --> 00:24:13.356 Um, if you try to build, um, too much financialization before you've, the industry has had time to build out the, um, the physical settlement, then I think what you have is you have, um- 160 00:24:14.704 --> 00:24:25.664 A market that's very easy to manipulate, and a market that has a lot of basis risk, where, um, the price you may be hedging against actually has nothing to do with the price that, um, the market, um- Yeah. Yeah, yeah... 161 00:24:25.724 --> 00:24:26.844 you can actually buy the capacity for. 162 00:24:26.964 --> 00:24:38.704 Yeah, so for, for listeners, uh, we, we talked about this briefly last time, but, um, uh, f- people are sh- often shocked to find out that, let's say the market for credit default swaps is 12 times the m- the, the size of the bonds that they insure. 163 00:24:38.744 --> 00:24:48.424 Mm-hmm. Uh, something like that. Or, like, uh, the, the markets for mortgage, um, sort of reinsurance or, uh, all that is, is, is larger than the underlying physical settlement. 164 00:24:48.464 --> 00:24:58.004 So th- that's why physical versus cash settled. Yeah. And, and, like, basically cash settlements are, like, exactly the kind of, uh, the side bet that you see in The Big Short. Mm-hmm. 165 00:24:58.084 --> 00:25:06.604 Um, where, uh, they, they may, uh, may or may not have anything to do with the underlying thing itself. Yeah. And it may, may be used as a proxy for overall other stuff. Yes. That... 166 00:25:06.664 --> 00:25:17.264 And you don't want to- that to go out of control. Yeah. So to be clear, we think, um, cash settlement as a future concept is a great idea. Yeah. Um, we just think it should be derived from physical settlement. Yeah. 167 00:25:17.344 --> 00:25:26.864 Um, because as you said, it's a side bet. Um, and it would be kind of like if you... I mean, it'd be kind of like if you made a cash settle in the future before you made the physical settlement. 168 00:25:27.264 --> 00:25:33.224 Um, it- Which I think people are kind of doing already. Uh, yeah, totally. Yeah. People are trying to make cash settlement before you make the physical settlement. 169 00:25:33.264 --> 00:25:42.244 And people do that for all sorts of things, including- Yeah... startup stock. Yes. Right? Like, you can buy and sell a sh- a made-up share of OpenAI, but it's- Yeah... nothing to do with the actual share. Yes. 170 00:25:42.244 --> 00:25:57.184 [laughs] Um, and lots of those go very poorly. [laughs] Um, like, it would be kind of bad if you made a, um... It would be bad if you started selling, um, the s- the derivative on a stock that doesn't exist yet. Um... 171 00:25:57.204 --> 00:26:02.584 [laughs] Yeah. Yeah. Well, so I mean, one thing that we do... Uh, so I used to be a currency trader, right? Yeah. And, and interest rates as well. 172 00:26:02.924 --> 00:26:09.484 And one, one

thing we do is you have a, a board that gets together and says, like, "This is the London Interbank Offered Rate, LIBOR." Yes. 173 00:26:09.484 --> 00:26:20.504 "And we all agree, like, this is the, the, the monthly fixing, and we all trade to that fixing." Yes. Uh, i- is that even meaningful at some point in three years from now? [laughs] So critically, LIBOR was manipulated. 174 00:26:20.804 --> 00:26:29.944 Yes. Um, our concern at the moment is- 'Cause it's 12 banks. Yeah. Yes. Um, well, our, our concern is there's a very similar problem, um, where you can manipulate an index. 175 00:26:30.364 --> 00:26:43.324 Um, our impression is why you can manipulate that index is because a lot of the indexes are based on a very small amount of data that has, um, uh, not, not a lot of correlation with the actual price of buying a new reserve contract. 176 00:26:43.864 --> 00:26:51.904 Um, that's what we're concerned about. [laughs] Okay. Interesting. Um, all right. I, I think, I think that's the, the, the rough state of things. Uh-huh. 177 00:26:51.984 --> 00:27:01.964 Uh, um, I think the, the other part of just, just today is, uh, you, you, you, you talked with Oliver, you, you and, uh, you're doing an event with Jump, and your, your friends with Silicon Data. 178 00:27:02.284 --> 00:27:09.904 Uh, what are you learning from all these other players? Like, who, who are the major players that you would wanna call out as, like, doing interesting things? Ooh. Um, 179 00:27:11.024 --> 00:27:15.924 uh, so we're big fans of Silicon Data and Compute Change. Um, I think they were also very early to this. 180 00:27:16.264 --> 00:27:26.624 Um, and I think they are doing, like, a reasonable shot at the, a path that we think, um, that we'll likely end up being partners with. Um- And can you dist- distinguish, like, your, your different paths? 181 00:27:26.864 --> 00:27:35.584 So, um, the general gist is they're doing cash settlement, we're doing physical settlement. Um, we think both can exist and should exist. Mm-hmm. Um, and that they're sort of mutually beneficial to each other. 182 00:27:35.924 --> 00:27:45.224 Um, and so totally fine. Um, so I think they are one of the earliest in the space, and I think they are, um, very clearly took the bet, um, well before anybody else did, and I think that's, like, very honorable to do. 183 00:27:45.744 --> 00:27:55.204 Um... Um, you mentioned, uh, Jump or- Semi Analysis, Jump. Like, these are the players emerging. Like, as far- Yeah... like, you were my first introduction to this. Yeah. 184 00:27:55.224 --> 00:28:01.424 And then, and I see all these other guys coming in, and I'm like, "Okay, like it's, something's happening here." I don't know the landscape as clearly as you do. Yeah. 185 00:28:01.464 --> 00:28:09.544 So, um, the categories of folks that we've seen is there's folks that are trying to set better standards. So Semi Analysis, I think, is working to try and create better standards. 186 00:28:10.084 --> 00:28:16.504 Um, there's folks that are trying to do, um, residual value insurance, and so on. Um, there's a, a number of different companies in the space. 187 00:28:16.544 --> 00:28:20.344 I don't know any of them super well, but, um, I know, know this as a concept, and I think that's important. 188 00:28:20.844 --> 00:28:29.404 Um, there are folks that are trying to do, um, a deposit, um, uh, lending where, like, they'll front your credit risk for the cloud. 189 00:28:29.924 --> 00:28:40.964 There's an ecosystem of companies that are forming around how do we resolve the financial bottlenecks of compute. Um, and it is cool to see. I think e- effectively what you want in this ecosystem is buyers and sellers. 190 00:28:41.024 --> 00:28:51.844 You want an analyst, you want, uh... We said insurance or, uh, what, what's the- Uh, residual value insurance. Res- residual, yeah. Yeah. Um, and you, we want this, like, healthy ecosystem out there. Mm-hmm. 191 00:28:51.924 --> 00:28:56.784 Um, but obviously you guys are owning the problem of physical settlement, which, which is really interesting. 192 00:28:57.164 --> 00:29:05.804 Um, okay, so I mean, we can go into, like, the, the hard tech side because obviously you are also hiring- Yeah... uh, for extremely cracked engineers. Yeah. Uh, describe your stack. 193 00:29:06.284 --> 00:29:13.264 Um, we do all the things a cloud does- [laughs]... without being the cloud. Um, as in we don't own the hardware. 194 00:29:13.544 --> 00:29:18.664 Um, the reason why we don't own the hardware is because if we did, we would be able to manipulate the forwards market. 195 00:29:19.244 --> 00:29:30.064 So we have to be Switzerland, we have to be independent, um, which creates a lot of sort of strange things about the company. Um, but in order to do physical settlement, you have to do site visits. 196 00:29:30.084 --> 00:29:38.044 You have to visit the data center. Um, you have to, um, you have to design the bill of materials and ensure that, um... 197 00:29:39.024 --> 00:29:51.844 You, you have to basically do, like, supply chain consulting, um, where you have relationships with different OEMs that, um, ensure that things get delivered on time. Um, then you have to, um, do cluster architecture. 198 00:29:51.944 --> 00:30:00.004 Um, uh, so you're sort of designing the cluster itself. Um, then you're, um, rolling out racks, or you're working with partners to roll out racks and supervising that. 199 00:30:00.464 --> 00:30:13.024 Um, then you're doing, uh, backplane management, so you're taking over the, um, uh, PDUs and the switches and the DPUs and, um, UFM and BMC, and you're operating, um, the whole, um, harness around the nodes that you're selling. 200 00:30:13.044 --> 00:30:23.968 When you say you, as in SF Compute employees? Yeah. Wow. All right. Um, so, uh- When you buy a node from a cloud, um, there's the bare metal. Yeah. But there's all this shit around the bare metal. Yeah, yeah. 201 00:30:24.048 --> 00:30:37.328

Um, and in order to make that bare metal node work, you have to operate everything else. Then we- Mm. Because I just like, traditionally the job of the data center is to extract some, some of this away from you. Yeah. 202 00:30:38.508 --> 00:30:44.748 We're, we're the ones that do that. You're, you're [laughs] Okay. Yeah, right. [laughs] Um, but you're not, you're not in the data center business. You're not the, sort of Equinixes of the world. 203 00:30:45.108 --> 00:30:56.848 We're, we're not in the data center business, sort of. Yeah. Um, what we do work with is... So we offer supercomputing as a service. Um, and what this means is folks come to us and say, 204 00:30:57.808 --> 00:31:10.128 "You know, I've got X," where X is I've got, you know, X megawatts here, and maybe sometimes they mean I've got a data center, um, that's, like, I would like to turn into a GPU cluster. 205 00:31:10.468 --> 00:31:18.728 Like, my options are I could sell the GPU clust- or I could sell the data center to somebody, or I could become the cloud. And then we go to them and say, "Great, um, we will turn you into the cloud. 206 00:31:18.888 --> 00:31:24.268 We're the supercomputing part. You're the data center cl- Let's partner together, and then we will sell, and all make money." Um, 207 00:31:25.288 --> 00:31:33.188 sometimes it's people who have something that could turn into a data center, in which case we're gonna basically project manage that up to the layer. 208 00:31:33.648 --> 00:31:45.688 Um, and then we will work with those far- We'll work with those folks to turn their powered land, um, or whatever they have, um, into a data center by working with partners. Um, and then we take over at- Okay... 209 00:31:45.708 --> 00:31:55.748 the supercomputing part. Okay. This requires us to have some sort of data center expertise in-house, but we don't self-perform builds. Um, so yeah. Yeah. 210 00:31:55.808 --> 00:32:06.328 So it's, uh, somewhat modular because you- you have different scenarios you need to deploy under. Yeah. Yeah. Uh, someone once described us as like, um, uh, like Palantir for compute, um- [laughs]... because 211 00:32:07.268 --> 00:32:21.048 it's like these, each time it's this new rodeo. Um- Yeah... but-- And we like, we, we deeply partner with folks on both sides. Yeah. Uh, in primarily the US? Primarily the US, but we, uh, operate around the world. Yeah. 212 00:32:21.108 --> 00:32:28.028 Uh, at some point, does SF Compute become a misnomer [laughs]? Uh, so when- [laughs] So we're deploying clusters all around the world. Yeah. 213 00:32:28.068 --> 00:32:37.868 And when we do that, we're working with people who themselves are trying to be clouds all around the world. Um, they look like sovereign compute players. Um, and our pitch to sovereign compute is amazing. 214 00:32:38.108 --> 00:32:48.508 Uh, let's help you set up a cloud in your own country. Um, and we will bring the San Francisco offtake. We'll bring the San Francisco expertise. Ah. We'll set up the clusters. Um, that's our whole business. Ah. 215 00:32:48.888 --> 00:32:57.368 Um, we're gonna make it financially- Who are, who are your main, uh, your biggest announced partners that you can name? Um, nobody I can name- Really?... basically. Wow. Okay. 216 00:32:57.408 --> 00:33:05.988 We will have some announcements, um, in a week or two, um, that we expect to come on. Yeah. All, all things sort of, uh- Sure, sure. Good luck. Yeah. 217 00:33:06.428 --> 00:33:13.648 Uh- No, I mean, uh, I, I think that's really interesting for every country to like, who's, who's like investing in sovereign AI to- Yeah... uh, figure this out. 218 00:33:13.768 --> 00:33:21.948 Um, and it- it's interesting that like SF is, is still sort of like that brand name and, uh- Mm-hmm... that, that is, that is very marketable. Uh, so, um, I think 219 00:33:22.868 --> 00:33:32.408 one of the interesting things is just thinking through this market. Mm-hmm. Um, NVIDIA is obviously like the 800-pound gorilla in the room. Yeah. Um, why don't they-- Why didn't they make SF Compute? 220 00:33:32.448 --> 00:33:39.708 And again, like I, I, you know, the answer's gonna be market manipulation, but like let's just really get into the incentives of the multiple players. Uh, y- yeah. 221 00:33:39.748 --> 00:33:49.448 So our impression is that NVIDIA can't make it, but benefits tremendously from it existing. The reason why they can't make it is because they would then not be independent enough for people to trade on it. 222 00:33:49.608 --> 00:33:55.848 If you don't trade on it, um, then you don't have enough liquidity for lenders to finance against the forwards market. Yeah. 223 00:33:55.908 --> 00:34:04.428 This is also why Crusoe and CoreWeave and Lambda, why the hyperscalers can't make it, why Base10 and Fireworks and Modal can't make it, 'cause they have too much demand on that side. 224 00:34:04.948 --> 00:34:19.048 Um, this is a thing that benefits the entire industry because it makes the financing costs go down. But nobody can make it unless they give up the ability to do all the big inference money and all the big cloud deals. 225 00:34:19.548 --> 00:34:26.088 Um, you have to kill your business in order to make this. Mm. Yeah. That's why nobody does it. It's like structurally they're not set up to do that. Yeah. 226 00:34:26.148 --> 00:34:35.888 Uh, and you're, you're make, you're basically reducing friction, and that's all your revenue. Mm-hmm. Um, you've announced some benchmark- uh, some milestones. Actually, I forgot to s- say that up front. 227 00:34:35.928 --> 00:34:48.068 Uh, can you repeat what they are and, uh, you know, just take a victory lap a little bit? Uh- [laughs]... close to several hundred million dollars of volume. Um, we have a significant number of new clusters on the way. 228 00:34:48.328 --> 00:35:01.808 Yeah. Um, and then the things that I think I'm most excited about, um, is we've, um, shipped a whole bunch of new features, um, that make SF Compute very viable for platform as a service companies. 229 00:35:02.288 --> 00:35:14.068 Um, an example is, um, we allow customers to subdivide their InfiniBand fabric, um, without having to ask us. So many clouds, when you go to buy them, they will give you one InfiniBand fabric. 230 00:35:14.488 --> 00:35:23.128 Um, and then if you want to re- redraw the lines of that InfiniBand fabric, uh, you need to contact them, and they will then do something for you. 231 00:35:23.548 --> 00:35:35.068 Um, we now can do that on the fly for customers, um, which I think is really great, um, because it interfaces with the market really well, um, where you want to be able to redraw things in order to scale up and down your clusters. 232 00:35:35.568 --> 00:35:41.788 Um, we don't- Would you describe this as like you're virtualizing the data center and making it programmable? Yeah. We're virtualizing the data center and making it programmable. Yeah. 233 00:35:41.828 --> 00:35:51.768 So all the stuff that you would have gotten if you had owned the hardware, we're trying to virtualize that. Yeah. Um, which I think is, um, exciting to us. It's funny, like the last time this happened was like the cloud. 234 00:35:52.008 --> 00:36:02.928 Yeah. Like the [laughs] Yeah. Uh, which is, which is really cool. Uh- And then f- uh, uh, finally, uh, you know, I think, uh, what is your call to action for, for people, um, you know? Is it, is it engineers? 235 00:36:03.008 --> 00:36:04.988 Is it like, "Give me your supply"? Like what, what is it? 236 00:36:05.448 --> 00:36:13.888 Um, if you would like to own a GPU cluster and you want the residual value from it, you wanna have it for your portfolio, um, we would love to work with you in order to build that GPU cluster. 237 00:36:14.288 --> 00:36:20.188 We will take over, and we will, uh, do the supercomputing portions of that and ensure that things go right. We own the SLA. 238 00:36:20.548 --> 00:36:28.668 And then if it turns out that that was a bad idea, that you need out of that cluster, we can immediately sell that cluster to somebody else so it doesn't hit your balance sheet. Very simple pitch. Mm-hmm. 239 00:36:28.688 --> 00:36:43.148 Well, thank you so much for your time. You're, you're doing a great service to the industry, and I just am happy to be your friend. [laughs] I appreciate it. All right. Thanks. That's it. [upbeat music]