Agent Engineering with Pydantic + Graphs — with Samuel Colvin

WEBVTT
1
00:00:00.120 --> 00:00:14.360
Hello, AI engineers. In 2023, we heard that Pydantic is all you need at the first AI Engineer Summit. In 2024, we heard again that Pydantic is still all you need at the World's Fair.
2
00:00:15.320 --> 00:00:28.420
Since then, that opinion has become consensus, with OpenAI and others officially recommending Pydantic in their SDK and docs, as we covered on our Michelle Parkras episode on OpenAI's structured output.
3
00:00:29.940 --> 00:00:47.120
Now, in 2025, we're back to complete the epic trilogy with Samuel Colvin, creator of Pydantic and LogFire, who has just launched his Pydantic AI framework that will extend the principles and sublime developer experience of Pydantic to your building of agents.
4
00:00:47.840 --> 00:00:55.600
We are very proud to also announce the new workshops track for the second AI Engineer Summit in New York City on February 22nd.
5
00:00:56.420 --> 00:01:06.220
This is a brand-new day added to the schedule because we had so many great workshop signups and is available at no extra charge to everyone who already bought tickets.
6
00:01:07.040 --> 00:01:22.080
We are honored that Samuel will be teaching his first-ever workshop on Pydantic AI there, alongside other luminaries from OpenAI, Anthropic, DeepMind, Vercel, AWS, Neo4j, Letter, Solana Foundation, and more.
7
00:01:23.600 --> 00:01:28.020
Twenty-five tickets remain for the workshop day. Full schedules have now been released.
8
00:01:28.900 --> 00:01:39.620
Our new website now lists our speakers and talks from DeepMind, Anthropic, OpenAI, Meta, Jane Street, Bloomberg, BlackRock, LinkedIn, and more.
9
00:01:41.060 --> 00:01:56.260
Sponsorships have sold out, but you can still apply for a late ticket at apply.ai.engineer. See you in two weeks. Watch out and take care. [upbeat music] Hey everyone, welcome to the Late in Space podcast.
10
00:01:56.540 --> 00:02:09.080
This is Alessio, partner and CTO at Decibel Partners, and I'm joined by my co-host Swyx, founder of Small AI. Good morning. Uh [chuckles] and, uh, today we're very excited to have Sam Colvin join us from Pydantic AI.
11
00:02:09.240 --> 00:02:14.560
Welcome. Thank you so much for having me. Yeah, it's great to be here. Sam, I heard that Pydantic is all we need. Is that true?
12
00:02:15.200 --> 00:02:22.400
I would say you might need Pydantic AI and LogFire as well, but, um, it gets you a long way, that's for sure. Pydantic almost basically needs no introduction.
13
00:02:22.460 --> 00:02:35.220
It's, you know, it's, uh, almost three hundred million downloads in December, and obviously, uh, in the previous podcasts and discussions we've had with Jason Lu, uh, he's been a big fan and promoter of Pydantic and AI.
14
00:02:35.740 --> 00:02:48.220
Yeah, it's, it's, it's weird because obviously we didn't-- I didn't create Pydantic originally for, for uses in AI, obviously it predates LLMs, but it's like we've been lucky that it's been picked up by that community and, and used so, so widely.
15
00:02:48.680 --> 00:02:58.380
Actually, maybe, maybe we'll hear it right from you. Like what is Pydantic and maybe a little bit of the origin story. The best name for it, which is not quite right, is a validation library.
16
00:02:58.420 --> 00:03:03.960
And I-- we get some tension around that name because it doesn't just do validation, it will do coercion by default.
17
00:03:04.420 --> 00:03:17.680
We now have strict mode, so y-you can disable that coercion, but like by default, if you say you want, uh, an integer field and you get in a string of one hundred and twenty-three, it will convert it to a hundred and twenty-three and a bunch of other sensible conversions.
18
00:03:18.080 --> 00:03:25.780
And as you can imagine, the, like, semantics around exactly when you convert and when you don't is complicated. But because of that, it's more than just validation.
19
00:03:26.140 --> 00:03:35.940
Back in 2017, when I first started it, the, like, different thing it was doing was using type hint to define your schema. That was controversial at the time. It was, like, genuinely disapproved of by some people.
20
00:03:36.000 --> 00:03:47.460
I think the success of Pydantic and libraries like FastAPI that build on top of it means that today that's no longer controversial in Python, and indeed lots of other people have, like, copied that route.
21
00:03:48.040 --> 00:03:56.660
But yeah, it's a data validation library that uses type hints for the, for the most part, and obviously does all the other stuff you want, like serialization on top of that. But yeah, that's the core.
22
00:03:57.160 --> 00:04:06.180
Do you have any fun stories on how JSON schemas ended up being kind of like the structure output standard for LLMs, and were you involved in any of these discussions?
23
00:04:06.240 --> 00:04:10.400
Because I know OpenAI was, you know, one of the early adopters. So did they reach out to you?
24
00:04:10.600 --> 00:04:18.060
Was there kind of like a structure output council that in open source that people were talking about, or was it just at random? No, very much not.
25
00:04:18.120 --> 00:04:28.700
So I originally didn't implement JSON schema inside Pydantic, and then Sebastian, Sebastian Ramirez, FastAPI came along and, like, the first I ever heard of him was over a weekend.
26
00:04:28.760 --> 00:04:35.760
I got like fifty emails from him, or fifty, like, emails as he was committing to Pydantic, adding JSON schema long pre version one.
27
00:04:36.200 --> 00:04:47.480
So the reason it was added was for OpenAPI, which is obviously closely akin to JSON schema. And then, yeah, I don't know why it was JSON sc-- w- that got picked up and used by OpenAI.
28
00:04:47.920 --> 00:04:59.180
It was obviously very convenient for us because it meant that not only can you do the validation, but because Pydantic will generate you the JSON schema, it will-- it kind of can be one source of t- source of truth for structured outputs and tools.
29
00:04:59.580 --> 00:05:08.020
Before we dive in further on the, on the AI side of, of, of things, something I'm mildly curious about, and obviously there's, uh, Zod in, in JavaScript land.
30
00:05:08.600 --> 00:05:20.500
Every now and then there's a new sort of in vogue validation library that, that takes over for quite a few years, and then maybe like some-something else comes along. Is Pydantic done, like the core Pydantic?
31
00:05:20.640 --> 00:05:30.100
[chuckles] I've just come off a call where we were redesigning some of the internal bits. There will be a V3 at some point, which will not break people's code half as much as V2.
32
00:05:30.440 --> 00:05:40.000
As in V2 was the, was the massive rewrite into Rust, but also fixing all the stuff that was broken back from like version zero point something that we didn't fix in V1 'cause it was a side project.
33
00:05:40.440 --> 00:05:50.160
We have plans to move some of the-- basically store the data in Rust types after validation, not convert them to Python types.
34
00:05:50.520 --> 00:06:00.700
So then if you were doing like validation and then serialization, you would never have to go via a Python type. We reckon that can give us somewhere between three and five times, another three to five times speed up.
35
00:06:01.240 --> 00:06:12.828
That's probably like the, the biggest thing. Also, like changing how easy it is to basically extend Pydantic and define- How particular types, like for example NumPy arrays are, are validated and serialized.
36
00:06:13.288 --> 00:06:25.848
But there's also stuff going on, for example, Jitter, the JSON library that does-- in Rust, that does the, the JSON parsing has, uh, SIMD implementation at the moment only for AMD64. So there were like, we can add that.
37
00:06:26.128 --> 00:06:32.028
We need to go and add, add SIMD for other, other instruction sets. So like there's a bunch more we can do on performance.
38
00:06:32.528 --> 00:06:40.688
I don't think we're gonna go and revolutionize Pydantic, but it's gonna continue to get faster, continue, hopefully to get-- allow people to do more advanced things.
39
00:06:41.168 --> 00:06:48.448
We might add a binary format like CBOR for serialization, for when you'll just wanna put the data into a database and probably load it again from Pydantic.
40
00:06:48.508 --> 00:06:54.368
So, so there are some, some things that will come along, but like for the most part it will just get-- should just get faster and cleaner.
41
00:06:54.748 --> 00:07:10.208
From a focus perspective, I guess like as a, as a founder too, like how did you think about the AI interest rising, and then how do you kind of prioritize, okay, this is worth going into more the-- and we'll talk about Pydantic AI and, and all of that.
42
00:07:10.688 --> 00:07:18.348
What was maybe your early experience with LLMs, and when did you figure out, okay, this is like something we should like take seriously and focus more resources on it?
43
00:07:18.728 --> 00:07:27.208
I'll answer that, but I'll answer, which I think is like a, a kind of parallel question, which is Pydantic's weird because Pydantic existed obviously before I was starting a company.
44
00:07:27.568 --> 00:07:32.968
I was working on it in my spare time, and then beginning of '22 I started working on the rewrite in Rust basically.
45
00:07:32.988 --> 00:07:37.828
And I worked on that-- I worked on it full-time for a year and a half, and then once we started the company, people came and joined.
46
00:07:38.348 --> 00:07:42.468
And it's a weird-- it was a weird project because that would never get signed off inside a startup.
47
00:07:42.568 --> 00:07:51.168
Like we're gonna go off and three engineers are gonna work full-on for a year in Python and Rust writing like thirty thousand lines of Rust just to release open source free Python library.
48
00:07:52.128 --> 00:07:57.028
The, the result of that has been excellent for us as a company, right? As in it's made us remain entirely relevant.
49
00:07:57.188 --> 00:08:12.608
It's like Pydantic is not just used in the SDKs of all of the AI libraries, but I can't say which one, but one of the big foundational model companies when they upgraded from Pydantic V1 to V2, their number one internal metric of performance is time to first token.
50
00:08:13.008 --> 00:08:14.308
That went down by twenty percent.
51
00:08:14.528 --> 00:08:25.588
So you think about all of the actual AI going on inside, and yet at least twenty percent of the CPU or at least the latency inside requests was actually Pydantic, which shows like how widely it's used.
52
00:08:25.628 --> 00:08:32.028
So we've benefited from doing that work, although it didn't-- would've never have made financial sense in, in most companies.
53
00:08:32.508 --> 00:08:46.208
In answer to your question about like how do we prioritize AI, I mean, the honest truth is we've spent a lot of the last year and a half building good general purpose observability inside Logfire and making Pydantic good for general purpose use cases.
54
00:08:46.628 --> 00:08:58.768
And the AI has kind of come to us, like we just-- not that we want to get away from it, but like the appetite, uh, both in Pydantic and in Logfire to go and build with AI is enormous because it kind of makes sense, right?
55
00:08:58.808 --> 00:09:06.548
Like, if you are starting a new greenfield project in Python today, what's the chance that you're using GenAI? Eighty percent, let's say. [laughs] Globally.
56
00:09:06.608 --> 00:09:10.268
Obviously, it's like a hundred percent in California, but even worldwide it's probably eighty percent.
57
00:09:10.308 --> 00:09:22.908
And so everyone needs that stuff and there's so much yet to be figured out, so much like space to do things better in the ecosystem in the way that like to go and implement a database that's better than Postgres is a like Sisyphean task.
58
00:09:22.948 --> 00:09:30.768
Whereas building, uh, tools that are better for GenAI than some of the stuff that's about now is not very difficult, putting the actual models themselves to one side.
59
00:09:31.308 --> 00:09:36.808
And then at the same time then you release Pydantic AI recently- Yeah... which is a, um, you know, agent framework.
60
00:09:37.268 --> 00:09:46.568
And early on, I, I would say everybody like, you know, LangChain and like, uh, Pydantic, kinda like a first class support, a lot of dif-- these frameworks were trying to use you to be better. Yeah.
61
00:09:46.608 --> 00:09:55.428
What was the decision behind we should do our own framework? Were there any design decisions that you disagree with? Any workloads that you think people didn't support well?
62
00:09:55.748 --> 00:10:00.348
It wasn't so much like design and workflow, although I think there are some, some things we've done differently.
63
00:10:00.488 --> 00:10:10.008
I think looking in general at the ecosystem of agent frameworks, the engineering quality is far below that of the rest of the Python ecosystem.
64
00:10:10.048 --> 00:10:19.168
There's a bunch of stuff that we have learned how to do over the last twenty years of building Python libraries and writing Python code that seems to be abandoned by people when they build agent frameworks.
65
00:10:19.228 --> 00:10:26.148
Now, I can kind of respect that, particularly in the very first agent frameworks like LangChain, where they were literally figuring out how to go and do this stuff.
66
00:10:26.528 --> 00:10:32.448
It's completely understandable that you would like basically skip some standard best practice to kind of get something built.
67
00:10:32.648 --> 00:10:43.568
I am shocked by the like quality of some of the agent frameworks that have come out recently from like well-respected names, which it just seems to be opportunism and I have little time for that.
68
00:10:43.608 --> 00:10:50.548
But like the early ones, like I think they were just figuring out how to do stuff and just as lots of people have learned from Pydantic, we were able to learn a bit from them.
69
00:10:51.008 --> 00:11:00.608
I think from like the, the gap we saw and the thing we were frustrated by was the production readiness, and that means things like type checking, even w-- if type checking makes it hard.
70
00:11:00.648 --> 00:11:13.448
Like Pydantic AI, I will put my hand up now and say it has a lot of generics and you need to-- it's probably easier to use it if you've written a bit of Rust and you really understand generics, but like-- And that is-- We're not claiming that that makes it the easiest thing to use in all cases.
71
00:11:14.048 --> 00:11:20.608
We think it makes it good for production applications in big systems where type checking is a no-brainer in Python.
72
00:11:21.028 --> 00:11:25.988
But there are also a bunch of stuff we've learnt from maintaining Pydantic over the years that we've gone and done.
73
00:11:26.048 --> 00:11:37.948
So every single example in Pydantic AI's documentation is run as part of tests, and every single print output within an example is checked during tests, so it will always be up to date.
74
00:11:38.228 --> 00:11:42.688
And then a bunch of things like, like I say, are standard best practice within the rest of the Python ecosystem.
75
00:11:42.768 --> 00:11:54.948
But are not followed surprisingly by some AI libraries like coverage, linting, type checking, et cetera, et cetera, where I think these are no-brainers, but like weirdly they're not followed by some of the other libraries.
76
00:11:55.208 --> 00:12:05.808
And can you just give a overview of the framework itself? I think there's kinda like the LLM calling frameworks. There are the multi-agent frameworks. There's the workflow frameworks.
77
00:12:05.848 --> 00:12:07.648
Like, uh, what, what does Pydantic AI do?
78
00:12:08.008 --> 00:12:19.518
I glaze over a bit when I hear all of the different sorts of frameworks, but I, I like-- And I, I, I will tell you, when I built Pydantic, when I built Logfire, and when I built Pydantic AI, my methodology is not to go and like- Research and review all of the other things.
79
00:12:19.598 --> 00:12:27.698
I kind of work out what I want, and I go and build it, and then feedback comes, and we adjust. So the fundamental building block of Pydantic AI is agents.
80
00:12:28.198 --> 00:12:35.098
The exact definition of agents and how you wanna define them is obviously ambiguous, and our things are probably sort of agentlets.
81
00:12:35.318 --> 00:12:41.518
Not that we would wanna go and rename them to agentlet, but, like, the point is, you probably build them together to build something that most people would call an agent.
82
00:12:41.578 --> 00:12:52.498
So an agent in our case has, you know, things like a prompt, like system prompt and some tools and a structured return type if you want it. That covers the vast majority of, of cases.
83
00:12:53.018 --> 00:13:00.878
There are situations where you wanna go further in the most complex workflows where you want graphs, and I resisted graphs for quite a long time.
84
00:13:00.998 --> 00:13:06.678
I was s-sort of, of the opinion you didn't need them, and you could use standard, like, Python flow control to do all of that stuff.
85
00:13:07.098 --> 00:13:15.978
I had a few arguments with people, but I, I basically came around to, yeah, I can totally see why graphs are useful. But then we have the problem that by default they're not type safe.
86
00:13:16.098 --> 00:13:25.938
Because if you have a, like, addEdge method where you give the names of two different edges, there's no type checking, right? Even if you go and do some... And not, not all the graph libraries are AI specific.
87
00:13:25.998 --> 00:13:36.818
So there's a, there's a graph library called-- But it allow-- it does like a, basically does runtime type checking, ironically using Pydantic to try and make up for the fact that, like, fundamentally, they're graphs and not type, type safe.
88
00:13:36.858 --> 00:13:43.458
Well, I like Pydantic, but it... th-that's not a real solution to have to go and run the code to see if it's safe. There's a reason that static type checking is so powerful.
89
00:13:43.998 --> 00:13:59.158
And so we kind of, from a lot of iteration, eventually came up with a system of using normally data classes to define nodes, where you return the next node you want to call and where we're able to go and introspect the return type of a node to basically build the graph.
90
00:13:59.198 --> 00:14:10.678
And so the graph is inherently type safe. And once we got that right, I, I, uh, was and am incredibly excited about graphs. I think there's like masses of use cases for them, both in GenAI and other development.
91
00:14:10.758 --> 00:14:18.698
But also software's all gonna have interact with GenAI, right? It's gonna be like web. There's no gonna longer be a, like a, a web department in a company, is there?
92
00:14:18.738 --> 00:14:24.498
There's just like all the developers are building for web, building with databases. The same is gonna be true for GenAI. Yeah.
93
00:14:24.578 --> 00:14:37.918
I've seen on your docs you call an agent a container that contains a system prompt, function tools, structure result, dependency type, model, and then model settings. Are the graphs, in your mind, different agents?
94
00:14:38.078 --> 00:14:48.538
Are they, uh, different prompts for the same agent? What are like the structures in your mind? So we were compelled enough by graphs once we got them right that we actually merged a PR this morning.
95
00:14:48.578 --> 00:14:58.018
That means our agent implementation without changing its API at all is now actually a, a, a graph under the hood, as in it is, it is built using our graph library.
96
00:14:58.358 --> 00:15:02.538
So graphs are basically a lower level tool that allow you to build these complex workflows.
97
00:15:03.018 --> 00:15:10.018
Our agents are technically one of the many graphs you could go and build, and we just happened to build that one for you because it's a very common, commonplace one.
98
00:15:10.518 --> 00:15:19.658
But obviously there are cases where you need more complex workflows where the current agent assumptions don't work, and that's where you can then go and use graphs to build more complex things.
99
00:15:19.978 --> 00:15:23.538
You said you were cynical about graphs. Uh, what changed your mind specifically?
100
00:15:24.078 --> 00:15:38.318
I guess people kept giving me examples of things that they wanted to use graphs for and my like, "Yeah, but you could do that in standard flow control in Python," became a like less and less compelling argument to me because I've maintained those systems that end up with like spaghetti code.
101
00:15:38.698 --> 00:15:42.858
Yeah. And I could see the appeal of this like structured way of defining the workflow of my code.
102
00:15:43.358 --> 00:15:51.778
And it's really neat that like just from your code, just from your type hints, you can get out a Mermaid diagram that defines exactly what can go and happen. Right. Yeah.
103
00:15:52.058 --> 00:16:02.338
Y-uh, you do have very neat, uh, implementation of, um, sort of inferring the graph from type hints I guess is- Yeah... is what I would call it. Yeah, I think the question always is I, I have gone back and forth.
104
00:16:02.438 --> 00:16:18.898
Um, I, you know, used to work at Temporal where we would actually spend a lot of time complaining about graph-based workflow solutions like AWS Step Functions, and we would actually say that we were better because y-you could use normal control flow that you already knew and worked with.
105
00:16:19.378 --> 00:16:21.918
Yours, I guess, is like a little bit of a nice compromise.
106
00:16:21.958 --> 00:16:33.378
Like, um, it looks like normal Pythonic code, but you just have to keep in mind what the type hints actually do, uh, with the quote unquote "magic" that, uh, the graph construction does. Yeah, exactly.
107
00:16:33.438 --> 00:16:44.638
And if you look at the internal logic of actually running a graph, it's incredibly simple. It's basically call a node, get a node back, call that node, get a node back, call that node. If you get an end, you're done.
108
00:16:44.938 --> 00:16:57.918
We will add in soon support for, well, basically storage, so that you can store the state between each node that's run, and then you be, uh, the idea is you can then distribute a graph and run it across compute.
109
00:16:57.958 --> 00:17:07.438
And also, I mean, the other weird-- the other bit that's really valuable is across time because it's all very well if you, if you look at like lots of the graph examples that like Claude will give you if it, if it gives you an example.
110
00:17:07.478 --> 00:17:13.678
It gives you this lovely enormous Mermaid chart of like the workflow, for example, managing returns if you're an e-commerce company.
111
00:17:14.118 --> 00:17:24.578
But what you realize is some of those lines are literally one function calls another function, and some of those lines are wait six days for the customer to print their like piece of paper and put it in the post.
112
00:17:24.958 --> 00:17:32.218
And if you're writing like your demo project or your, your like proof of concept, that's fine because you can just say, "And now we call this function."
113
00:17:32.638 --> 00:17:41.858
But when you're building-- When you're in real, in real life, that doesn't work. And now how do we manage that concept of basically being able to start somewhere else in the, in our code?
114
00:17:41.918 --> 00:17:49.898
Well, this graph implementation makes it incredibly easy because you just pass the node that is the start point for carrying on the graph and it continues to run.
115
00:17:50.278 --> 00:17:57.778
So it's things like that where I was like, "Yeah, I can just imagine how things I've done in the past would be fundamentally easier to understand if, if we had done them with graphs."
116
00:17:58.238 --> 00:18:08.438
You say im-imagine, but like right now does Pydantic AI actually resume, you know, six days later like you said? Or is this, this is like a theoretical thing we can go someday? I think it's basically Q&A.
117
00:18:08.718 --> 00:18:18.338
So there's an AI that's asking the user a question and effectively you then call the CLI again to continue the conversation and it basically instantiates the node and calls the graph with that node again.
118
00:18:18.894 --> 00:18:27.574
Now we don't have the logic yet for effectively storing state in the database between individual nodes. That, uh, we're gonna add soon. But, like, the rest of it is basically there.
119
00:18:28.134 --> 00:18:42.014
It does make me think that not only are you competing with LangChain now and obviously Instructor, uh, and now you're going into sort of the more, like, uh, orchestrate-y things like Airflow, Prefect, Daxter, those guys.
120
00:18:42.514 --> 00:18:51.574
Yeah. I mean, we-we're good friends with the Prefect guys, and Temporal have the same investors as us, and I'm sure that my investor Bogumil would not be too happy if I was like, "Oh yeah, by the way- [laughs]...
121
00:18:51.634 --> 00:19:01.074
as well as trying to take on Datadog, we're also going off and trying to take on Temporal and everyone else doing that." Obviously, we're not doing all of the, the infrastructure of deploying that, right? Yet, at least.
122
00:19:01.414 --> 00:19:03.234
We're, you know, we're just building a Python library.
123
00:19:03.274 --> 00:19:11.854
And, like, what's crazy about our graph implementation is, sure, there's a bit of magic in, like, introspecting the return type, you know, extracting things from a union, stuff like that.
124
00:19:11.894 --> 00:19:22.414
But, like, the actual calls, as I say, is literally call a function and get back a thing and call that. It's, like, incredibly simple, and therefore easy to maintain. The question is, how useful is it?
125
00:19:22.474 --> 00:19:30.874
Well, I don't know yet. I think we have to go and find out. We have a whole-- We've had a, like, slew of people joining our Slack over the last few days and saying, "Tell me how good Pydantic AI is.
126
00:19:30.934 --> 00:19:39.754
How good is Pydantic AI versus LangChain?" And I refuse to answer. That's your job to go and find that out, not mine. We've built a thing. I'm compelled by it, but I'm obviously biased.
127
00:19:39.974 --> 00:19:46.534
The ecosystem will work out what the useful tools are. Bogumil was my board member when I was at Temporal, and, uh- Yeah...
128
00:19:46.574 --> 00:19:56.994
I, I think, I think just generally also having been a workflow engine investor and participant in this space, uh, it's a big space. Like, everyone needs different flavors of orchestration. Yeah.
129
00:19:57.034 --> 00:20:02.514
The one thing that I would say, like y-yours, you know, as a library, you don't have that much control over, over the infrastructure.
130
00:20:03.074 --> 00:20:15.394
I do like the idea that each new agents or whatever unit of work, whatever you call that, should spin up in its sort of isolated boundaries. Yeah. Whereas y-yours, I think, run-- everything runs in the same process.
131
00:20:15.954 --> 00:20:24.634
But you ide-ideally wants to sort of spin out its own little container of things. I agree with you a hundred percent. And we will... It would work now, right?
132
00:20:24.814 --> 00:20:34.034
As in, in theory, you're just like, as long as you can serialize- Yeah... the calls to the next node, you just have to-- all of the different containers basically have to have the same, the same code.
133
00:20:34.094 --> 00:20:40.214
I mean, I'm super excited about Cloudflare workers running Python and being able to install dependencies.
134
00:20:40.474 --> 00:20:52.834
And if Cloudflare could only give me my invitation to the, the private beta of that, we would be exploring that right now, because I'm super excited about that as a, like, compute level for some of this stuff, where exactly what you're saying, basically.
135
00:20:52.894 --> 00:21:01.974
You can run everything as an individual, like, worker function and distribute it, and it's resilient to failure, et cetera, et cetera. And it spins up, like, a thousand instances simultaneously.
136
00:21:01.994 --> 00:21:09.374
You know, you, you want it to be sort of truly serverless, uh, at once. Uh, actually, I, I know we have some Cloudflare friends who are listening, so, uh, hopefully we'll-- they'll get in front of the line.
137
00:21:09.394 --> 00:21:15.574
In London especially. [laughs] I was in Cloudflare's office last week shouting at them about other things that frustrate me. I have a love-hate relationship with Cloudflare.
138
00:21:15.614 --> 00:21:25.474
Their, their tech is awesome, but because I use it the whole time, I then get frustrated. So, um, yeah. I'm sure I will, I will, I will get there soon. Just a side tangent on Cloudflare. Is Python supported, uh, full?
139
00:21:25.854 --> 00:21:37.594
I actually wasn't fully aware that-- of what the status of that thing is. Yeah. So, so Pyodide, which is Python running inside the browser in Scripting, is supported now by Cloudflare.
140
00:21:37.634 --> 00:21:51.054
They just-- They basically-- They're having some struggles working out how to manage, ironically, dependencies that have binaries, in particular Pydantic, because these workers where you can have thousands of them on a given metal machine, you don't wanna have a different...
141
00:21:51.094 --> 00:21:58.794
You, you basically wanna be able to have a shared, shared memory for all the different Pydantic installations, effectively. That's the thing they, they work out-- they're, they're working out.
142
00:21:58.834 --> 00:22:09.274
But Hood, who's my friend, who is the primary maintainer of Pyodide, works for Cloudflare, and that's basically what he's doing, is working out how to get Python running on, running on Cloudflare's, like, network.
143
00:22:09.934 --> 00:22:17.594
I mean, the, the nice thing is that your binary is really written in Rust, right? Yeah. Which also compiles the WebAssembly. Yeah.
144
00:22:17.654 --> 00:22:27.454
So maybe there's a way that you build-- you have just a different build of, uh, Pydantic and, uh, that ships with, uh, with whatever your distro for Cloud-Cloudflare workers is. Yes. That's exactly...
145
00:22:27.534 --> 00:22:39.814
So Pyodide has builds for, for Pydantic core and for things like NumPy and basically l- all of the popular, like, binary libraries. Yeah, it's just basic... And you're doing exactly that, right?
146
00:22:39.854 --> 00:22:47.614
You're using Rust to compile to WebAssembly, and then you're calling that shared library from Python, and it's unbelievably complicated, but it works. Okay.
147
00:22:47.754 --> 00:22:57.214
Staying on graphs a little bit more, and then I wanted to go to some of the other features that you have in Pydantic AI. I see in, in your docs there are sort of four levels of agents.
148
00:22:57.274 --> 00:23:07.354
There's single agents, there's agent delegation, programmatic agent handoff, uh, that seems to be what OpenAI Swarms would be like, and then the last one, graph-based control flow.
149
00:23:07.734 --> 00:23:15.554
Would you say, like, that those are sort of the mental hierarchy of, of how these things go? Yeah, roughly. Okay. Uh, you, you- Uh... you had some, uh, expression around OpenAI Swarms.
150
00:23:16.014 --> 00:23:22.034
Well, a-and indeed, OpenAI have got in touch with me, and basically, maybe I'm not supposed to say this, but basically said- [laughs]...
151
00:23:22.074 --> 00:23:29.354
uh, that Pydantic AI looks like what Swarms would become if it was production ready. So- Yeah... yeah, I mean, like, yeah, which makes sense. Awesome. Awesome.
152
00:23:29.394 --> 00:23:37.294
Yeah, I mean, in fact, it was specifically saying, how can we give people the same feeling that they were getting from Swarms that led us to go and implement graphs?
153
00:23:37.754 --> 00:23:47.034
Because my, like, just call the next agent with Python code was not a satisfactory answer to people, so it was like, okay, we gotta go and have a better answer for that, that, like, led us to get to graphs. Yeah.
154
00:23:47.174 --> 00:23:55.854
I mean, it's a minimal viable, uh, graph in, in, in some sense. So what are the s- the shapes of graphs that people would-- should know?
155
00:23:56.014 --> 00:24:08.174
Uh, so the way that I would phrase this is I think Anthropic did a, a very good public service and also kind of surprisingly influential blog post, I would say, when they wrote Building Effective Agents.
156
00:24:08.334 --> 00:24:17.654
Uh, we actually have the authors coming to speak, uh, at my conference in New York, uh, which I think you're giving a workshop at. Yeah. I'm trying to work it out- Okay... but yes, I think so. Tell me if you're not.
157
00:24:17.754 --> 00:24:28.194
[laughs] But, but yeah, I mean, like I... That, that was the first, I think, authoritative view of like what kinds of graphs exist at agents, and let's give each of them a name so that everyone is on the same page.
158
00:24:28.634 --> 00:24:37.518
So I'm just kinda curious if, if you have community names or top five patterns or graphs. I don't have top five pa-patterns of graphs.
159
00:24:37.778 --> 00:24:42.238
I, I would love to see what people are building with them, but, but like it's been-- it's only been a couple of weeks.
160
00:24:42.438 --> 00:24:56.518
And of course, the, the point is that like, because they're relatively unopinionated about what you can go and do with them, they don't suit-- Like, you can go and do lots of, lots of things with them, but they don't have the structure to, to go and have like specific names as, as much as perhaps like some other systems do.
161
00:24:56.578 --> 00:25:06.438
I think what our agents are, which you have a name and I, I can't remember what it is, but this basically system of like decide what tool to call, go back to the center, decide what tool to call, go back to the center, and then exit.
162
00:25:06.778 --> 00:25:13.978
One form of graph, which as I say, like our agents are effectively one implementation of a graph, which is why under the hood they are now using graphs.
163
00:25:14.318 --> 00:25:23.098
And it'll be interesting to see over the next few years whether we end up with these like predefined graph names or graph structures or whether it's just like, "Yep, I built a graph."
164
00:25:23.218 --> 00:25:30.498
Or whether graphs just turn out not to match people's mental image of what they want and die away. We'll see. I think there is always a appeal.
165
00:25:30.718 --> 00:25:40.418
Um, every developer eventually gets graph religion and goes, "Oh yeah, everything's a graph." And then they, they probably over-rotate and go, go too far into graphs- Yeah...
166
00:25:40.458 --> 00:25:46.458
and then they have to learn a whole bunch of DSLs, and then they're like, "Actually, I didn't need this," and they scale back a little bit. Yeah. I'm at the, I'm at the beginning of that process.
167
00:25:46.518 --> 00:25:52.538
I'm, I'm currently- [laughs]... a, like, graph maximalist, although I haven't-- can't say I've put actually any into production yet. But, um, yeah.
168
00:25:52.558 --> 00:26:03.378
[laughs] This has a lot of philosophical connections with other work coming out of UC Berkeley on compounding AI systems. I don't know if you know of OCare.
169
00:26:03.738 --> 00:26:15.358
This is the Gartner world of, of things where they, they need some kind of industry terminology to, to sell it to enterprises. I don't know if you, you know about any of that stuff. I haven't. I probably should.
170
00:26:15.418 --> 00:26:19.718
I should probably do it 'cause I should probably get better at selling to enterprises. But no, no, I don't. Not right now.
171
00:26:20.178 --> 00:26:33.038
This is really the, the argument is that instead of putting everything in one model, you have more control and more maybe observability to if you break everything out into composing little models- Yeah...
172
00:26:33.338 --> 00:26:38.438
and chaining them together. Uh, and obviously then you'll need an orchestration framework to do that. Yeah. And, and it makes complete sense.
173
00:26:38.478 --> 00:26:55.478
And one of the things we've seen with agents is they work well when they work well, but when they-- even if you have the observability through Log5 that you can see what was going on, if you don't have a nice hook point to say, "Hang on, this has all gone wrong," you have a, a relatively blunt instrument of basically erroring when you exceed some kind of limit.
174
00:26:56.018 --> 00:27:03.578
But like what you need to be able to do is effectively iterate through these runs so that you can have your own control flow where you're like, "Okay, we've gone too far."
175
00:27:03.618 --> 00:27:13.758
And that's where one of the neat things about our graph implementation is you can basically call next in a loop rather than just running the full graph, and therefore you have this opportunity to, to break out of it.
176
00:27:13.798 --> 00:27:27.578
But, but yeah, basically it's the same point, which is like if you have too big a unit of work, to some extent, whether or not it involves gen AI, but obviously it's particularly problematic in gen AI, you only find out afterwards when you've spent quite a lot of time and/or money- [laughs]...
177
00:27:27.598 --> 00:27:34.558
when it's gone off and done, done the wrong thing. I'll drop one this-- We're not gonna resolve this here, but I'll drop this, and then we can move on to the next thing. Yeah.
178
00:27:34.598 --> 00:27:41.918
This is the common way that we, we developers talk about this, and then the machine learning researchers look at us and laugh and say, "That's cute."
179
00:27:41.958 --> 00:27:50.518
And then they just train a bigger model, and they wipe us out in the next training run. So I think there's a certain amount of we are fighting the bitter lesson here.
180
00:27:51.018 --> 00:28:00.598
We're fighting AGI, and you know, when AGI arrives, this will all go away. Obviously, on latent space, we don't really discuss that because I think AGI is kind of this hand-wavy concept that isn't- Yeah...
181
00:28:00.618 --> 00:28:02.998
super relevant. Uh, but I think we have to respect that.
182
00:28:03.518 --> 00:28:16.238
For example, you could do chain of thoughts with graphs, and you could manually orchestrate a nice little graph that, that does like reflect, think about if you need more, more inference time computes, you know, that's the hot term now.
183
00:28:16.598 --> 00:28:22.658
Yeah. And then think again and, you know, scale that up. Or you could train Strawberry and, and DeepSeek or one. Right.
184
00:28:22.658 --> 00:28:31.838
[laughs] I saw someone saying recently, oh, they were, they were really optimistic about, uh, agents because models are getting faster exponentially.
185
00:28:31.878 --> 00:28:45.578
And I like took a certain amount of self-control not to describe that it wasn't exponential, but I-- my, my main point was if, if models are getting faster as quickly as you say they are, then we don't need agents, and we don't really need any of these abstraction layers.
186
00:28:45.978 --> 00:28:50.118
We can just give our model an, you know, access to the internet, cross our fingers and hope for the best.
187
00:28:50.518 --> 00:28:57.478
Agents, agent frameworks, graphs, all of this stuff is basically making up for the fact that right now the models are not that clever.
188
00:28:57.878 --> 00:29:08.778
In the same way that if you're running a customer service business and you have loads of people sitting answering telephones, the less well-trained they are, the less that you trust them, the more that you need to give them a script to go through.
189
00:29:09.178 --> 00:29:15.718
Whereas, you know, so if you're running a bank and you have the lots of, uh, customer service people who you don't trust that much, then you tell them exactly what to say.
190
00:29:15.798 --> 00:29:23.578
If you have-- If you're doing high net worth banking, you just employ people who you think are gonna be charming to other rich people and set them off to go and have coffee with people, right?
191
00:29:24.018 --> 00:29:33.958
And the same is true of, of models, right? The more intelligent they are, the less we need to tell them like structure what they go and do and constrain the, the routes in which they take. Yeah. Yeah. Agree with that.
192
00:29:34.478 --> 00:29:41.118
Uh, so I'm happy to move on. There's sort of other, other parts of Pydantic AI that are, uh, worth commenting on, and this is like my last rant, I, I promise.
193
00:29:41.478 --> 00:29:51.338
So obviously every framework needs to do its sort of model adapter layer, which is, oh, you can- Yeah... easily swap from OpenAI to Claude to Groq.
194
00:29:51.558 --> 00:30:03.698
Uh, you also have, which I didn't know about, Google GLA, which I, I didn't really s- know about until I saw this in your, in your docs, which is Generative Language API. I assume that's AI Studio. Yes.
195
00:30:04.058 --> 00:30:14.998
Google don't have good names for it. So Vertex is very clear. That seems to be the API that, that like some of the things use, although it returns five oh three about twenty percent of the time, so. [laughs] Vertex?
196
00:30:15.398 --> 00:30:20.158
No, Vertex fine, but the- Oh, oh, GLA. Yeah... generative language API. Yeah. I, I agree with that. Yeah. Yeah.
197
00:30:20.178 --> 00:30:30.638
So, so we have, again, another example of like where I think we go the extra mile in terms of engineering is we run on every commit, at least commit to main, we run tests against the live models. Oh, okay.
198
00:30:30.638 --> 00:30:37.438
Not, not lots of tests, but like a handful of them. And we had a point last week where, yeah, GLA one was failing every single run.
199
00:30:37.498 --> 00:30:41.078
One of their tests would fail, and we-- I think we might even have commented that, that out that one at the moment.
200
00:30:41.138 --> 00:30:51.370
So like all of the models fail more often than you might expect, but like- That one seems to be particularly likely to fail. Yeah. But, but Vertex seems to... Is the same API, but much more reliable.
201
00:30:51.910 --> 00:31:00.730
My rant here is that, you know, versions of this appear in LangChain and all, every single framework has to have its own- Yep... little thing or a, a version of that.
202
00:31:01.190 --> 00:31:06.630
I would put to you, and then, you know, this is, this can be agree to disagree, that y- this is not needed in Pydantic-AI.
203
00:31:07.010 --> 00:31:19.050
I would much rather you adopt a layer like LiteLLM or what's the other one in, in, in JavaScript, Portkey, and that's their job. They focus on that one thing, and they, they normalize APIs for you.
204
00:31:19.510 --> 00:31:29.030
All new models are automatically added, and you don't have to duplicate this in-inside of your framework. So, for example, if I wanted to use DeepSeek, I'm out of luck 'cause Pydantic-AI doesn't have DeepSeek yet.
205
00:31:29.330 --> 00:31:32.210
Yeah, it does. Oh, it does. Okay. I'm sorry. [laughs] Yeah, but you know what I mean.
206
00:31:32.530 --> 00:31:39.430
Does-- should this live in your code, or should it live in a, in a layer that's kind of y-your API gateway that's a, that's a defined piece of infrastructure that people have?
207
00:31:39.770 --> 00:31:52.930
And I think if, if a company who are well-known, who are, like, respected by everyone had come along and done this at the right time, maybe we should have done it a year and a half ago and said, "We're gonna be the, like, universal AI layer," that would have been a, a, like, credible thing to do.
208
00:31:53.510 --> 00:32:01.990
I've heard varying reports of LiteLLM, uh, is the truth, and it didn't seem to have exactly the type safety that we needed.
209
00:32:02.510 --> 00:32:11.290
Also, as I understand it, and again, I haven't looked into it in great detail, part of their business model is proxying the request through their, through their own system to do the generalization.
210
00:32:11.910 --> 00:32:22.190
That would be an enormous put-off to, to an awful lot of people. Honestly, the truth is I, I don't think it is that much work unifying the model. I get where you're coming from. I kind of see your point.
211
00:32:22.530 --> 00:32:35.530
I think the truth is that everyone is centralizing around OpenAI's SD, uh, API as the, as the one to do. So DeepSeek, uh, support that. Groq with a K support that. Uh, Ollama also does it. Well, pro...
212
00:32:35.650 --> 00:32:43.290
I mean, if there is that library right now, it's more or less the OpenAI SDK. Right. Okay. And it's, like, very high quality. It's w-well type checked.
213
00:32:43.790 --> 00:32:52.530
It uses Pydantic, so I'm biased, but I mean, I think it's, I think it's pretty well-respected anyway. There's different ways to do, do this. 'Cause also, like, not y- it's not just about normalizing APIs.
214
00:32:52.570 --> 00:32:56.530
You have to do secret management [laughs] and, and all that stuff. Yeah. And there's also...
215
00:32:56.630 --> 00:33:09.590
'Cause there's, like, Vertex will-- There's Vertex and Bedrock, which to one extent or another, effectively they host multiple models, but they don't unify the API, but they do, uh, unify the auth, as I understand it.
216
00:33:09.610 --> 00:33:12.610
Although we are-- we're halfway through doing Bedrock, so I, I don't know about it that well.
217
00:33:12.650 --> 00:33:20.670
But, like, they're kind of weird hybrids because they support multiple models, but, but like, like I say, the auth is centralized. Yeah. I'm surprised they don't unify the API.
218
00:33:20.710 --> 00:33:25.470
That s- that seems like, uh, something that I would do. You know, we can- Yeah... we can discuss all this all day. There's a lot of APIs.
219
00:33:25.950 --> 00:33:29.610
I agree it would be nice if there was a universal one that, that we didn't have to go and build.
220
00:33:30.070 --> 00:33:37.450
And I guess the other side of, you know, writing model and picking models like evals, how do you actually figure out which one you should be using? I know you have one.
221
00:33:37.870 --> 00:33:43.350
First of all, you have very good support for mocking in unit tests, which is something that a lot of other frameworks don't do.
222
00:33:43.430 --> 00:33:52.590
So, you know, my favorite Ruby library is VCR, um, because it just, you know, it just lets me store the HTTP request and replay them. That part I'll kinda skip.
223
00:33:52.650 --> 00:34:03.470
I think you have basically, like, this test model where, like, just through Python, you try and figure out, uh, what the model might respond without actually calling the model, and then you have the function model where people can kinda customize outputs.
224
00:34:03.850 --> 00:34:13.190
Yep. Any other fun stories maybe from there, or is it just what you see is what you get, so to speak? On those two, I think what you see is what you get. On the evals, I think watch this space.
225
00:34:13.350 --> 00:34:22.490
I think it's something that, like, again, I was s-somewhat cynical about for some time. Still have my cynicisms about some of the... Well, it's unfortunate that so many different things are called evals.
226
00:34:22.750 --> 00:34:27.610
And it would be nice if we could agree what they are and what they're not. But look, I think it's a really important space.
227
00:34:27.770 --> 00:34:36.050
I think it's something that we're gonna be working on soon, both in Pydantic-AI and in Logfire to try and support better. 'Cause, like, y- it's an unsolved problem. Yeah.
228
00:34:36.090 --> 00:34:45.570
You do say in your doc that anyone who claims to know for sure exactly how your eval should be defined can safely be ignored. [laughs] We'll delete that sentence when we tell people how to do their evals.
229
00:34:45.670 --> 00:34:56.190
[laughs] Exactly. I was like, "We need, we need a snapshot of this today." And so let-let's talk about eval. So there's kinda like the vibe evals, which is what you do when you're building, right?
230
00:34:56.230 --> 00:35:03.330
Because you cannot really, like, test it that many times to get statistical significance. And then there's the production eval.
231
00:35:03.430 --> 00:35:14.110
So you also have Logfire, which is kinda like your observability product, which I tried before. It's very nice. What are some of the learnings you've had from building an observability tool for lamps?
232
00:35:14.470 --> 00:35:23.510
Um, and yeah, a-as people think about evals even, like, what are the right things to measure? What are, like, the right number of samples that you need to actually start making decisions?
233
00:35:24.050 --> 00:35:30.390
I'm not the best person to answer that is the truth, so I'm not gonna come in here and tell you that I think I know the answer, uh, on the exact number.
234
00:35:30.450 --> 00:35:41.130
I mean, we can do some back of the envelope statistics calculations to work out that, like, having thirty probably gets you most of the statistical value of g- having two hundred for, you know, by definition fifteen percent of the work.
235
00:35:41.530 --> 00:35:50.450
But the exact, like how many examples do you need, for example, that's a much harder question to answer because it's, you know, it's deep within the how models, uh, operate.
236
00:35:50.910 --> 00:36:02.850
In terms of Logfire, one of the reasons we built Logfire the way we have, and we allow you to write SQL directly against your data, and we're trying to build the, like, powerful fundamentals of observability, is precisely because we know we don't know the answers.
237
00:36:02.890 --> 00:36:09.530
And so allowing people to go and innovate on how they're gonna consume that stuff and what-- how they're gonna process it is in-- we think that's valuable.
238
00:36:09.570 --> 00:36:28.770
Because even if we come along and offer you an evals framework on top of Logfire, it won't be right in all regards, and we want people to be able to go and innovate and being able to write their own SQL, connect to the API and effectively query the data like it's a database with SQL allows people to innovate on that stuff and, and that's what, you know, that allows us to do it as well.
239
00:36:28.830 --> 00:36:36.110
I mean, we do a bunch of, like, testing what's possible by basically writing SQL directly against Logfire a-as any user could.
240
00:36:36.450 --> 00:36:46.830
I think the other, the other really interesting bit that's going on in observability is OpenTelemetry is centralizing around semantic attributes for GenAI. So it's a relatively new project.
241
00:36:47.230 --> 00:36:59.210
A lot of it's still being added at the moment, but basically the idea that, like- They unify how, uh, both SDKs and or agent frameworks send observability data to, to any OpenTelemetry en-endpoint.
242
00:36:59.250 --> 00:37:08.310
And so again, we, we can go and having that unification allows us to go and like basically compare different libraries, compare different models much better.
243
00:37:08.630 --> 00:37:11.470
That stuff's in, uh, in a very like early stage of development.
244
00:37:11.950 --> 00:37:20.070
It's one of the things we're gonna be working on pretty soon is basically, I suspect Pydantic AI will be the first Asian framework that implements those semantic attributes properly.
245
00:37:20.090 --> 00:37:24.370
Because again, we control Pydantic AI, and we can say this is important for observability.
246
00:37:24.450 --> 00:37:41.130
Whereas most of the other Asian frameworks are not maintained by people who are trying to do observability, with the exception of LangChain, where they have the observability platform, but they chose not to go down the OpenTelemetry route, so they're like plowing their own furrow and there's, you know, they're a lot-- they're even further away from standardization.
247
00:37:41.710 --> 00:37:52.870
Can you maybe just give a quick overview of how OTel ties into the AI workflows? There's kinda like the question of is, you know, a trace and a span like a LLM call? Is it the agent?
248
00:37:53.090 --> 00:37:56.950
Is kinda like the broader thing you're, you're tracking? How should people think about it? Yeah.
249
00:37:57.010 --> 00:38:07.990
So they have a-- there's a PR that I think may have now been merged from someone at IBM talking about remote agents and trying to support this concept of remote agents within GenAI.
250
00:38:08.430 --> 00:38:15.370
I'm not particularly compelled by that because I don't think that, like, that's actually by any means the common use case, but, like, I suppose it's fine for it to be there.
251
00:38:15.770 --> 00:38:21.490
The m-majority of the stuff in OTel is basically defining how you would instrument a given call to an LLM.
252
00:38:21.550 --> 00:38:27.530
So basically the actual LLM call, what, what data you would, uh, send to your telemetry provider, how you would structure that.
253
00:38:27.930 --> 00:38:38.370
Apart from this slightly odd stuff on, uh, remote agents, most of the, like, agent-level consideration is not yet implemented in-- is not yet decided effectively, and so there's a bit of ambiguity.
254
00:38:38.410 --> 00:38:46.290
Obviously, what's good about OTel is you can in the end send whatever attributes you like. But yeah, there's quite a lot of churn in that space in exactly how we store the data.
255
00:38:46.350 --> 00:38:54.410
I think the-- one of the most interesting things, though, is if you think about observability traditionally, it was, sure, everyone would say, "Our observability data is very important. We must keep it safe."
256
00:38:54.470 --> 00:38:59.730
But actually, companies work very hard to basically not have anything that sensitive in their observability data.
257
00:38:59.810 --> 00:39:08.670
So if you're a doctor in a hospital and you search for a drug for an STI, the sequel might be sent to the observability provider, but none of the parameters would.
258
00:39:08.710 --> 00:39:15.850
So it wouldn't have the, like, patient number or their name or the drug. With GenAI, that, that distinction doesn't exist because it's all just, like, messed up in the text.
259
00:39:15.930 --> 00:39:26.610
If you have that same patient asking an LLM how to-- what drug they should take or how to stop smoking, you can't extract the PII and not send it to the observability platform.
260
00:39:26.670 --> 00:39:35.450
So the sensitivity of the data that's gonna end up in observability platforms is gonna be at a, like, basically different order of magnitude to what's in a-- what, what you would normally send to Datadog.
261
00:39:35.510 --> 00:39:44.690
Of course, you could make a mistake and send someone's password or their card number to Datadog, but that would be seen as a, as, as a, like, mistake, whereas in GenAI, a lot of data is gonna be sent.
262
00:39:44.710 --> 00:39:59.810
And I think that's why companies like LangSmith and are trying hard to offer observability on-prem because there's a bunch of companies who are happy for Datadog to be cloud-hosted but want self-hosted, self-hosting for this observability stuff w-with GenAI.
263
00:40:00.250 --> 00:40:02.510
And are you doing any of that today?
264
00:40:02.550 --> 00:40:12.990
Because I know in each of the spans you have, like, the number of tokens, you have the context, you're just storing everything, and then you're gonna offer kinda like a self-hosting for the platform, basically is the idea.
265
00:40:13.210 --> 00:40:21.670
Yeah. We'll offer self-host. So, so we have scrubbing roughly equivalent to what the other observability platforms have. So if we, you know, if we see password as the key, we won't send the value.
266
00:40:21.710 --> 00:40:31.990
But like, like I say, that doesn't really work in GenAI. So we're, we're accepting we're gonna have to store a lot of data, and then we'll offer self-hosting for those people who can afford it and who need it.
267
00:40:32.490 --> 00:40:39.350
And then this is, I think, the first time that most of the workload's performance is depending on a third party. You know?
268
00:40:39.410 --> 00:40:47.290
Like if you're looking at Datadog data, usually it's your app that is driving the latency and, like, the memory usage and all of that.
269
00:40:47.690 --> 00:40:55.710
Here you can have spans that maybe take a long time to perform because the GLA API is not working or because, like, OpenAI is kinda like, uh, overwhelmed.
270
00:40:56.110 --> 00:41:00.870
Do you do anything there since, like, the provider is almost, like, the same across customers, you know?
271
00:41:00.910 --> 00:41:10.650
Like, are you trying to surface these things for people and say, "Hey, this was, like, a very slow span, but actually all customers using OpenAI right now are seeing the same thing, so maybe don't worry about it," or?
272
00:41:10.970 --> 00:41:21.210
Not yet. We do a few things that, uh, people don't generally do in OTel. So we send, we send information at the beginning of a trace as well as, uh, sorry, at the beginning of a span, as well as when it finishes.
273
00:41:21.530 --> 00:41:24.050
By default, OTel only sends you data when the span finishes.
274
00:41:24.490 --> 00:41:34.030
So if you think about a request which might take, like, twenty seconds, even if some of the intermediate spans finished earlier, you can't basically place them on the page until you get the, the top-level span.
275
00:41:34.070 --> 00:41:42.150
And so if you're using standard OTel, you can't show anything until those requests are finished. When those requests are taking a few hundred milliseconds, it doesn't really matter.
276
00:41:42.190 --> 00:41:51.810
But when you're doing GenAI calls or when you're, like, running a batch job that might take thirty minutes, that, like, latency of not being able to see the span is, like, crippling to understanding your application.
277
00:41:51.830 --> 00:41:59.250
And so we've-- we do a bunch of slightly complex stuff to basically send data about a span as it starts, which is closely related. Yeah.
278
00:41:59.710 --> 00:42:10.350
Any thoughts on all the other people trying to build on top of OpenTelemetry in different languages too? There's like the OpenTelemetry project, which doesn't really roll off the tongue.
279
00:42:10.890 --> 00:42:19.430
But how do you see the, yeah, the, the future of these kind of tools? Is everybody gonna have to build-- Why does everybody want to build their own open source observability thing to then sell?
280
00:42:19.810 --> 00:42:31.890
I mean, we, we are not going off and trying to instrument the likes of the OpenAI SDK with the new semantic attributes because at some point that's gonna happen, and it's gonna live inside OTel, and we might help with it, but we're a tiny team.
281
00:42:31.930 --> 00:42:33.650
We don't have time to go and do all of that work.
282
00:42:34.110 --> 00:42:46.870
So OpenTelemetry, like, interesting project, but I suspect eventually most of those semantic-- like, that instrumentation of the big-- of the SDKs will live, like I say, inside the main OpenTelemetry repos.
283
00:42:47.370 --> 00:42:56.178
What happens to the agent frameworks, what data you basically need at the framework level to get the context is kind of unclear. Uh, I don't think we know the answer yet.
284
00:42:56.558 --> 00:43:16.118
But, I mean, I was on the-- I guess this is kinda semi-public 'cause it was on-- I was on the call with the OpenTelemetry call last week talking about GenAI, and there was someone from Arize talking about the challenges they have trying to get OpenTelemetry data out of LangChain where it's not, uh, like, natively implemented, and obviously they're having quite a tough time.
285
00:43:16.158 --> 00:43:26.118
And I was realizing, I hadn't really realized this before, but how lucky we are to primarily be talking about our own agent framework where we have the control rather than trying to go and instrument other people's.
286
00:43:26.618 --> 00:43:36.398
Sorry, I actually didn't know about this semantic conventions thing. It looks like, yeah, I mean, it's, it's merged into main OTel. What should people know about this? I had n- I'd never heard of it before. Yeah.
287
00:43:36.438 --> 00:43:38.538
I, I, I think it looks like a, looks like a great start.
288
00:43:38.578 --> 00:43:50.538
I think there's some, there's some unknowns around how you send the messages that go back and forth, which is kind of the m-most important thing of all, and that has moved out of attributes and into OTel events.
289
00:43:50.938 --> 00:44:00.177
OTel events in turn are moving from being on a span to being their own top-level API where you send data. So there's a, like, there's a bunch of churn still going on.
290
00:44:00.658 --> 00:44:04.318
I'm impressed by how fast the OTel community is moving, moving on this project.
291
00:44:04.698 --> 00:44:15.418
I guess they, like everyone else, get that, that, you know, this is important and it's something that, like, people are crying out to get instrumentation of, so, so I'm, I'm kind of pleasantly surprised at how fast they're moving, but it makes sense.
292
00:44:15.798 --> 00:44:21.818
I, I'm just kinda browsing through the specification. Like, I can already see that this basically bakes in whatever the previous paradigm was.
293
00:44:22.238 --> 00:44:32.918
So now they, they have like, you know, genai.usage.prompt tokens and genai.usage.completion tokens, and obviously now we have reasoning tokens as well. And then only one form of sampling, which is top P.
294
00:44:33.338 --> 00:44:44.878
But you're basically baking in or sort of reifying things that you think are important today, but, like, it, it's not a super foolproof way of- Yeah... doing this for future. [laughs] Yeah.
295
00:44:44.938 --> 00:44:50.598
I mean, that's what's neat about OTel is you can always go and send another attribute, and that's fine. Sure. It's just there are a bunch that are agreed on. Okay.
296
00:44:50.618 --> 00:45:04.857
But I would say, you know, to come back to your previous point about whether or not we should be relying on one centralized abstraction layer, this stuff is moving so fast that if you start relying on someone else's standard, you risk basically falling behind because you're relying on someone else to keep things up to date.
297
00:45:05.258 --> 00:45:13.818
Or you fall behind because you got, you got other things going on. [laughs] Yeah, yeah. That's fair. That's fair. Any other observations just about building Logfire, actually? Uh, let's just talk about this.
298
00:45:14.178 --> 00:45:22.278
You had recently announced, uh, Logfire. I was kind of only familiar with Logfire because of your Series A announcement. I actually thought you were making a separate company.
299
00:45:22.358 --> 00:45:25.758
I remember some amount of confusion with you when, when that an- when that came out.
300
00:45:25.818 --> 00:45:34.918
So, uh, yeah, so to be clear, it's Pydantic, Logfire, and the company is, is one company that has kind of two products, an open source thing and a, and a observability thing, correct? Yeah.
301
00:45:35.318 --> 00:45:44.258
I, I was just kinda curious, like any learnings building Logfire? Like, uh, so classic question is do you use ClickHouse? Is this like the, the, the standard persistence layer? Uh- Any, any learnings doing that?
302
00:45:44.798 --> 00:45:54.798
We don't use ClickHouse. We started building our database with ClickHouse, moved off ClickHouse onto Timescale, which is a Postgres extension to do analytical databases. Wow.
303
00:45:55.058 --> 00:46:02.618
And then moved off Timescale onto DataFusion, and we're basically now building... Well, it's, it's DataFusion, but it's, uh, a kind of our own database.
304
00:46:03.158 --> 00:46:11.618
Bogumil is not entirely happy that we went through three databases before we chose one, I'll say that, but, like, we've got to the right one in the end. I think we could have realized that Timescale wasn't right.
305
00:46:12.078 --> 00:46:19.278
I think ClickHouse-- They both taught us a lot, and we-we're in a great place now, but like, yeah, it's been a, it's been a real journey on the database in particular. Okay.
306
00:46:19.438 --> 00:46:27.398
So, you know, as a database nerd, I have to, like, double-click on this, right? Like, so ClickHouse is supposed to be the ideal- Yeah... backend for anything like this.
307
00:46:27.858 --> 00:46:38.398
And then moving from ClickHouse to Timescale is another counterintuitive move that I didn't expect because, you know, Timescale is like an extension on top of Postgres, not super meant to-- for like high volume logging.
308
00:46:38.438 --> 00:46:45.478
But like, uh, yeah, tell us, tell us that those, uh, decisions. So at the time, ClickHouse did not have good support for Post-- for JSON.
309
00:46:45.818 --> 00:46:55.558
I was speaking to someone yesterday and said ClickHouse doesn't have good support for JSON and got roundly stepped on because apparently it does now. So they've obviously gone and built their proper JSON support.
310
00:46:56.018 --> 00:47:06.198
But, like, back when we were trying to use it, uh, I guess a, a year ago or a bit more than a year ago, everything had to be a map, and maps are a, are a pain to try and do, like, looking up JSON type data.
311
00:47:06.218 --> 00:47:16.658
And obviously all these attributes, everything you're talking about there in terms of the GenAI stuff, you can choose to make them top-level columns if you want, but the simplest thing is just to put them all into a big JSON pile.
312
00:47:17.158 --> 00:47:18.538
Uh, and that was a problem with ClickHouse.
313
00:47:18.698 --> 00:47:30.678
Also, ClickHouse had some really ugly edge cases, like by default, or at least until I complained about it a lot, ClickHouse thought that two nanoseconds was longer than one second 'cause they compared intervals just by the number, not the unit.
314
00:47:31.138 --> 00:47:35.598
And I complained about that a lot, and then they caused it to raise an error and just say you have to have the same unit.
315
00:47:35.978 --> 00:47:40.378
Then I complained a bit more and event- I think as I understand it now, they have some-- they convert between units.
316
00:47:40.438 --> 00:47:46.358
But like stuff like that, when all you're looking at is, when a lot of what you're doing is comparing the duration of spans, was really painful.
317
00:47:46.758 --> 00:47:51.478
Also, things like you can't subtract two date times to get an interval, you have to use the date sub function.
318
00:47:51.918 --> 00:48:06.258
But, like, the fundamental thing is because we want our end users to write SQL, the, like, quality of the SQL, how easy it is to write matters way more to us than if you're building like a platform on top where your developers are gonna write the SQL, and once it's written and it's working, you don't mind too much.
319
00:48:06.578 --> 00:48:08.738
So I think that's like one of the fundamental differences.
320
00:48:09.198 --> 00:48:25.558
The other problem that I have with ClickHouse and in fact Timescale is that, like the ultimate architecture, the like Snowflake architecture of binary data in object store queried with some kind of cache from nearby, they both have it, but it's closed source and you only get it if you go and use their hosted versions.
321
00:48:25.878 --> 00:48:34.378
And so even if we had got through all the problems with Timescale or ClickHouse, we would end up like, you know, they would wanna be taking their eighty percent margin and then we would be wanting to take...
322
00:48:34.698 --> 00:48:42.518
That would basically leave us less space for margin. Whereas DataFusion properly open source, all of that same tooling is open source.
323
00:48:42.698 --> 00:48:49.738
And, and for us as a team of people with a lot of Rust expertise, DataFusion, which is implemented in Rust, we can literally dive into it and go and change it.
324
00:48:50.158 --> 00:49:02.898
So for example, I found that there were some slowdowns in ti- in DataFusion's string comparison kernel for doing like string contains, and it's just Rust code and I could go and rewrite the, the string comparison kernel to be faster.
325
00:49:03.378 --> 00:49:08.666
Or for example- Datafusion, when we started using it, didn't have JSON support. Obviously, as I've said, it's something we needed.
326
00:49:09.106 --> 00:49:14.066
We were able-- I was able to go and implement that in a weekend using our JSON parser that we built for Pydantic core.
327
00:49:14.646 --> 00:49:28.906
So it's the fact that like Data- Datafusion is like, for us, the perfect mixture of a toolbox to build a database with, not a database, and we can go and implement stuff on top of it in a way that, like, if you were trying to do that in Postgres or in ClickHouse...
328
00:49:28.946 --> 00:49:39.266
I mean, ClickHouse would be easier 'cause it's C++ and relatively modern C++, but, like, as a team of people who are not C++ experts, that's much scarier than Datafusion for us. Yeah. That was a beautiful rant.
329
00:49:39.266 --> 00:49:45.706
[laughs] That's funny. Most people don't think they have agency on these projects. They're kinda like, "Oh, I should use this," or, "I should use that."
330
00:49:45.806 --> 00:49:55.386
They're not really like, "What should I pick so that I contribute the most back to it?" You know? So, uh, but I think, like, you obviously have a open source first mindset, so that makes a lot of sense.
331
00:49:55.546 --> 00:50:07.426
I think if we were probably better as a, as startup-- uh, we're a better startup and faster-moving and just, like, uh, like, headlong determined to get in front of customers as fast as possible, we should have just stuck with ClickHouse.
332
00:50:07.926 --> 00:50:18.326
I hope that long term we're in a better place for having worked with Datafusion, um, and we, like, we're quite engaged now with the Datafusion community. Andrew Lamb, who maintains Datafusion, is an advisor to us.
333
00:50:18.486 --> 00:50:25.426
We're in a really good place now, but yeah, it's definitely slowed us down relative to just, like, building on ClickHouse and, and moving as fast as we can. Okay.
334
00:50:25.586 --> 00:50:36.226
We're about to zoom out and do Pydantic Run and all the other stuff, but, you know, my last question on Logfire is really, you know, at some point, you run out of sort of community goodwill just because like, "Oh, I use Pydantic.
335
00:50:36.306 --> 00:50:49.726
I love Pydantic. I'm gonna use Logfire." Okay. Then you start entering the territory of the Datadogs, the Sentries, and the Honeycombs. Yep. So where are you going to really spike here or differentiator here?
336
00:50:50.566 --> 00:50:55.626
I wasn't writing code in two thousand and one, but I'm assuming that there were people talking about, like, web observability.
337
00:50:56.046 --> 00:51:02.126
And then web observability stopped being a thing, not because the web stopped being a thing, but because all observability had to do web.
338
00:51:02.506 --> 00:51:10.066
If you were talking to people in two thousand and ten or two thousand and twelve, they would have talked about cloud observability. Now that's not a term because all observability is cloud first.
339
00:51:10.326 --> 00:51:11.846
The same is gonna happen to GenAI.
340
00:51:11.946 --> 00:51:21.706
And so whether or not you're trying to compete with Datadog or with Arize and LangSmith, you've got to do first-class-- you've got to do general purpose observability with first-class support for AI.
341
00:51:22.126 --> 00:51:27.326
And as far as I know, we're the only people really trying to do that. I mean, I think Datadog are starting in, in that direction.
342
00:51:27.366 --> 00:51:34.206
And to be honest, I think Datadog is a much, like, scarier company to compete with than the AI-specific observability platforms.
343
00:51:34.246 --> 00:51:42.826
Because in my opinion, and I've also heard this from lots of, lots of customers, AI-specific observability where you don't see everything else going on in your app is not actually that useful.
344
00:51:43.206 --> 00:51:57.586
Our hope is that we can build the first general purpose observability platform with first-class support for AI, and that we have this open source heritage of making-- of putting developer experience first that other companies haven't done.
345
00:51:57.726 --> 00:52:10.886
For all I'm a, like, fan of Datadog and what they've done, if you search Datadog logging Python, and you just try as a, like, a non-observability expert to get something up and running with Datadog and Python, it's not trivial, right?
346
00:52:10.946 --> 00:52:22.086
That's something Sentry's done amazingly well, but, like, there's enormous space in most of observability to do DX better. Since you mentioned Sentry, I'm curious how you thought about licensing and all of that.
347
00:52:22.166 --> 00:52:31.286
Obviously, you're MIT license, you don't have any rolling license like Sentry has, where, like, you can only use an open source, like the one-year-old version of it. Was that a hard decision?
348
00:52:31.846 --> 00:52:41.126
So to be clear, Logfire is closed source. So Pydantic and Pydantic AI are MIT licensed and like, you know, properly open source and then, and then Logfire for now is, is completely closed source.
349
00:52:41.486 --> 00:52:53.326
And in fact, the, the struggles that Sentry have had with licensing and the, like, weird pushback the community gives when they take something that's closed source and make it source available just meant that we just avoided that whole, whole subject matter.
350
00:52:53.386 --> 00:53:07.866
I think the other way to look at it is like in terms of either headcount or revenue or dollars in the bank, the amount of open source we do as a company is we've got to be up there with the, with the most prolific open source companies and like, like I say, per head.
351
00:53:08.326 --> 00:53:18.586
And so we didn't feel like we were morally obligated to make Logfire open source. We have Pydantic. Pydantic is the, the, uh, you know, a foundational library in, uh, Python.
352
00:53:18.926 --> 00:53:24.725
That and now Pydantic AI are our contribution to open source, and then Logfire is like openly for, for profit, right?
353
00:53:25.045 --> 00:53:32.146
As in we're not claiming otherwise, we're not sort of trying to walk a line if it's open source, but we're really we wanna make it hard to deploy, so you probably wanna pay us.
354
00:53:32.626 --> 00:53:38.386
We're trying to be straight that it's to pay for. We could change that at some point in the future, but it's not, uh, an immediate plan.
355
00:53:38.786 --> 00:53:49.186
All right, so the first one I, I saw this new-- I don't know if it's like a product you're building, the Pydantic.run, which is a Python browser sandbox. What was the inspiration behind that?
356
00:53:49.306 --> 00:53:59.766
We talk a lot about code interpreter for, for lamps. I'm an investor in a company called E2B, which is a code sandboxes as a service for remote execution. Yep. What's the Pydantic.run story?
357
00:54:00.186 --> 00:54:10.766
So Pydantic.run is again, completely open source. I, I have no interest in making it into a product. We just needed a sandbox to be able to demo Logfire in particular, but also Pydantic AI.
358
00:54:11.006 --> 00:54:26.406
Um, so it doesn't have it yet, but I'm, I'm gonna add basically a proxy to OpenAI and the other models so that you can run Pydantic AI in the browser, see how it works, tweak the prompt, et cetera, et cetera, and we'll have some kind of limit per day of what you can spend on it or, like, what, what the spend is.
359
00:54:26.826 --> 00:54:36.586
The other thing we wanted to be able to do was to be able to, when you log in to Logfire, we have quite a lot of drop off of, like, a lot of people sign up, find it interesting, and then don't go and create a project.
360
00:54:36.626 --> 00:54:46.666
And, uh, my intuition is that they're like, "Oh, okay, cool, but now I have to go and, like, open up my development environment, create a new project, do something with the right token. Ugh, I can't be bothered."
361
00:54:46.766 --> 00:54:54.466
And then they, they drop off, and they forget to come back. And so we wanted a really nice way of being able to say, "Click here, and you can run it in the browser and, and see what it does."
362
00:54:54.926 --> 00:55:04.166
As I think happens to all of us, I sort of started, started seeing if I could do it a week and a half ago, got something to run, and then ended up, you know, improving it, and suddenly I've spent, spent a week on it.
363
00:55:04.206 --> 00:55:13.342
But I think it's useful. Yeah. I remember maybe a couple, two, three years ago, there were a couple companies trying to build- In the browser terminals exactly for this.
364
00:55:13.482 --> 00:55:20.182
It's like, you know, you go on GitHub, you see a project that is interesting, but now you gotta, like, clone it and run it on your machine. Sometimes it can be sketchy.
365
00:55:20.682 --> 00:55:27.222
This is cool, especially since you already make all the, the docs runnable in your docs. Like you said, you kinda test them.
366
00:55:27.622 --> 00:55:33.922
It sounds like you might just have So yeah, the plan is that on every example in Pydantic AI- Yeah...
367
00:55:33.942 --> 00:55:48.562
there's a button that basically says Run, which takes you into pydantic.run, has that code there, and depending on how hard we wanna push, we could also have it, like, hooked up to Logfire automatically, so there's a like, "Hey, just come and join the project," and you can see what that looks like in, in Logfire.
368
00:55:48.962 --> 00:55:55.342
That's super cool. Super cool. Yeah, I think that's one of the biggest... personally, for me, one of the biggest drop-offs from open source projects.
369
00:55:55.502 --> 00:56:01.302
It's kinda like do this, and then as long as something-- as soon as something doesn't work, I just drop off, so. It takes some discipline.
370
00:56:01.342 --> 00:56:10.422
You know, like, um, there's been very many versions of this that I've been through in my career where, uh, you had to extract this code and run it, and it always falls out of date.
371
00:56:10.682 --> 00:56:19.442
Often, we would have these, this concept of transclusion, where we have a separate code examples repo that we want with that, and that'd be pulled into our docs and- Yeah... it never wor- never really works.
372
00:56:19.522 --> 00:56:22.082
It takes a lot of discipline, so kudos to you on this [laughs].
373
00:56:22.102 --> 00:56:30.662
And it was, it was years of maintaining Pydantic and people complaining, "Hey, that example's out of date now," that eventually we went and built, uh, pytest examples, which is another...
374
00:56:31.282 --> 00:56:39.282
the hardest to search for open source project we ever built because obviously, as you can imagine, if you search pytest examples, you get examples of, uh, how to use pytest.
375
00:56:39.322 --> 00:56:54.322
But the pytest examples will basically go through both your code inside your docstrings to look for Python code and through Markdown in your docs and extract that code and then run it for you and run linting over it and soon run type checking over it.
376
00:56:54.382 --> 00:57:01.222
So and that's how we keep our examples up to date. But now, now we have these, like, hundreds of examples, all of which are runnable and, and self-contained.
377
00:57:01.642 --> 00:57:08.062
Or if they, if they refer to the previous example, it's already structured that they have to be able to import the code from the previous example.
378
00:57:08.142 --> 00:57:16.782
So why don't we give someone a nice place to just be able to actually run that using OpenAI and see what the output is? Lovely. All right, so that's kinda Pydantic.
379
00:57:16.922 --> 00:57:28.942
In the notes here, I, I just like going through people's X, uh, account, not Twitter. Um, so for four years, you've been saying we need a plain text successor to Jupyter Notebooks. Yeah.
380
00:57:29.322 --> 00:57:37.022
I think people maybe have gone the other way, which may get even more opinionated, like with, uh, X and, like, uh, all these kinda, like, notebook companies. Well, yes.
381
00:57:37.182 --> 00:57:50.102
So in reply to that, uh, someone replied and said Marimo is that, and sure enough, Marimo is, is really impressive, and I've subsequently spoken to, spoken to the Marimo guys and got to angel invest in their, I can't rem- I think it's seed round.
382
00:57:50.502 --> 00:58:01.062
So, like, Marimo is very cool. It's, it's doing that. And they-- Marimo also-- notebooks also run in the browser, again, using Pyodide. In fact, I nearly didn't build pydantic.run because we were just gonna use Marimo.
383
00:58:01.102 --> 00:58:12.042
But my concern was that people would think Logfire was only to be used in notebooks, and I wanted something that, like, ironically felt more basic, felt more like a terminal to, so that no one thought it was, like, just for notebooks.
384
00:58:12.322 --> 00:58:20.402
Yeah. There's a lot of, uh, notebook haters out there. And indeed, my-- I have very strong op-opinions about the, you know, proper no- like Jupyter Notebooks.
385
00:58:20.762 --> 00:58:28.962
This idea that, like, you have to run the cells in the right order. I mean, a whole bunch of things. It's basically, like, worse than Excel or similarly bad to Excel.
386
00:58:29.502 --> 00:58:32.562
Oh, so you, you are a notebook hater [laughs], then invested in a notebook.
387
00:58:33.042 --> 00:58:43.842
I have this rant called Notebook, which was, like, my, my attempt to build an alternative that is but mostly just a rant about the ten reasons why no-notebooks are just as bad as Excel.
388
00:58:44.042 --> 00:58:53.822
But Marimo et al., the, the new ones that are text-based, at least solve a whole bunch of those problems. Agree with that. Yes. I, I was kind of wishing for something like a better notebook, and then I saw Marimo.
389
00:58:53.902 --> 00:59:00.522
I was like, "Oh yeah, these guys have... are ahead of me on, on this." Um- Yeah... I don't know if I would do the sort of annotation-based thing.
390
00:59:00.562 --> 00:59:08.202
Like, you know, a lot of people love the, "Oh, annotate this function," and it, it just adds magic. I think similarly to what Jeremy Howard does with his stuff.
391
00:59:08.602 --> 00:59:23.842
It seems a little bit too magical still, but hey, it's a big improvement from notebooks. Yeah. Yeah, agreed. Just as on the LLM usage, like the IPYMB file, it's just not, it's just not good to put in, uh, in LLMs.
392
00:59:24.042 --> 00:59:35.502
So, um- Yeah... just that alone, I think should be a- [laughs] It's just not good to put in LLMs. True. It's really not. They freak out. They're like- True. Not good to put in Git either. I mean, I freak out. Okay.
393
00:59:35.622 --> 00:59:43.182
Well, we will, we will kill IPYMB at some point. And yeah, any other takes? Uh, I was gonna ask you just, like, broaden out just about the, the London scene.
394
00:59:43.222 --> 00:59:46.722
You know, what's it like building out there, you know, over, over the pond?
395
00:59:47.202 --> 00:59:53.042
I'm a, I'm an evening person, and the good thing is, like, I can get up late and then work late because I'm speaking to people in the US a lot of the time.
396
00:59:53.482 --> 00:59:59.502
So I got invited just earlier today to, uh, some drinks reception about AI at, uh, 10 Downing Street with the Prime Minister.
397
00:59:59.902 --> 01:00:07.822
So I'm, I'm feeling positive about the UK right now and AI, but I think, look, w- like everywhere that isn't the U- US and China knows that we're, like, way behind on AI.
398
01:00:08.342 --> 01:00:18.062
I think it's good that the UK is, like, beginning to say, "This is an opportunity, um, not just a risk." I keep being told, "You should be at more events. You should be, like, you know, hanging out with AI people more."
399
01:00:18.482 --> 01:00:26.262
My instinct is, like, I'd rather sit at my computer and write code. I think that, like, is probably a more effective way of, uh, getting people's attention.
400
01:00:26.322 --> 01:00:31.982
I'm like, a bit of me thinks I should be sitting on Twitter, not on, not in San Francisco chatting to people.
401
01:00:32.442 --> 01:00:38.982
I think it's probably a bit of, bit of a mixture, and I, I could probably do with being in the, in the States a bit more, and I think I'm gonna be over there a bit more this year.
402
01:00:39.502 --> 01:00:48.582
But, like, uh, there's definitely the risk if you're in somewhere where everyone wants to chat to you about, about code where you don't write any code, and that's, that's a failure mode.
403
01:00:48.922 --> 01:00:57.802
I, I would say, yeah, uh, uh, definitely for sure there is a scene, and, um, you know, one, one way to really fail at this is to, is to just be involved in that scene and, and have that eat up your time.
404
01:00:57.842 --> 01:01:07.062
But be at the right events and, uh [laughs] the ones that I'm running are, are, are, are, are good events, hopefully [laughs]. Okay. Well, come to AI Internet. Promise.
405
01:01:07.562 --> 01:01:14.502
What I say is, like, use those things to produce high-quality content that travels in a different medium than you normally would be able to.
406
01:01:14.922 --> 01:01:23.450
Because there's some selectivity, because there's a broad-- there's a focused community on that thing, uh, they will discover your work more, it will be highly produced.
407
01:01:23.530 --> 01:01:31.990
You know, that, that's the pitch over there on, on why at least I do conferences. And then in terms of talking to people, I, I always think about it as a, a three strikes rule.
408
01:01:32.470 --> 01:01:48.110
So after a while it gets repetitive, but maybe like the first ten, twenty conversations you have with people, if the same stuff keeps coming up, that is an indication to you that people, like, want a thing, and it helps you prioritize in a more long form way than you can get in shallow interactions online, right?
409
01:01:48.170 --> 01:01:57.770
Yeah. So that in-person eye to eye, like, [laughs] this is my, my pain at work, and you see the pain, and you're like, "Oh, okay. Like, if I do this for you, you will love, uh, our tool."
410
01:01:57.830 --> 01:02:04.950
And, like, you can't really replace that. It's customer interviews, really. Yeah, I agree entirely with that. I think that, I think there's a-- you're, you're right on a lot of that.
411
01:02:04.990 --> 01:02:10.910
And I think that, like, it's very easy to get distracted by what people are saying on, on Twitter and LinkedIn. That's another thing.
412
01:02:10.950 --> 01:02:20.670
Very hard to correct for which of those people are, are actually building this stuff in production in, like, serious companies and which of them are on day four of learning to code, 'cause they have equally strident opinions.
413
01:02:20.750 --> 01:02:31.430
[laughs] And in their, like, few characters, they, they seem equally valid. But which one's real and which one's not, or, or which one is from someone who really knows their stuff is, is hard to know. Anything else, Sam?
414
01:02:31.750 --> 01:02:36.790
What do you wanna get off your chest? [laughs] Nothing in particular. I think we-- I've really enjoyed our conversation.
415
01:02:37.330 --> 01:02:42.690
I, I would say, I think if anyone who has, like, looked at, at Pedantic AI, we know it's not complete yet.
416
01:02:42.730 --> 01:02:49.710
We know there's a bunch of things that are missing, embeddings, like, uh, storage, MCP and, and tool sets and stuff like that.
417
01:02:50.090 --> 01:02:58.270
We're trying to be deliberate and do stuff well, and that involves not being feature complete yet. But, like, keep coming back and looking in a few months because we're, we're pretty determined to get there.
418
01:02:58.690 --> 01:03:07.050
We know that this stuff is like, whether or not you think that AI is gonna be the next Excel, the next internet, or the next industrial revolution, it's gonna affect all of us enormously.
419
01:03:07.090 --> 01:03:17.210
And so as a company, we, we get that, like, making Pedantic AI the best agent framework is existential for us. You're also the first series A company I see that has no open roles for now.
420
01:03:17.730 --> 01:03:23.750
Every founder that comes on our podcast, the, the call to action is like, "Please come work with us." [laughs] We are not hiring right now.
421
01:03:24.150 --> 01:03:31.230
W- I wanna-- I would love, uh, bluntly for LogFire to have a bit more commercial traction and a bit more revenue before I, before I hire some more people.
422
01:03:31.570 --> 01:03:40.690
It's quite nice having a few years of runway, not a few months of runway, so I'm not in any, any great appetite to go and, like, destroy that runway overnight by hiring another- Yeah... another ten people.
423
01:03:40.810 --> 01:03:50.810
Even if, like, we h-- the whole team is, like, rushed off their feet, kind of doing, as you said, like, three to four startups at the same time. Awesome, man. Thank you for joining us. Thank you very much.
424
01:03:50.810 --> 01:04:03.690
[outro music]