Reducing Entropy in Agentic Software

WEBVTT
00:00:13.000 --> 00:00:46.000
I'm Dan Gerlanc, and welcome to Agents and Engineers, the podcast about agentic AI and software engineering. Today we have Jacob Young with us. He's the founder of Sancho Studio, a boutique software consulting agency. Before that, he was a security software engineer at Zoom and a software engineer at Keybase, among other roles. Jacob, great to have you with us today.
00:00:46.000 --> 00:00:49.000
Dan, good to be there.
00:00:49.000 --> 00:01:26.000
Yeah, so we had chatted a little bit before, but I know that you've actually, as part of your consulting work, done some technical due diligence for companies that are going into deal pipelines or different things like that. And today you're seeing, of course, people using agentic, software engineering, vibe coding. How does that How does that look when it gets to these make or break moments? What have you seen there?
00:01:26.000 --> 00:02:32.000
Yeah, it's interesting, to, to speak kind of broadly, I don't think any of the best practices have settled right now. so when people are kind of getting to close on deals, both buyers and sellers don't have a list of what good looks like yet. and so oftentimes it's the role of someone doing due diligence to make that assessment knowing that the target is moving. Which is super interesting to say that, the models were one way six months ago, now they're a different way today. And we're more of advising on setting vectors and saying, you know, you want to see your teams and your assets heading in this direction, and how they use AI and AI tools rather than being prescriptive and saying, yeah, only use Codex or only use Claude or have to have MCPs or LSPs or skills, none of that. It's more of assessment of are they able to contain and wrangle what they're working on or are they able to kind of lose track of what's going on with the number of people and the number of agents floating around.
00:02:32.000 --> 00:02:44.000
And have you seen practices between companies that do well with agentic coding versus not?
00:02:44.000 --> 00:04:13.000
Yeah, so this is interesting. It might get into my big thesis on agentic coding both as like a user and someone who assesses teams, which is that I I I like to look for convergence is like this word I've I've stuck to. It's like no matter how big the team is, no matter what the software is, no matter what the agents are, are they all converging on the same grounded reality of what the software should be? Not at a line level, but at, you know. Are you randomly introducing dependencies? Are you breaking the tests? Are you writing one-off scripts with emojis in them? That sounds like kind of not convergent. And so looking for teams that are saying, okay, how can we make our processes convergent? And that might just be prompts, might be skills, really good things are like languages that have language server protocols, LSPs, right? Other linters that can give fast feedback to your coding agents. All those are little tools in the arsenal to help teams be you know, consistent with what they're working on when everyone is kind of on their own with these agents, rather than sitting shoulder to shoulder and talking about the code, they're sitting shoulder to shoulder and saying, what do we want to design and build? And then they go off on their own with their agents where no one can see them. Then they come back with these PRs. The question is, you know, are they following the practices that the rest of the team are doing? And so that's basically what I've seen work. Anything to keep things Consistent and cohesive.
00:04:13.000 --> 00:04:31.000
And so is that something like there is a shared set of skills and running everything through linters, tests, and CI or running automated PR reviews, things like that.
00:04:31.000 --> 00:06:13.000
Yeah, I think that there are a lot of interesting things that like for example, skills are helpful, right? LSPs are helpful, linters are helpful, code review is helpful. The most helpful thing, the the thing I think that leads to convergence the best is if you codify what good looks like. And that doesn't always mean for example, like we know that agents are great at tests, insofar as if you have a big test harness and you can get the agents to say, you know, the tests will pass, that's useful. But codifying what good looks like is more of these are our standards and practices, this is what we know we like to see. Don't introduce new abstractions, these are our existing abstractions, reuse them. So you're trying to constrain the agents to reinforce existing patterns. That can show up as prompting, that can show up as skills, that can show up as like again, linters, for example. I'm sure we'll talk about Golang and other programming languages have excellent linters that you can define your own. And I've seen teams who say, like, great, let's just turn all of our best practices into lints and we'll run them against our agents and that'll give them feedback. so it's a whole arsenal of things, but I think the the the theme and the best practice is actually at development time rather than at code review. And I think a lot of teams say, hey, we're gonna sling code and we're just gonna let that whole PR system you know handle it. We'll let either the agents do review or or our teams do review, and I think that's the wrong place to do it. too much information asymmetry. I've got f forty thousand lines of code to review, it's like how can I know if it's any good? so bringing that into the development cycle, the skills, LSPs, linters, all the above are excellent.
00:06:13.000 --> 00:06:41.000
So do you see those being run like the linter is on every change in code or every commit? Like I know in my practice with if I'm using Python, I've done things like add a Claude commit or a Claude workflow hook that just anytime a file gets changed, I run the linter or not every file that has been updated, something like that.
00:06:41.000 --> 00:07:47.000
Yeah, I think that the hooks in Claude and Codex, I'm sure has them as well, are extremely useful. The it's it's a fine line because in the kind of context engineering standpoint, if you introduce noise too early on, you can kind of get your agent conversation or your to to derail if it's seeing errors too early on, it'll just get fixated on those errors. So it's a very, very thin hook, like you were saying, usually on file write or on commit. so I find like formatting on file write and linting on file write are great. And then you can if there's a test suite or a bigger linting suite, doing that on commit is better because commit represents like the agent's like kind of finished what it's been doing or thinking of and can handle a larger piece of feedback, like 40 errors or 40 failures of test cases, but on file write is usually the wrong place to do that. So like little little nudges, LSP on file write formatting on file write and like little lints on file write and then bigger things on commit hooks have kind of I've seen steer towards better output.
00:07:47.000 --> 00:07:55.000
And are there larger documentation or architectural patterns that teams are adding, like having documentation
00:07:55.000 --> 00:07:56.000
Mm-hmm.
00:07:56.000 --> 00:08:17.000
that's here's an architecture markdown file or here's specs and plans. Something is that something you've seen as part of how to keep this convergence best in line with where the code's going.
00:08:17.000 --> 00:09:21.000
Yeah, every team I've ever seen myself when I've been developing and teams that have come across kind of as twins or in due diligence, every single team has had some asymmetry of like, we can document our entire code base, we can set up our architecture, and the minute they do that, it's immediately out of date. and I think people don't value how much work it takes to keep kind of architecture diagrams up to date. because there's no mechanism to make the code re-render or re-document your architecture the minute you've changed it. and so it's a it's a fine practice, but it requires a lot of hand holding. And so more often than not, I see teams that have these 40, 50 markdown files on architecture documentation and API boundaries and microservices. And I go and I look under the hood and I'm like, that was two weeks ago and it's already irrelevant. so there's this waste in documentation that I don't think is really useful unless you have some sort of system to rerender or regenerate your documentation based on code as being the source of truth.
00:09:21.000 --> 00:09:29.000
And have you I mean I would think that having them be out of date is actually harmful or could be harmful because now
00:09:29.000 --> 00:09:30.000
Mm-hmm.
00:09:30.000 --> 00:09:48.000
you're anti converging from where you want the code to go. You're giving the models wrong information. Have you seen is there any tooling or systems that you've seen to keep these in line in an automated fashion that are effective?
00:09:48.000 --> 00:10:45.000
I haven't seen them myself. I'm sure that if you went on GitHub, someone has made 10 versions of this. The Claude documentation, I think the Claude team has put out some really interesting tidbits. And I don't know if they've made this as a part of the Claude Code tool, but they mentioned at one point internally that they have a system called Magic Docs, which is that they've wired up a bunch of documentation and code files that are dependent on one another. And they've done the hard work of basically saying, when these set of when this directory or when these set of code files change, trigger an agent to go like take a look and say, do I need to update the documentation? that seems internal to them and I haven't seen a single tool that does that. But I imagine there are plenty of teams who are thinking about this agentically first and would probably arrive at the same conclusion. but I haven't done it myself in my client work and I haven't seen anyone else do it coming across the desk it's pretty early days.
00:10:45.000 --> 00:11:26.000
Yeah, I've seen I know in Claude, if you ask in Claude Code questions about Claude capabilities, you can sometimes see it perusing its own internal documentation in a progressively disclosed way. And I think Codex, the OpenAI Codex team, they also I saw an article about how they set up documentation for new projects, but I haven't similarly seen any project that actually implements this. But always interested to to hear if someone has tackled it. I don't I don't think quite yet.
00:11:26.000 --> 00:11:30.000
Yeah, I don't think so either.
00:11:30.000 --> 00:11:51.000
And you've you've worked in security. What do you see as the biggest security risks or that you've things that teams might not be paying attention to with agentic development?
00:11:51.000 --> 00:11:59.000
Hm, do you mean on the security risks of using agents in development or security in general?
00:11:59.000 --> 00:12:26.000
More if they're building when they're building software agentically, how do they keep track of or how do they do continuous auditing for security versus before someone in theory would be doing this as they're building the code, now you kind of have to do it post hoc in a sense.
00:12:26.000 --> 00:15:03.000
Hmm. I think that suite of problems is really amenable to the code review workflow system. I think that if you have strong security engineers on your team or a CISO who has great policies, those are very easily codified into 10, 15 different individual prompts that you can run and chain together in a code review process and say like, hey, did we degrade the bit length of our keys to sign cookies? Did we not do SQL injection protection? Did we and you can just anything from the OWASP, right, you know, top ten security things, you can pull in a lot of these practices. They're very easily represented in text and instead of rules, and the agents love to follow them because they like being helpful. biggest issue is just not having that at all. If you're slinging code and you're putting it into production and you're thinking, yeah, my agent along the way is gonna look for Am I using AES-ECB versus AES-GCM It's gonna know the difference. It's not gonna know the difference at all. It's gonna say, great, I've seen this in my training corpus a bunch, and unless you know the model, the frontier model company is doing all the hard work of you know putting those decisions into the system prompt or into the harness, like you can get anything out. I've seen people use the wrong number of iteration cycles on hash functions in production, right? And why and you ask why? the agent told me to do that. I'm like, that number's off by an order of magnitude. That's completely wrong. and so taking a step back, you know, again, leveraging the existing security team and knowledge that you have, codifying it and using really strong and rigorous gates, I think is probably the best thing all teams can be doing right now. especially given that, you know, things like Claude Mythos and and all these other agents are doing kind of vulnerability, like stringing together really complex vulnerability detections, the onus is on software development. teams to actually like bake that into their process rather than wait for another agent sweep to kind of come along and say, here are all the vulnerabilities. Because attackers can exploit these things at way faster rates than they were able to. Way faster. So by the time you put it in production, you know, if someone's running an autonomous loop of agents trying to break your, you know, public API or whatever it is, like you could while you were sleeping, they could have already chained together a perfect exploit. so it it's a very fun cat and mouse time right now and I think that yeah, teams and and software developers need to take a lot more responsibility than they used to just because of the rate of escalation that's happening.
00:15:03.000 --> 00:15:27.000
Yeah, that's definitely something I've thought about and discussed with some other folks that yeah the speed at which this happens now is just so much faster than anything before, so it's not a matter of if it's about when and that when is sooner as today, not in a week or a month.
00:15:27.000 --> 00:16:46.000
Yeah, I mean the nice part is that all of the security skills leverage expertise is actually sitting in these models. It's a matter of how much do you care about the quality of what you're building, right? That a lot of teams are feeling these new tools l as if they're like a energy drink or a boost or like a a rush to get them to do what they were doing faster and they're hoping and praying that along the way it's gonna be secure, but I think are on what you're building and whether it's a greenfield or a brownfield project, you need to step back and say, like, I can get better security in my software if I'm willing to slow down and think versus, you know, my security is quote good enough and I'll let the model think for me. yeah, biggest I my background is in cryptography, so biggest mistake I see is people misusing cryptography. Now that the quote, the agent can do encryption for me, I assure you the agent cannot. Like the a these agents models are terrible at cryptography. they make very convincing usages of functions, hash functions, signing, key generation. It's all equal looks right and you know, the oldest story in cryptography is that, you know, all you need is one bit of information in order to string together a vulnerability. So yeah helps to pause, slow down, step back when it comes to security. Still.
00:16:46.000 --> 00:17:12.000
Yeah, that that has been my experience is where things need to be exactly right is where the agents tend to not work as well. Like the exact set of steps or something where the average the average answer isn't necessarily the one you write. It's that you need the exact answer.
00:17:12.000 --> 00:17:35.000
Yeah. again, unless you can like know that answer ahead of time and you're someone who has that experience, the agents are gonna do great. And if you're not, then you're just kind of at the whim of hoping you get a good prompt. that's a pretty scary place to be sometimes. And so from the the security standpoint, it's a mixed bag. you see everything. it's really interesting.
00:17:35.000 --> 00:18:01.000
So we've you'd mentioned before you kind of have a general thesis around working with LLMs. I think we've maybe hit on a few points about it, but what what is the what have you come to as kind of a guiding principle from your time working with different companies and really everything that's been changing in the last six months, a year?
00:18:01.000 --> 00:21:36.000
Yeah, it's been interesting. a year ago, almost exactly June of last year, I was a bit of detractor, right? We were kinda in the sonnet era of models and they they were good, but they weren't great. It was still very much like pasting out of chat. So I was thinking to myself, you know, they're not quite there yet. We'll see what happens in a year. Same story as everyone else. October, November rolls around, Opus comes out and you're like, yep, this is actually quite good. My thinking has evolved massively in that time. Obviously I'm I'm very pro coding agent and I think they're incredibly useful and powerful. I've been surprised at two things that I think are missing, right? And I talked on one of them already, which is that I don't think that software engineers or even entire companies have a good understanding of convergence. They have a good understanding of shipping software and code. They have a good understanding of developing and building things and they can measure that. But they don't really think or care about, are all of my engineers actually playing in the same key? Are they playing on the same tempo? If you're gonna use kind of the the musical analogy. It's like I've got this band, everyone's got an instrument, but I'm letting them play like totally out of key, like a middle school orchestra or something. And it's kind of comical to watch everyone like use their own agents, use their own models, have their own systems. So aside from the processes, No one is really thinking about, am I like converging the output of all of these different teams using different tools? That's one. And then the kind of bigger thesis is actually that coding agents can make code very messy very quickly, even today with the better models. It it takes almost no effort to take a code base that was cohesive and had nice abstractions and turn it into something that has 40 duplications of the same piece of code. It's got 40 different ways to do things. And as a cryptographer, someone who's done cryptography, right? Not a cryptographer, but software engineer who's worked in cryptography, right? That immediately sounds to me like entropy, right? And I'm like, yes. You know, you know, when you work with cryptography, you're trying to maximize entropy to make something secure. When you're working with like text, like and you're writing English or you're writing computer code, you actually want to minimize entropy. You want to find that path through the maze of designing and building something where You can create the right abstraction, that's good enough to get the job done. That simplifies the problem for both yourself and your and your coding agent, but you know, it doesn't take so much effort to write like the Haskell pure function version of it. And my kind of big take is that we need a measurement for entropy in computer code. we have a measure of entropy in the English language. We know what the English language is supposed to look like, and when you put random characters out there, that's further away from expected English, right? computer code doesn't really have that measurement because it's textual, but it's also logical. and there might be a hundred ways to open a file and read the contents of it, right? And so there's no expected way that a computer program should be sequenced or or indexed. And so it's a very difficult, or I don't even know if it's possible to measure entropy in computer code, but I think it's a very important measurement to think about. with coding agents because you want your agents to be reducing the entropy that is being produced rather than increasing it. And that kind of ties together with both teams and processes converging, but also the computer code itself being like lower and lower entropy rather than higher and higher.
00:21:36.000 --> 00:22:13.000
Yeah, so there are some measures like cyclomatic complexity or the number of branches, things like that. Have you seen teen or have you seen processes used to measure and explicitly track this complexity, or how do you think we can do it in a way that automatically helps agents to make this as something to optimize. Or are we are we just not not there yet?
00:22:13.000 --> 00:24:45.000
Yeah, I think from like an academic standpoint, I don't even know if we're there yet. I'd love to s I've looked it I haven't seen the research I was expecting. cyclomatic complexity is a really interesting one. It's been around a long time and it's just a measurement of like how nested and and complex a compu a computer program, how many branches are there, how many conditions, how many ways are this or can this function or this program branch and hit different things? And that's that's a decent measure of complexity. but kind of similarly like any metric you o optimize for can be kind of misoptimized. And so there are a lot of numbers to stitch together. I've been playing with a tool that I've been building internally. I'd love to release it one day. It's not really where I want it to be, which is a combination of the size of a code base, how connected it is, how cyclically complex it is. But then I think the real juice on top of that is can you statically understand if there are many functions that do the same thing, but slightly differently. And I think that's really where the measurement of entropy happens is like, you know, dryness is like a good example of low entropy. It's like, if I don't repeat myself, then there are fewer places where this thing happens. So it's slightly less entropic, right? So there's just, you know, more organized rather than more disorganized. And as you build more and more complex systems, you expect it to become more complex, right? So I would expect the cyclomatic complexity to go up if I'm doing something like a, you know, f autopilot on a car or in a plane, right? That that that makes sense, right? So that's not a the best measurement in and of itself. And so I think all of those numbers kind of together maybe tell a story and it's very difficult to balance like, okay, do I care about complexity, do I about how how can how deep is the dependency graph? That's another measurement of is it really messy or not. so there's a combination of like structural organization on the file system. There's a measurement of how complex it is, there's a measurement of like how duplicated it is. And I I don't maybe maybe that's another model to train perhaps on all of these things. Who knows? but it's a really interesting question statically. And I've been tinkering with building kind of scores for this and the number one lesson has been it depends, right? There's there's no universal answer. And that you end up tuning it to the specific use case and you still, you know, it's it's subjective at the end of the day. but I think every team and every company probably needs some internal measurement of that fit to their use case. And if you're not thinking or measuring that, then you don't really know if you're, you know, you know, scrambling scrambling what you're building or not.
00:24:45.000 --> 00:25:52.000
Yeah, I could think is there some way of clustering data using kind of data science tooling, clustering kind of functions in a sense that do you have basically the same instructions that only differ by one line, something like that, or two lines, right? Like these semantically should look pretty close, or even if you're doing something that's pre like a prefix analysis of generated code. And but yeah, it's gonna be different, I imagine. Let's say you have a library that just has a lot of connectors to different tools, like a data processing library, right? A lot of those are gonna be pretty different and probably not connected to anything else except for some common logging or other things. But even then, yeah, if you didn't if those were the logging code was repeated every time, you'd probably see that as a there's something that should be centralized.
00:25:52.000 --> 00:25:53.000
Yeah.
00:25:52.000 --> 00:25:53.000
Yeah.
00:25:53.000 --> 00:28:45.000
A good proxy for this is that there are a lot of programming languages in the training corpus for frontier models that are relatively consistent. I can think of a lot of languages surface the kind of best practices and common structures into the programming language itself, right? you could think of like a Golang, right? Sometimes a Java, even like we'll say Rails being a kind of convention. style framework built on top of Ruby, you know, these programming languages and tools like surface the quote right way to do things in the text themselves. And then you train a model on that, which is auto-regressive, it it determines the natural structure of text. You know, that's the whole point of the models. You give them raw text and then they they interpret and understand the structure of these things in higher dimensional vector spaces and then they retain them with memory and you know pass it through all these different layers of the model and all of a sudden it it can understand structure, right? And if today, with the lack of like measuring things statically, as you said, I think a really good proxy is like, okay, well, the programming languages can be translated between each other way more easily than they used to be, right? That's what that's what these LLMs give us, like translation, right? From English to other languages, from one programming language to another. Then the kind question becomes if the cost of changing your programming language is relatively small, and you're a team that used to write Python, for example, would you get a speed up by switching to something like Go, right? Which is like in my opinion, a great language built by Google, maintained by Google, f wildly boring, right? Super uninteresting, right? It's like almost terse to be, it's almost like offensive to programmers sometimes. And that that's how it was designed. But the byproduct of that is like, my gosh, like it's language model. fuel. Like they love that stuff. Like they love a if error not equals nil, repeat every single time. They'll never mess it up. And so there I've there's an interesting decision to be made. Like, if you want to reduce the complexity or the surface area of what you're building, you want to get better results at, maybe changing programming languages is actually a variable in that equation, right? And to go from something like a C or a JavaScript, right? To something like a Rust, a Go or a a Rails even. Like that might be a really useful thing. And you know, you could as as has been done and kind of famously popular, throw an agent at something overnight, fall asleep, wake up, and say, hey, now it's in Rust. It's an interesting world we live in. And I think programming language choice is a great variable to control when you care about complexity, consistency, convergence. You want things to be well behaved when you're in a fully agentic workflow in a software development lifecycle, try another programming language. Might help.
00:28:45.000 --> 00:29:05.000
Yeah, so I've seen I've seen this actually with Wes McKinney, who wrote the Python Pandas library originally lately. I think all his code has been Go on the back end plus JavaScript front end. Have you seen Teams doing these rewrites?
00:29:05.000 --> 00:31:26.000
Yeah, so I have actually. I've both seen it happen on its own accord and I've also been in I I've had clients where I've been a tech lead and I've also made the same suggestion. Python's always had a long history of being backed by C, right? That's the kind of like Python's secret is like most of all the Python libraries are just C, which is great. And that that's what makes it so useful and fast, especially for data science and ML. I worked with a company in the healthcare space that ended up doing that. They were a kind of TypeScript first team. TypeScript is actually a great language for just typing in general. Like the f the type system is fantastic, really well designed. fun fact, I'm sure this probably crossed a lot of people's like the TypeScript team within Microsoft rewrote the TypeScript compiler in Go, right? That's like pre-AI. They did a one-for-one port for the same reason, performance, you know, errors, etc. but when I was working with this company, they similarly made the same transition. They had modeled their domain really, really well. They had used a type system, they had a nice API boundary, they had good RPC boundary, data model, ORM, everything. And it's extremely amenable to point a language model at a well structured piece of code and say, hey, you know, do a one-for-one port. Here's what I care about, you know, here are all like the nuances of the language, this is what goes over, this is what doesn't. Some generics yes, some generics no, alba al algebraic data types yes or no. and they're able to do a really good job. And then you can spend a minimal amount of effort cleaning things up and making those changes. That team got concurrency for free, right? You know, Golang being a highly concurrent language. so you know, immediate speed up in there, API processor P99s go down, their P95s go down from like, you know, 800 milliseconds to a 150 right way faster. It's much more reliable. very consistent, strong tool chain. And these days they also remove themselves from a whole suite of npm supply chain attacks. you know, Python and JavaScript are getting hit pretty hard. not to say that the other languages are immune, but at least for the time being it's lot harder to attack a the cargo Rust supply chain or the Go supply chain, because they're compiled.
00:31:26.000 --> 00:31:54.000
Yeah, that is one thing I've wondered with Go being that it has a pretty strong standard library that you can almost get by just with the standard library or a few select packages versus Python or JavaScript. Yeah, you're you've got a pretty deep dependency chain pretty quickly in most cases.
00:31:54.000 --> 00:33:44.000
Yeah, I wrote a blog post called Use Boring Languages with LLMs and it was my my my pitch for using Go for the majority of things. You know, Keybase used Go, Zoom used a lot of Go. and that was purely, I guess, driven by Chris Coyne and Max Krohn because Go had a good cryptography story actually. There were good cryptography packages in Go that weren't linked to OpenSSL in a lot of ways. and there's obviously still kind of custom work that was done there, but by and large, you had the backing of Google and a bunch of awesome applied cryptographers in the standard library, as you said, plus all of the amazing standard library features you get in Go, like HTTP routing, file system work, concurrency, et cetera. it makes it extremely useful and you get these nice clean binaries. that are statically linked, they're cross-platform, they've got garbage collection built in. And so your maintenance cost of your software is really low. your supply chain risk is extremely low. Like you get to do all these fun embedded or web service first things with minimal dependencies. it's a natural kind of language model programming language. it's not perfect, but it does so much of what you want from a computer program, but like you said, for free and by default, thanks to the standard library and the tools. You get Google putting billions of dollars behind it every single year. and anytime you need to jump out and use something else, like a JavaScript for the web or anything, you know, GPU related, if you're doing that, like you can you can break break the mold and and jump outside. very helpful these days.
00:33:44.000 --> 00:33:50.000
How about the Rust verse Go choice? How'd you feel there?
00:33:50.000 --> 00:36:22.000
Yeah, I'm I'm less of a pedant on the particulars of programming languages. I've written a lot of Rust. I think it's one of the best modern programming languages, extremely thoughtful, it's extremely powerful. I think a lot of people should be using Rust and I think it it does a lot for security as well. I think Rust's kind of arc on cryptography is kind of hitting its stride. It has a long history in these like Ethereum contracts and other blockchains. That's kind of like where it's cutting its teeth, but I think it it's still yet to break out into the the world of we'll say like TLS and start replacing kind of core infrastructure in C. It's getting its way into the Linux kernel. but from a coding agent standpoint, Rust has a very expressive type system, which is its superpower, but it may from time to time lead a language model astray, right? And I haven't seen language models build beautiful abstractions in Rust. I've seen them do a lot of great work with the borrow checker and with the type system, which is what everyone says. Like, wow, I put Codex against Rust and it got all these awesome borrow checking and type errors was able to resolve concurrency and mutexes and all this like you know shared data access. And like, wow, it's so great. And then you look at the code and you're like, holy crap, this is terrible Rust code. Like there's no traits here, there's no generics, there's there's nothing. Like and you look at the Rust standard library, look at these awesome packages that people have written in Rust that abstract away. All this amazing complexity into the type system. And then you look at what the language models do, and they feel like kind of like a a a tenth grader with a crayon, right, trying to design a rocket ship. You're like, come on, like this is terrible. so I'm not a not a big Rust vs. Go person. I think they're both great, but I'd love to see the type system used more powerfully by language models. And that might that might be someone who comes along and either fine-tunes a model or it might be a frontier lab that says, no, actually we're a Rust first company. I think of OpenAI actually being as a pretty heavy Rust user. I think Codex is written in Rust and they've got a they've they purchased the Astral, right, which rewrote UV also in Rust. So it seems like they have like a language stack internally that might be Rust first. And I would be I wouldn't be surprised if in you know the next six to twelve months that OpenAI becomes the company and says, yeah, we we write Rust better than most people because Chat GPT you know, seven point whatever is trained on it better and we use it internally, here you go. And that's that's your gateway into writing better Rust. But for today it seems like most language models spit out Go a little bit better.
00:36:22.000 --> 00:37:42.000
Yes, definitely the language choice one versus the other. It's just what different trade-offs, right? And I think with language models and I early on I tested I built something in Go, a CLI tool in Go, and a CLI tool in Rust, because I wanted to see the comparison, how agents did with it and yeah, it's still My finding was yeah, you have the trade-off, your compile time is basically nothing in Go. Like it's as fast as Python or faster sometimes, actually. Versus Rust, yeah, you have if you're not using the type system well or the agents not, you lose a lot of those advantages. Plus you have a slower compile time. And you still the Rust standard library is also a lot smaller, so you're still you're still going out into a pretty deep dependency chain for most libraries I've found. So yeah, interested to and definitely interested on on my end as well to see how that shakes out the next six to twelve months. Can the languages, can the language models get better at the abstraction layer here?
00:37:42.000 --> 00:37:48.000
non?
00:37:48.000 --> 00:38:02.000
And what have you have you seen at the individual developer level? Is there a big difference between folks who get a lot out of LLMs versus those who don't?
00:38:02.000 --> 00:38:17.000
Yeah, I've been I've been fortunate enough to both like advise and lead a few teams in the last couple of years. The multiplier on we'll say like unproductive or developers is
00:38:17.000 --> 00:38:18.000
Okay.
00:38:18.000 --> 00:41:39.000
quite high. I've seen quite a few teams who have like a relatively nascent, maybe it's a startup, for example, and they have this amazing ability to move quickly, and all it takes is one developer who keeps slinging bad code to drag the entire team down. And this can be a team of five people to 15 people, right? so these these are force multiplier tools and the best developers are ones who have cut their teeth before AI. They have scar tissue to show for computer programs and software. When they're prompting their AIs, they are intimately familiar about what goes wrong and they can see into the future and know, like, yeah, I don't want you to do all of these things in advance because I know that those are bad practices and they go wrong. For example, like I don't want you to pull independencies, right? I don't want you to break norms, I don't want you to log PII. Like they'll say all these things in their prompts. They'll build all their skills to reinforce this. And at the individual developer level, the kind of I want to hit the moon with a slingshot kind of energy of not just vibe coders, because technically I'm vibe coding, right? I'm writing computer code. I'm writing prompts. I'm not even looking at the code because I know what how it's supposed to behave. And any aberrant behavior like I pick up on immediately. But there are a lot of vibe coders who have never had the background of writing code by hand, suffering painfully through debugging processes, making mistakes in production, and they'll try to shoot for the moon way too early. and those people in a team are force multipliers for drag. They introduce way more bugs than are than developers used to introduce. They they they break the coding patterns way faster than they used to. And I've seen some teams who have effectively You know, it's funny because I've seen teams who have effectively like let go of team members who are slinging bad code much quicker. In theory, you would expect, I have a one developer who's not even junior, but maybe isn't even, you know, playing on the same tune as everyone else. In theory, because they have coding models, they should be able to participate in the common practices much easier. but I think there's this like addictive or, you know, very hard to turn away siren song of, you know, I can just prompt my way out of a lot of things. If if the tests are broken, I'll just say fix it, right? Two words. and those engineers I've seen teams like let go of them actually quicker because they realize that they can do more damage and the cost to actually train them is a lot higher. so it's funny how coding models should be something to help developers upskill, but also they're a net negative as well. So yeah senior engineers are having their moment right now. not to say that junior engineers don't have a place, they do, and they deserve to be trained and and worked with, but I think kind of almost like when you go to math class and they make you write down the equations on pen and paper, just as for the exercise, even though the calculator can do it. I would expect, you know, people who don't have as much professional or production experience need to slow down, write things by hand, sit with it quicker, gain an understanding and an intuition. so that they're actually helpful and productive rather than net harmful.
00:41:39.000 --> 00:41:52.000
So would you tell junior developers that hey, even though you can do this with an LLM, you've gotta actually go and spend the time and learn. You can't just be building stuff with models.
00:41:52.000 --> 00:41:55.000
Yeah. Yeah.
00:41:55.000 --> 00:42:00.000
So you actually have to understand what's happening here.
00:42:00.000 --> 00:43:56.000
Yeah, I don't have a link handy, but Brown, Brown University in the CS department recently ran a really great undergraduate course where I think about last year, they started teaching computer programming at a really early level to CS undergrads only using LLMs. And the point of the class Brown being a very in multidisciplinary school being, hey, We're gonna make you use these LLMs and in the process of using them, we're going to learn about what your experience was. Were you able to design Tetris? Were you able to design Tetris with reverse gravity? You know, how did your experience prompting an agent work? Did you understand the code well? and so they're using the students both as kind of like learning material, but they're also teaching them CS fundamentals at the same time. so the I can't remember which professors, but they went on Oxide and Friends, which is a different podcast. And they kind of shared their learnings there for about an hour and a half. It's a great lesson. the Oxide podcast. and I think the biggest takeaway is like, you know, these are these are CS undergrads who are trying to learn computer science right as language models are coming out, and they're saying, please don't make me use these anymore. I don't understand what's going on. You know, I completed the assignment, but I I just don't get it. And it's almost like making your kid smoke. a full pack of cigarettes after you catch them smoking one cigarette and they're saying like please no more. I've had enough. And so there's there's a fine line here. And so the kind of message of junior developers is like, let these things assist you, but you're gonna still have to do the hard work. the hard work never goes away. just because you have a calculator doesn't mean you get to, you know, skip on learning arithmetic. and so it's it's the story of learning a skill of the human. And you have to go slow, you have to go at the same pace that your brain thinks. and that's true for software as well.
00:43:56.000 --> 00:44:28.000
Yeah, definitely. I think in some way LLMs actually make it so that you can learn anything. Like you have a tutor, let's say you want an example of something, you could be like, Hey, L model, make me an example, or create a quiz for me to learn something like this. So the tooling is there to learn it, but there's no substitute for Putting in the time.
00:44:28.000 --> 00:46:27.000
Absolutely. I think another really good piece of advice I've given people is that like you just because you can get information out of a chatbot or an LLM, it's almost irrelevant. You can get any information from anywhere at any time. You can do this via Google. The question becomes, once you get new information and you're trying to learn something that's difficult, how where do you put that information and then how do you return to it? I'm a big like flashcards person, right? If I'm really trying to like understand something and maybe even internalize it, I have to revisit it. And a good flashcard system like brings back information repetitively so that it starts to enter your long-term memory. If you're not, if you're just asking chatbots things and you're getting answers, you probably know it goes in one ear and out the other. Like you knew you had the conversation with Chat GPT and you did retain some information about whatever like health diagnosis you cared about, but like, did you really remember everything about it? It gave you this 10 page thing. Like, no, of course you didn't. And so the skill is now in like, okay, I can be tutored in anything. But will that stick? And as a human, like as a person, like how do I learn? And like how do I have to return to information in order to retain it and make it better? You know, I ask chatbots to make me quizzes. I also have them make me flashcards. I have them, you know, give me problem sets where I don't get the answer out. and it's it's all in service of like me getting better. And so I have to artificially give myself challenges. Otherwise I'll never get better. Just having information doesn't make me a better programmer, right? Just being able to like get the answer for some algorithm doesn't make me a better programmer. Doing the work makes me a better programmer. and so aside from getting information from a tutor, you also have to take that information and put it into a system that forces you to have resistance, intellectual resistance, mental resistance in the same way you get resistance going to the gym and training your body. it's the same story as it always was. Actually.
00:46:27.000 --> 00:46:38.000
So do you set aside time every day or how do you make sure that even though you're working with these tools, you're still getting better?
00:46:38.000 --> 00:48:05.000
Yeah, this is a great like kind of like tie back to like I think where skills are helpful. Like a lot of people are using agent skills to produce output. Another really helpful way is to flip it in the inverse is to use agent skills to repackage information and then put it into places that you care about long term. Right? So you might be trying to learn something about cryptography, right? You can ask a coding agent to generate things for you, which is super helpful. But unless you codify what you learned into a set of like problems or challenges for yourself, it's probably not gonna stay. So if I was learning cryptography, I'd say, okay, well I want to learn, you know, about signing and verification, I want to learn about you know key length attacks, I wanna learn about man in the middle things. Could you please turn this into a bunch of challenges for me and make them really difficult? And you know what the answer should be, but make it so that like you this would be like a textbook question. and then you save those on the side and then you work through them on your own without the LLM, right? Just purely like, you know, make me a a puzzle, write write the test cases that that I should be able to pass, then I have to implement the implementation. That's true for information. You built flashcards where it omits, you know, the information you care about, and you can kind of tune it, but it's the same thing. and I think it's really helpful to think of skills as a way to extract learning from information rather than just produce information or produce code. so going backwards is really, really helpful.
00:48:05.000 --> 00:48:12.000
Do you have a set of skills that you use for that or is there open source tools you find?
00:48:12.000 --> 00:50:25.000
Yeah. so a really informative piece of writing was by Andy Matuschak. I think I'm saying his name last his his yeah, I think I'm saying his name correctly. He did the big Tools for Thought guy. pre-AI. You know, he's done a lot of independent research on his blog about what it means to have memory systems. He's got a very famous interconnected set of notes. You think like Obsidian, right? but Andy's a Andy's a researcher and he wrote a paper probably about a month or so ago. I think like Machines for Thought, right? Where he did independent research on like can LLMs and agents be good tools to write flashcards? And his overwhelming conclusion, both empirically and subjectively, was like, no, actually. they're not quite there yet. And I and my memory and my experience in in using these tools is that they're they they struggle to omit information. maybe that's about alignment, maybe that's a system prompt. But it's very hard, I think, for language modules to like think abstractly and think like, you want to know this and I have all the information, therefore I will like omit it. and unless you introduce the right things, you you don't always get the best flashcards out. so I have a bunch of skills that I use to write flashcards in my flashcard app. And knowing that I think their ability to omit information is not the best. So I'm still very hands-on in like tweaking things. I haven't seen anything open source. Andy hasn't really seen anything open source, probably because the research he found was not very like conducive to this being like a one-shot style problem. But it does get you most of the way there, which is that there's a lot of toil in writing quizzes, flashcards, and problem sets for yourself, right? And it will do that work for you, which is extremely helpful, but you'll still have to be your own kind of advocate insofar as, yeah, I got the flashcards and I got the problems, but you have to think really hard, like, is this really going to challenge me? and you can do some some feedback cycles there. but so yeah, they're all homegrown. and I don't think anyone's really come along with like a universal system that does this well. And if they have, you know, who knows if they're any good.
00:50:25.000 --> 00:50:32.000
Yeah, similarly I was looking into recently summarization systems, which I think
00:50:32.000 --> 00:50:33.000
Mm.
00:50:33.000 --> 00:51:34.000
falls along a similar line, and I was surprised that I always thought the latest generations of models summarization was considered more of a straightforward problem, but actually no, it's still largely unsolved. Now what's the best way to do this? And I think this gets back to it. What's what's the information you can leave out? What are the things that need to be included is a hard question and sometimes maybe one that's not easy for a computer to answer because it's w it's really about picking the trade-offs, which ultimately is something that historically we have to answer, right? Do you use Go or Go or Rust, different different trade-offs, but which which do you want and which are most appropriate for your problem?
00:51:34.000 --> 00:52:31.000
Yeah, absolutely. yeah, don't don't subject yourself to a machine that thinks for you. they're extremely useful and kind of AGI conversation aside. yeah, the best practices are still involved with you know doing the hard work yourself, even if there's kind of toil removed. And I've had the pleasure of working with like a lot of great teams that are AI first and AI enabled, and all of them have you know, people as the center of the loop. It doesn't mean that they're hands-on in the code editor, but they are, you know, thinking hard, making all the decisions themselves, and then outsourcing labor, cognitive labor to models. yeah. I I still don't think the kind of Gas Town model is is here yet. even if the tooling is there, for example, like you know, multi-agents or I think like Claude just released like dynamic agent harnesses, right? Which is like ultracode.
00:52:31.000 --> 00:52:32.000
Yeah, I think they just shipped
00:52:32.000 --> 00:52:33.000
If the tool is there that's
00:52:33.000 --> 00:52:38.000
that in a recent version of or with Claude 4.8 Opus 4.8
00:52:38.000 --> 00:52:49.000
Yeah. Yeah. Just because the tool is there doesn't mean that the that the output is there as well. and yeah, it still takes a lot of work for you.
00:52:49.000 --> 00:53:05.000
So do you think that's a component that people are still or some people are overestimating with LLMs right now, really be able to go end to end in software without human a lot of human guidance?
00:53:05.000 --> 00:55:03.000
Yeah. The you know, it's funny, sometimes when you read internet conversations before AI, right, just on the internet, people will comment on any topic, right? And the thing that's always missing is like what is their experience and context, right? might be on like some topic you know a lot about, 3D printers, bicycles, music, right? And people say, this is good and this is bad, right? But what you really need to do is like broad a little bit and say, like, what are you doing? Are you playing in an orchestra? Are you a professional athlete? Are you a hobbyist, right? And you hear a lot of narrative from people saying, like, I can ship all this code fully autonomously. Congratulations. Next question is, what are you actually doing? How are you using it? Are you one person? Are you an entire group of people? Are you a hundred billion dollar Series F company? and we don't let the conversation go that far more often than not. And so Greenfield projects are the probably the the right field for people to pluck from and say, wow, I went from zero to X with all these autonomous agents. Great, awesome. Like talk to me about putting that into customer clouds, right? Talk to me about, you know, sourcing real contracts that are worth money. talk to me about regulatory compliance. Talk to me about security. And most of the questions most of the time those answers are, I haven't gotten there yet. Right. And so there's this asymmetry of like are you a soloist versus are you, you know, working in a group? Is it a Greenfield project? Is it a Brownfield project? We often don't add those qualifiers to the discussion. And so the narratives are very much like look, we've got these multi agent systems that are working autonomous autonomously. That's great. But like, are you building Uber or are you building, you know, an app for your friends? and I think that even the large companies these days, your Googles and your Ubers are probably pulling back a little bit as well for the same reasons. a little bit harder to run these systems autonomously in Brownfield. projects that already exist.
00:55:03.000 --> 00:55:35.000
Yeah, folks I've spoken with, I found the pattern tends to be either running a few agents at a time, maybe two or three on parallel work. I have talked to people who are running more software factory type systems, but they have of folks I've talked to so far have all been mostly working solo. So
00:55:35.000 --> 00:55:49.000
Exactly.
00:55:35.000 --> 00:55:49.000
have not seen that in a larger org yet. Of course, if it would be interesting to always always wanna hear how that's working, but have yeah, have not seen it yet.
00:55:49.000 --> 00:56:47.000
Yeah, as a consultant, I'm I'm oftentimes brought in to try to make that a reality. And a lot of the art of consulting and and giving leadership advice is to say, like, you don't need to shoot for the moon. You can't fire all your developers yet, but you can get a lot out along the way if you choose the right path. and I think enablement is gonna be here for a long time, unless of course someone cracks AGI. That's a whole separate conversation. Personally, I'm not really thinking we're gonna get there. but for the Brownfield existing companies, you know, the number one source is toil, right? And then there are a few areas like security where you get kind of multiplicative advantages, right? Which is like just with the right thinking and the right constraints, you can eliminate a huge class of your security problems. And then you add the right guardrails and you can maybe ship a little bit faster, but you can't take your hands off the wheel and you can't take your eyes off the road yet.
00:56:47.000 --> 00:56:59.000
Do you think there is now a ideal team size that's smaller than it was before because we can get more done with agents?
00:56:59.000 --> 00:58:06.000
Yeah, yeah, there is. things like QA engineers, right? like the number of for example like the number of DevOps people you used to need, right? Kind of go you need like really specialized DevOps people who are really senior. they can have a lot of kind of they can have a lot of impact, right? Just like so again, th reducing the headcount not for skill, but for maybe labor or toil is probably like the right metric. And a team of we'll say like six, right, with what a really senior mobile person, a really senior systems person, a really senior DevOps person, and a really senior designer, right, can get a lot more done than a team of fifteen, right? We had three DevOps people, you know, two designers, right? so it's that's a clear benefit and that's here to stay, I think, which is amplifying existing skill sets. and I would expect Teams to be a lot smaller, but you don't want a team of one. team of one is too small.
00:58:06.000 --> 00:58:27.000
Right, you still once you get, I think, to highly specialized skills, there's just not you can't be practicing all the things you need to know to be competent in them. It's just not possible. So y that that can't go that's not gonna go away, I don't think.
00:58:27.000 --> 01:00:14.000
No, it it won't. And as someone who like, if you have any amount of depth in your professional background, wherever it may be, legal software, right? Infrastructure, cryptography, the hopefully at that point in your life you know what you don't know. And when you have like a depth of knowledge in one area and you look at a new domain, the very first thing you should think is like, that's interesting. I want to understand that, but holy crap, I probably couldn't do an excellent job. And if you have that intuition and sensation of like I would love to learn about data pipelines, infrastructure, ML training, right? But you also know that like, I'm an expert in this other domain, it should tell you that like even if you don't know everything about this new area, you should know that there's like a high activation cost and there's a lot that you don't know that you don't know. And one of the biggest failures is treating language models as these oracles that do know everything and can tell you everything. When in reality the language models respond to expertise, right? The more of an expert I am in something, the bet the more I get out, right? And the more of a novice I am in something, the more I get novice style information out, but seemingly sounding like expert level advice. and so it's a bit of a siren song, which is that I would never trust the model to tell me everything about how like the intricacies of container orchestration works. E even though it's like public information and I have great intuition about it. Like I would not trust myself to like build Kubernetes from scratch just because a model told me that it works a certain way. Like I know enough to know that there are things I don't know, even if models are helpful. and that's that's a great kind of warning and heuristic for anybody who's trying to leverage agents and just say like I should probably talk to someone who's an expert in this and like have them leverage an agent with me rather than outsource expertise.
01:00:14.000 --> 01:00:48.000
Like with code, the agents don't get the abstraction right always. So expertise is often a combination of both abstraction and using that to understanding to provide the specifics. So until some models can have both or operate equally well at both levels, which I don't think they can right now without steering. You you need that expertise, I think.
01:00:48.000 --> 01:01:31.000
Yeah. Even if they continue to get better, which we think we all hope that they do, right? a kind of an an everything burger, right? Or an everything sandwich of like the whole world's knowledge, like almost by definition, like can't be that great, right? you know, we hope that like we can extract lots of great information from these tools and we hope that we can prompt agents, but I think that like just the breadth of all knowledge and all human information is very, very large and will always require people to, you know, both advise and strategize and ideate and think and steer. and so no maybe in volume kind of our work changes, but I think in depth it it never does.
01:01:31.000 --> 01:02:23.000
Yeah, and I think a lot of the information that makes expertise is not necessarily available for these models to even train on. Like if you've done debugging or I mean have expertise in a specific area, those the expertise you have, is it written down somewhere specifically in the way you've learned it? Probably not. So to expect the inputs that a model would need to get that don't exist because they haven't lived your life or gone they right. I don't I think that There's information you can't get into the system and that's still a major differentiator.
01:02:23.000 --> 01:04:01.000
Yeah, I r I saw this really interesting thing on Twitter the other day, which is a bunch of researchers and data scientists who were were showing a graph of whether or not experiments succeeded or failed. And they're they're plotting, you know, of all the papers written, were the papers written about successes or failures, or were they written about like uninteresting, neither success nor failure? Right. And the the commentary is actually like Really similar to that kind of famous diagram of an airplane in World War Two where that show all of the bullet holes with a plane that come back, right? And it's the classic example of like survivorship bias, which is basically saying, like, yes, all the planes landed back on the airfield and they had holes in these places. You shouldn't reinforce where the holes are, you should reinforce where the holes aren't, because the planes that went down were the ones that didn't come back exactly. And so there's this interesting kind of conversation of like all of our research. Are either wild successes or wild failures. If you want to train a model on all human knowledge, like most of human experimentation has not been documented, actually, because it just didn't work. And so in a lot of ways, like you really want a bigger corpus of like the full experience of knowledge and experimentation of the physical world that isn't really properly represented. So we have these huge swaths of the world that are like not properly captured by the internet. So there's a long way to go. And I think that area right there is where people fit in and will always fit in because the natural world evolves and a lot of it will never be represented as text, right?
01:04:01.000 --> 01:04:24.000
This is a slightly different direction, but managing agents is a different cognitive mode than writing code yourself. How does that feel over a day, a week? Are you more tired, less tired, tired in a different way?
01:04:24.000 --> 01:06:19.000
Yeah. I'm more tired. I think that like I'm burdened by knowing what can go wrong. So I do work with like on ten different projects with like two or three agents on each of them. And I am like holding spinning plates sometimes and I can get a lot done. but I'm aware of the whole world of ways in which things can go sideways. So I'm always very vigilant and like I have like my eyes on this like a hawk. I think in the kind of Steve Yegge like graph I'm like on like, you know, level six or level seven, right? If you're talking to terminal pain, I'm not fully like agenc yet. I do feel like unless when I'm not working on a particular code base or a set of languages and tools that I know really well, I do feel like I have kind of cognitive collapse or skill atrophy. Which is that there's just too much information happening in domains that I don't know enough about and I'm not giving enough time to it. And so I can feel myself abdicating, you know, understanding after enough of these sessions. When I work with things that I know intimately with agents, I feel like I'm still involved. I'm peeking under the hood, I'm looking, I'm asking the right questions, I am sometimes learning from the agents when they do things I don't expect. But it's when I leave my domain of expertise or my my kind of my my background. is when I'm like, great, you're making a mobile app, but like I have no clue if what you're doing is good or bad. And I think everyone has that intuition and it takes a certain kind of person to like let go and kind of say like, you know, I trust it implicitly and I don't want to care or look. It was great to be Dan. It was a great conversation.
01:06:19.000 --> 01:06:44.000
Jacob, thanks again for joining us here today. It was great to have you.