Transcrição completa de Making AI Useful with David Rasch

and building up and building up and building up and eventually get to the UI. Now, you know, of course, the agents promptly beefed in short after, but still, having done the exercise, and this is part of what I find is also really interesting about this, too, is like, I don't think I could have gotten a human to sit down with me and just chat through all of the fundamentals on this for 12 sequential hours. I think if I was doing that with a human as well, if I did get 12 sec 12 hours total, that would be spread out over a number of weeks, right? So being able to David Rasch: Yeah. Hunter Harris: refine my thinking for 12 hours was a very interesting process. And that was kind of that was like a a recent moment where I was like, whoa, like this is. An interesting change to process that's happening for me. But like getting those data foundations there is so important since w once that's stable, once you have things like domain driven design or tight schemas, etc., that really express your data, you and you know, like the quintessential anti-pattern is hey, we accidentally created a God object that has four thousand fields on our database or something and now Anytime we want to change something, it's now terrible and horrific. David Rasch: Mm-hmm. Hunter Harris: Which of course agents are I don't want to say want to do. I think they at this point know a little bit better than that, but still, you can fall into a number of petfalls pretty David Rasch: But their infinite patience can be a downside as much as a good like a good side, right? Where they're happy to sort through Hunter Harris: Mm-hmm. David Rasch: that for hours and hours. Hunter Harris: Yeah, they're happy to enable you to do absolutely horrific stuff if you ask for it. Which is, I mean, that's part of what's super interesting about this moment too, to me, is like there's there's like a whole cohort of I don't want to call them junior engineers because they're not really engineers, but they're becoming engineers through five coding. And they they're going in and Because they don't have the vocabulary that we do. They're asking AI accidentally for some pretty bad stuff. But then they're shipping it and getting it in front of customers and finding out, like, hey, here's some dead ends and painful things, and how do we work through this? And they end up chatting with the AI, because the AI has infinite patience to work with them. And the AI will be like, Hey, hey, have you heard of design patterns? It seems like you're maybe starting to it articulate an inbox pattern. They're like, okay, cool, I have no idea what that is. Go for it. And it resolves the problem. David Rasch: Ha ha ha ha. Hunter Harris: But now they have in the back of their head the words design pattern and inbox pattern in their head. And while they may not fully understand what that is, like it it becomes this really interesting growing set of tools accessible to a wide variety of people. David Rasch: I I hear a couple of

things that are really interesting to me. We could go in multiple directions, because you talked about both, even the narrative, hey, I spent 12 hours talking about the data model, the domain-driven design part of a problem, is almost counter to a lot of the AI coding narrative of I can just go get Hunter Harris: think it is, right? They're like, just one shot it. David Rasch: started, right? Just a one-shot there of like, hey, I've got a lot of experience on how to do these things. I'm building with AI. I'm even, you know, probably in the upper quartile at least of using these tools. And here I am talking about the data model with the AI for 12 hours, right? Not writing a bunch of code, not writing a bunch of tests, but doing actual design. So that's one thread. But then there's this other thread, which is this junior engineering thread, which is a fascinating one and of itself, because there's this. challenge right now with junior engineers where on one hand they are seeing a lot of challenge. Where do I find a junior position where I can cut my teeth and learn? And I can get started coding without knowing a bunch of these things. On the other hand, I have an infinitely patient tutor that, to your point, if I if it's going to put Hunter Harris: Mm-hmm. David Rasch: the dreadtown ground breadcrumbs down and I pick them up, I have this opportunity to learn things in a way. That we used to only do with pair programming or or very intentional mentoring, et cetera, that if I pick those up, right, if I take that opportunity to say, this isn't magic, I'm gonna decompose the problem. I'm gonna think about the pieces, I'm gonna understand what it was talking about and what the trade-offs are. I have an opportunity to get even wiser very quickly. And anyway, so those are the the two things I heard, because those are huge topics in and of themselves. Hunter Harris: Absolutely. I mean l let's talk a little bit about the junior engineers thing because there's like an aspect to that that I I want to make sure I clarify. So when I say junior engineers there, I mean non-technical people that are becoming engineers using vibe coding specifically. But I I think actually that kind of brings it full circle because I feel like right now in this moment, right now, with just the pace of change and whatnot I I just I think if you are a junior engineer and you're like, I want to just get a job, I just don't know if that this is the moment in which that's the wisest thing to do. Like I I honestly in my heart of hearts believe right now, if you're like I wanna do a career in CS, the easiest way to getting hired is to go try and build something and try and sell something, preferably in the opposite order I just said. David Rasch: Which make the paths the same, right? The two just Hunter Harris: Exactly, exactly. Which is fascinating, right? David Rasch: converge because now having the background first versus not having the background first, the actions are the same. Hunter Harris: Yeah, and the thing is the junior engineer types actually have, due to their CS fundamentals, they potentially have a very significant leg up on all the other people that are learning all of this. I mean, as we just said, AI is a very effective tutor. So yeah, I can tell you, but if you come in knowing what like version control is and what databases are and what like foreign keys are and Boolean and all these sorts of things. It really, really enables you to ask AI really effective questions. And as I was just saying, with like the design process too, like go back and forth with AI to fill in the remaining gaps as you encounter them. Like AI is a fascinating fair programmer and can be very effective. I mean, honestly, thank God we're we we got Opus five point five now. Because I have to say doing a twelve hour design session with Opus five probably would have like I I prefer to just like go to the dentist and have the dentist just directly pulling my teeth out or something like that, right? It would be less painful. but yeah, like I I don't know how to express to some of the like CS type people though that are graduating with CS degrees, like, yeah, okay, if you can get an internship at Mana and like slide in there, sure do it. But if you don't, figure out How to build a software thing and figure out how to start selling it. Because I mean, at least that's experience, right? And not only is it experience, it's multifaceted experience, because I really feel like like I feel like the days of the job being like, I'm gonna lock myself in a closet for five days with a fit of whiskey and a pack of smokes has safely passed by a few decades, even. I David Rasch: Yeah. Yeah, there's no there's no dark room coding alone that's really gonna be a growing industry these days. Hunter Harris: Yeah. Like and I I don't really know when that 'cause my whole experience is like engineering is a team sport, right? Even when I kinda like cowboyed it, it was still a team sport where I was trying to involve o other people. I think I just kind of like natively intuited part of that, I guess, or I just got really lucky in terms of my mentorship. I'm not sure. I don't remember ever thinking I was gonna lock myself in a room and do it all myself. David Rasch: But there's also some interest there, right? You probably didn't Hunter Harris: Mm-hmm. David Rasch: want to just lock yourself in a room. I know I didn't. I wanted Hunter Harris: Yeah, no. David Rasch: I wanted to be part of that team sport. I think one of the challenges for some of the folks that have gone into it historically is they did want to lock themselves in a room. And so there's a reckoning with that identity part of it. I know that's not exactly what we're talking about, but but we've got some selection bias. Hunter Harris: But I I totally agree, right? But but then also, I mean, to this point here too, it's really interesting because there's so many people in so many organizations that are

moving so fast, they are also siloing themselves, which is kind of paradoxically making this like I don't want to say making it, but some people are interpreting directions from on high to Treat engineering more as like back to a solo thing where it's like I'm gonna commandeer my legion of agents to cobble together some hot slop and throw it off on the internet or whatever. And for everybody listening, no shade on slop, like ship it and let the market decide. yeah, I I feel like the days of me judging slop I have to put behind me a little bit because I'm pretty sure. that I've consumed some stuff on the internet that is like quote unquote slot and I can't tell the difference. If I can't tell the difference, is it really slot? David Rasch: exactly. And and I think part of the role with the engineering hat on is some knowledge of whether it is or isn't, as the producer. Hunter Harris: Yeah. David Rasch: So back to the hey, I spent 12 hours working on it. You were investing to make sure part of it wasn't slop. Now you might have piled some slop on top of that, right? But you're applying wisdom to think about, hey, what parts of this are critical and lo load bearing, as Claude loves to tell us, versus Hunter Harris: Mm-hmm. God. David Rasch: what what parts of it are not. Hunter Harris: Yeah, I mean, I I will admit to tolerance of a certain amount of slop in there in certain places, yeah. But there there definitely are places where it's like, okay, cool, that'll be a foot gun, that'll be a critical bug, that'll be a foot gun. And it is important to know which is which, because at the end of the day, I mean and this is kind of like a paradox of speed that I think is not a new thing. I don't really believe there's anything in the way of new from AI, but If I want to move really, really fast strategically, that generally means moving slow on the day-to-day, right? And that doesn't mean that I necessarily like drag my feet, but God, what what's that quote that I love saying all the time? It's I think it's hours of planning can save no. it's hours of coding can save months of plan Why can't I say it? Months of coding can save hours of planning. That's it. David Rasch: Never heard that one. I like it. Hunter Harris: Yeah, I mean people live that David Rasch: I we've all felt that, right? Even even arguing with the AI to fix something, right? Where you sort of Hunter Harris: Exactly. David Rasch: know I've got to go I've got to backtrack here, but you get into it and, you know, you lose three hours and you're like, how do how come we're not we haven't fixed this yet? Hunter Harris: Yeah, that was I think yesterday, sunk cost fallacy again, where it's like, Okay, cool, we just spent like twelve hours going full circle where like we were trying to fix a bug and we did this whole cycle of fixes here, and now we've reached a point where the cycle of fixes we've fully reintroduced the first bug and we're back where we started. And I mean David Rasch: Yep. I I did this with billing systems about a month ago where, you know, Hunter Harris: Yeah. David Rasch: under the fallacy that the data was there and it was sound, it turns out it was not. And therefore there's no re there's there's the reason it wouldn't reconcile. There's the reason I couldn't always get it to add up to the same Hunter Harris: And it's nice when you can cut through that. And you know, ideally we'd get tools that would like, you know, for something like that, you'd expect the agent to be like, hey, just so you know, I'm got what I need, but unfortunately David Rasch: Ha ha ha. Hunter Harris: it's a a bit over optimized to be like, You sh you genius, you got it. We're gonna work around this. David Rasch: Every time it tells me, you're absolutely right, I say, That's just proof you have no idea what you're doing. Hunter Harris: Yeah, right. Like I cringe inside every time it's like you're absolutely right. Or like and this is again more of a Opus five thing than a five point five thing, but when it's like that's an incisive comment that proves that you have judgment and taste and I'm like, No, stop it. You're not real. Like, don't praise me this way. It's weird. David Rasch: Yeah. It's insidious that way, how it it models a relationship. It gives you impressions. My wife yells at me when I I tell the robot it's stupid and should be quiet. Because I sometimes I'm hopeful that if I do that, it'll make it into the feedback loop for the product folks. Hunter Harris: Mm-hmm. David Rasch: And so I'm I'm being intentionally rude to the robot in hopes that I I can save myself time in the future. Hunter Harris: I honestly like I have my moments of doing that too, but I I believe pretty strongly that that probably biases the model to be worse, so I try not to. But there are times when the frustration boils over and you're like, You knitwit. Why did you do that? David Rasch: Ha ha ha. Hunter Harris: That is the dumbest thing that I've ever seen. I literally just told you not to do that and you did it. Don't and it's like, you're absolutely right, I'm so sorry, I'll never do that again. Then it does it again one promptly. It's like I I wish David Rasch: Thirty seconds later. Hunter Harris: I could fully prompt it to stop apologizing, because I just need it to be like fully mechanical, like boom, boom, boom, boom, boom, check, boom, boom, boom, boom, boom, didn't do blank, didn't do blank, didn't do blank. And I can be like, cool, info, info, info, David Rasch: Yeah. Hunter Harris: loop. So I guess though bringing it back a little bit, in terms of this sort of like bad behavior from AI and Also really like time wasting behavior from AI in terms of what you're doing with your organization, like what cort sort of guardrails are you putting in to kind of like try and steer the agent towards you know being good as opposed to bad? David Rasch: I think the most the best results we've had in that space are to put in tight feedback loop back pressure

stuff. So anything we can Hunter Harris: Thousand percent. David Rasch: do to make so that like every time somebody comments on a PR, if it's something that the agent could have checked, that we could have automated in any way, our goal is to prevent anyone from ever having to think about. Right, if this is field naming, if this is convention, if this is you didn't add tests, if this is whatever, we try to put back pressure on the agent. Now, sometimes the back pressure gives the agent data and it still has to make a decision. But it still, even in those cases, it has to explain it. It surfaces it up. It says, hey, look, I ran this thing and my coverage isn't high enough. Here's why I think that's okay. Or hey, I ran Hunter Harris: Mm-hmm. David Rasch: this and my my type checks are now valid. So anytime we run into a build that's failing or something like that, that becomes immediate back pressure to make sure that we don't have humans wasting our time reviewing. Did the test pass? Does it follow this convention? Right? Did we put the code in the right place? Did we have the right roll-up of different solutions? we even added one recently to we kept having front-end components that would in Next.js accidentally include A database model and drag in the whole database library into the client library. And so we added we added deterministic checks for that of hey, if that happens, it fails. And it's it's all these things where in the olden days that would have all been possible. But some of those, as you know, you have to go do like delicate surgery of like, hey, how does this library work? And how do I whereas these days that's an example where I don't care if it's slop at all. I just need it to work. I'm never gonna look at that code. All I'm gonna do Hunter Harris: Right. David Rasch: is enjoy the benefits of the fact that I no longer have to think about that problem ever again. I never have to go analyze why did my client-side library add a megabyte of payload size because I accidentally included a file that went directly Hunter Harris: Mm-hmm. David Rasch: to the database. I'd say those are the primary things. I I think the second thing would be in the This space of considering that code is more expendable. In the old days, Hunter Harris: Mm-hmm. David Rasch: we'd think really hard about giving feedback that would be a lot of work, would be we'd we'd feel back pressure saying, well, can you go rename this everywhere? this really should have been the dependency change should have been inverted. Instead of adding this over here, you should have started over here and pulled. Those types of indirection or injection type. Type things, you would have at least I would have paused and said, Well, is it worth it? Right? This is all done. Hunter Harris: Mm-hmm. David Rasch: It's tested. we're good, we we've got good automation in place, so we'd see if drift occurs. Whereas these days, those types of changes are trivial to ask the agent to sort of rework

the the order and make it go the other way, which allows you to make it so that the code is hopefully more robust for the future, right? Because Hunter Harris: Mm-hmm. David Rasch: Those are inherently scaffolding to make it so that the next agent that goes and looks at it will hopefully do the right or follow the pattern as opposed or accept that there is less friction this way or more friction this way and use that as part of its decision making. I'd say those are the primary two things we've really gotten a lot of advantage out of. Hunter Harris: Well you touch on something that I've been kind of I don't know if struggling with is quite the right term, but something I've been like uneasy about specifically. But when I view a lot of the review process right now, I see a lot of people doing you know, AI generated coding, and then there's a bot that'll go and do an AI generated review on it, and then you have the humans do AI generated review on The MR after that. And so then at each step there, if you're not really, really careful, you'll like throw fable at something, for example. And they're gonna be like, hey, you got these like 85 issues you gotta fix. And then your agent's like, David Rasch: Ha ha ha. Hunter Harris: You got it, brother, and we're gonna knock out those 85 issues. I I speak in partially from like recent trauma here, because I had a project essentially wrap itself into a fail-close state. By looping on review and just n not critically responding to all of the critical review feedback to a point where it was like, this thing is bolted down, perfectly secure. No bugs can occur, in part because you cannot do anything with this piece of software. Everything no-ops in a fully secure way. David Rasch: Doesn't work at all. Hunter Harris: So we you listen up, brother, you are protected. You are maximum security. I'm like, no, I built this to do stuff, please no. and you know, I haven't that was just like essentially a process bug that I accidentally invoked because hey, I you know, I try different things and live to regret some of my experiments. But like I I guess dragging it back to the core question here. I see a a lot of process spinning out into you like yak shading and bike shedding and whatnot, where there's a bunch of things where You just kind of do them because they're low-hanging fruit, but at the same time, that work does still have cost. It's cheaper, but it does have cost. And that that's something that's a great concern of mine, really, right? Like I see organizations where cause like it's very easy to just tactically do that. And then you zoom up and you're like, hey, for this organization, 85% of the work being output right now is yak shaving. Like we're we are aggressively shaving all the yaks down to the bone. And that's no way to run a software company, right? So how do you remain or what are your suggestions for remaining pragmatic when the cost of doing a lot of this stuff is really, really cheap and you just c can kind of be like, yeah, free adjacent until free adjacent adds up to not really free anymore. David Rasch: I think for me, having the human in the loop somewhere in there is valuable. That doesn't mean no AI, but like to me, Hunter Harris: Somewhere. David Rasch: it's really easy in that review process for AI to do the whole inspection analysis conclusion judgment part of that process. And that's not, to your point, that generates all sorts of activity, some of which is valuable, some of which is noise. I I re I've reviewed a review of a a PR a while ago, this was maybe a month and a half ago, where the AIs all got fixated on some race condition with this piece of code. Now the piece of code is run about once a month. And so Hunter Harris: Exactly. David Rasch: my judgment as a human was delete all that crap. I don't care. That complexity Hunter Harris: Yeah. David Rasch: is not worth it to me. I'm glad that you all spent all the tokens in the world thinking about the different ways we can handle item potency on this request, but I just don't care. I would rather have Hunter Harris: On this thing that happens twelve times a year. David Rasch: right, I'd rather have two of those things in the incident where that happens than or at least the solution if I if I were to prevent this, the solution I would have chosen would have been far more simple as opposed to this solution that you've you've come up with. You know, unique tokens and all anyway, all sorts of things. Hunter Harris: Over engineering on top of o over engineering to solve David Rasch: So I I think in that case, one of the things that I've found valuable is actually to have AI tee up some summary of some of the major changes. we've like some of the coding agency, or not the coding agency, the like linear now does this automatically. It'll organize your PR into sections of hey, here's what this part of the changes do, here's what this part of the changes do, here's what this part of the so it'll sort of group them and give you some summary. And that information disclosure can be really helpful because. Hunter Harris: Mm. David Rasch: To the point earlier in the discussion, if I want to look at, hey, where have we changed data seams or state management or decisions, security, major API changes, those are all things I can not only review more easily if I've got a better understanding of why this code matters and what order it goes in. But also, to your point, I can then triage, hey, based on all this activity churn from the agent swarm, You know, making changes on this PR. How many how much of this actually matters? And then the other thing that I see not enough of is I see a lot of the old school agile blocking of work. well, we can't release it until we do all these things, as opposed to spinning out Hunter Harris: So David Rasch: and prioritizing and saying, that's cool, let's spin a ticket on that. And we can decide later whether that's important or not. Hunter Harris: Mm-hmm. David Rasch: I don't want to do the work right this second to figure out if that's a big deal. Whether it's gonna happen, whether we should do it, whether I care. And what you often find as you go back and review some of those tickets when you're looking at priority work, you say, Yeah, I don't really care that you cancel a bunch of them, which is beautiful because that's Hunter Harris: Well, yeah David Rasch: work saved as opposed to work done, you know, in the sort of you ain't gonna need it category for, hey, I'm gonna refactor this, I'm gonna move it around, I'm gonna change this thing. So the things I often end up looking at in a PR. Are I ask, I ask the agent questions like, hey, what what code looks similar across? Like, what are those architectural seams, changes in APIs, changes in schema, changes in decision making. I I look at screenshots, right? Like what are UI changes? Because those, just like the schema Hunter Harris: Mm-hmm. David Rasch: at the bottom, the UI changes at the top give me a great idea Hunter Harris: Absolutely. David Rasch: of what's all the goo in the middle. And that helps cut to me cut through some of the sort of performative agent churn. That just comes out of I analyze these 76 different problems, and there's this issue that this one thing might be null in this case, and next thing I know, I'm reading 400 words from whatever agent I'm working on, and it's all for a problem I really don't care about. Hunter Harris: Absolutely. Well, David, I wanna be respectful of your time. You wanna plug your pluggables? David Rasch: Yeah, so the main thing I'm doing is is working on this restaurant tech stuff. Other than that, you know, I I always love working with and coaching young and upcoming leaders. So if you're ever looking for somebody to bounce ideas off of or share some books and talk about it, please hit me up on LinkedIn. LinkedIn slash dot com slash D Rosh. Appreciate it. Hunter Harris: Yeah. And I'll put a link to your LinkedIn in the show notes for this as well. David Rasch: Appreciate it. Hunter Harris: and I mean I I think anyone who's watched this can really tell how easy it is to have a conversation with you about some of these technology topics, right? David Rasch: Yeah, it's a lot of fun and I I really appreciate, you know, you setting this up and spending this time with me. It's been a blast. Hunter Harris: Likewise, I mean I mean, as you can tell, I love just talking about this stuff, talking about different processes, procedures, all the sharp edges, all of the exciting use cases, all this stuff. But for everybody watching, thank you very much for joining us. I hope that this has been informative for you. Hope you've learned stuff. I hope that y you'll take some of these techniques back and you know, I AI a little harder, AI a little faster, and AI a little more successfully. and You know, until we have the next episode, I hope you all have a fantastic day. Peace.