文字起こし 🔗
Isaac Askew: Welcome to Never Rewrite. I'm Isaac Askew.
Jeffrey Sherman: And I'm Jeffrey Sherman, and today we're gonna focus on how Isaac's workflow
Isaac Askew: Today we're gonna focus on how Isaac
Jeffrey Sherman: and feelings about have changed in response to AI. since I was like two of the last three episodes is how I my workflows have changed. Now let's turn the tables and probe Isaac. So Isaac, do you feel more productive with AI? Well, first, I guess are you using AI? I know the answer is yes, but and do you feel more productive?
Isaac Askew: Episodes is how my workflows have changed. Well, first, I guess, are you using AI? I know the answer to the question. Yes. I am always using AI. Yeah, so you know, part of this this this topic today spawned from recent this I was asked the same question by somebody essentially giving me a questionnaire at my company. and they were basically saying, you know, do I feel more productive or less productive? Or you know, even? And if so, how what percentage more productive? And
Jeffrey Sherman: Yeah.
Isaac Askew: yeah, I don't know w what we're gonna do with these answers, but I told ten percent more productive. And I I know that seems like kind of a low number. and I said also it kind of depends. Like if I'm going to stop my work for the day and I have it doing tasks while I'm away from the computer, that's a hundred percent more productive as you know, as a whole person doing stuff.
Jeffrey Sherman: Infinitely. Yeah.
Isaac Askew: Right. and then in cases where it gives me some trash data to work with, which is not often, but I had this really rough week, the week before last, where I was essentially fighting Claude because it was convinced that what I was telling it was false and I'm like, No, I'm telling you, it's true. and I sp spent a lot of burnt tokens arguing with this thing. and it took, you know, I feel like It took three days to get something done that I could have done in a day, which is like the first time I'd experienced the inverse,
Jeffrey Sherman: Hmm.
Isaac Askew: which you usually it in every other case AI is always faster than me, but this particular case I don't know if their standards were confusing or the discussions that it pulled from Slack to help build context confused it too, because there is a back and forth on where to go, but yeah.
Jeffrey Sherman: Can you can can you expand 'cause I'm curious about where it went off track. It was building the wrong thing or it was building something badly? Or it was trying to debug something? What what was the lar well
Isaac Askew: All of those things. Yeah. So what it w what what the goal was for this particular thing is we when we integrate with some partners, we give them a code snippet to put on their page. And there's also this iframe thing that we have to consider because they might have an iframe in their form. and if you know we we basically are not able to inject data or pool data across iframe unless we do very creative things.
Jeffrey Sherman: Mm-hmm.
Isaac Askew: when we're trying to pass information. but fortunately, the people that we're integrating with are our partners and they're willing to update the iframes themselves and where they're they're hosted, to support any requests that we have that would be any kind of security issue, essentially that would normally bork any of our iframe
Jeffrey Sherman: Right, they're paying you for your services, so you're not like an ad where you're paying them to drop something on their website.
Isaac Askew: Right. So the integration is a bit more friendly.
Jeffrey Sherman: Yeah.
Isaac Askew: but we are working directly with them. We have you know we have access essentially to their scripts too, because we could tell them what to p what to edit on their end. So we had this new flow that we were going to integrate and I said, Okay, cool, my colleague has built this thing. I want you to build four other integrations that look just like that integration. Go. and then it kept telling itself, I can't do that because this thing's in an iframe that we don't control. You know, we don't host this thing, blah, blah, blah. And then I went back to it later. I'm like, actually, no, just you know, pretend like we can. And it was like confused by that. Like, no, we would we can't do this. This will be a security issue. This would not work. and back and forth I I had to keep convincing it. And like at the very end, it gave me this weird integration. And I'm like, this integration's like your own custom thing that you went off on. I told you to build it just like this other one. I mean, just like the other one, just replace the names, the colors. You know, we're using almost the exact same thing. And yeah, you gotta inspect the web page and see different anchors that you like the HTML of this page and see where to to integrate. But it's just just it's just so similar. Like please stop going off on your own and doing your own thing. And it would not listen until I finally
Jeffrey Sherman: Hmm.
Isaac Askew: said, actually, I own that domain, just like I lied to it. I own that domain that you're confused about, and I can change it anytime I want. The partner domain it was confused about. And then it was like, okay. And then finally even after having all the context of previous integrations where we did the exact same thing, there's like ten other examples of us doing that with other domains that we don't own. There's Slack conversations and confluence docs that we've pulled in that gave it context on how we're doing integrations and explains it. And it just it took it so and where I was using Opus. You know, I'm like, come on, man, like this is it's not fable, but I don't need fable for this kind of work.
Jeffrey Sherman: Right.
Isaac Askew: and I was just so baffled by how confused it got and how much hand holding it needed, 'cause I had never run into that b like that bad of a response before. And those are the days where I would say, Okay, I'm
Load full transcript
I'm trending towards negative productivity 'cause of all the rework. Right. But that's not as often. Yeah.
Jeffrey Sherman: Yeah, I haven't I haven't encountered Claude or Devin because I most of by weight
Isaac Askew: Most of five
Jeffrey Sherman: most of my time has been on Devon. I have not encountered it getting stuck like that. I I've encounter minor stuff where Claude this week is like to tell me that it d it ca it doesn't have the permissions to rebase something. I'm like I'm like, What you've rebased twenty things today. What now you tell me you can't do it? It's like, Yep. Yep, can't do it. I'm like, Okay,
Isaac Askew: Yeah. Like what you rebased twenty things today, but I'll run into that too. Yep, can't it.
Jeffrey Sherman: try It's like yeah now I can do it.
Isaac Askew: Those are weird. Yeah, I don't I don't know what the deal is with that or how to get it to stop doing that, but I run into that constantly where it's I can't run this command, but you can. I'm like you totally ran it like thirty minutes ago. Please do it again. Stop waiting for my permission. weird. Yeah. The the the inconsistency of the quality of the models is a very strange thing and the expense of them. I had a weird issue lately where my normal request suddenly I'm timing out at the five hour timeout window a lot sooner. And I'm not doing any crazy work or any act I'm not changing a whole lot of my own behaviors. It just seems like maybe even Opus itself is using more of the sub sub agent verification spinoffs that Fable I know does because I can see how many it spins off to verify. It's essentially saying,
Jeffrey Sherman: Okay.
Isaac Askew: I'm gonna do this thing before I get back to the person, let me spin off a bunch of things to criticize my own work, which is in essence replicating us coming back to it and telling it, check your work. They s they essentially built in their own protocol for that. but I think it's burning way more
Jeffrey Sherman: Which is good, it's super useful.
Isaac Askew: it is, and there's things I don't have to tell it to do anymore, but I think it's spinning off a little too many and burning too many tokens now and w without me really telling it to do that. I feel like somebody upgraded the Opus model and how it behaves or something. You know. But again, because of the inconsistencies of using some these models, I I I jokingly referred to to my to my friend who was interview interviewing me on this as it depends on the mood the AI is in today. Which again makes it more human than that. So
Jeffrey Sherman: Yeah.
Isaac Askew: you know, some days your you know, your colleague is just not there. Some days it needs five hours to come back to do to talk to you, I guess. The the moods the moods of these models certainly changes and affects affects productivity.
Jeffrey Sherman: Yeah, the anthropomorphization is amazing. I've
Isaac Askew: Mm-hmm.
Jeffrey Sherman: I have a friend who's a mathematician, and he does the the he structures it he
used to structure the payout models on slot machines. Right? So making sure that this thing
Isaac Askew: Mm-hmm.
Jeffrey Sherman: like the casino has one knobs like I want this thing to pay out at ninety-seven, ninety-five, whatever. And
Isaac Askew: Mm-hmm.
Jeffrey Sherman: modern slot machines have like thirty different payouts 'cause they have like the regular line things and they have special things and blah blah blah but and so he you know he his job was to make sure that the all the math would work out. And he would hear like and he so he knows the math and it's a just a pure statistical thing. Hit button, it spits out a number and then the machine does the thing. Plays the game.
Isaac Askew: all the math would work out. And he would hear like and he so he knows the map and it's just a pure statistical thing, hit button, it spits out a number and then the machine does the thing. Plays the game. And he would hear all these stories about, well if the machine has been doing this, then it must go and do that 'cause it's what's in this mood and none of that is in there. It's just the anthropomorphization of like
Jeffrey Sherman: And he would hear all these stories about, well if the machine has been doing this, then it must go and do that, because it's you it's in this mood. None of that is in there. It's just the anthropomorphization of like, the machine is in this mood. It's like now the the machine has no mood.
Isaac Askew: I think in this case yes. Because he was probably and this is pre AI, pre non deterministic routes, you know? so I think I mean I certainly feel a quality difference on some days versus other days. and and it's not just a me thing, like I mean in the communities that I've read on on Reddit, like whenever a new model comes out and there's like the the old one is not the favored one anymore. Or they release like the the tier for, you know, high effort versus ultra code, you know, there's like a huge difference but people can feel it. It's not just a unless it's just a collective delusion. You know, it could be that too, I suppose.
Jeffrey Sherman: It it could be. That that is the thing humans have is collective delusion.
Isaac Askew: So that was my answer though to to to that is like I feel ten percent more productive on average. but then it you know it depends. I feel like over I feel like last month I felt like fifty percent more. 'cause I mean now I'm doing a little bit more rework fighting the recent issue I just mentioned and then like the costs h make me hit a window faster so I can't use them for a bit. Right.
Jeffrey Sherman: Hmm.
Isaac Askew: So I'm like interesting, I need to change how I'm doing this or lower the model tier and maybe change my workflow. But I feel like my new programming that I do is actually learning how to program the AI that I'm using, learning the best ways to use the most efficiently without confusing them and to verify that his work is correct before I pass it off to proper QA channels.
Jeffrey Sherman: Yes, I feel that there very much is a meta programming aspect to programming now.
Isaac Askew: Mm-hmm.
Jeffrey Sherman: and if you are of a certain age, which I am, the you know, in two thousands it was very common you would have PHP generating JavaScript to
Isaac Askew: Maybe two thousands, it was very common you would have PHP generated JavaScript.
Jeffrey Sherman: generate HTML
Isaac Askew: Yeah. Mm-hmm.
Jeffrey Sherman: And like you would it was like, okay, mm, now I've got this piece here and I'm gonna generate this JavaScript thing and it's gonna generate the HTML and book boop book boop. Now it's all React. but back in my
Isaac Askew: I'm gonna generate this JavaScript thing and generate the HTML boop boop boop boop. Now it's all are. Yeah. But back is
Jeffrey Sherman: day old timer terrible things. and I I feel that way very much about the AI. It's like, okay, well, the fundamentals haven't changed. But now instead of writing the code, I'm telling write I'm using the tool to write the code.
Isaac Askew: And I I feel much about the air to okay well the fundamentals haven't changed. Mm-hmm. But now instead of writing the code I'm telling right have to get I still
Jeffrey Sherman: And I still have to get I still have to get the tool to use the fundamentals.
Isaac Askew: have to get the tool to use the fundamentals. Yeah, it's it's it it's very much back to you know, anthropomorphic concepts here, it's very much like now I'm managing a person and depending on their style Or their level of if I I mean like if I click and drag the effort down to like lowest effort, that's effectively making somebody dumber, right? Not not validating their work, not doing enough research, using the low least amount of brain power essentially. And so I need to treat that person differently, or that thing, you know, like the the model differently, because I'm not it it's gonna miss some things, or I need to understand who to ask for what. If I'm doing this big thing that requires the most expensive thing, I need to understand when I should pick that or I should go to you know before my principal engineer to ask about a systems question versus, can you change this color blue to something else? Which I don't want to make the principal engineer use that his salary to two. I can, you know, get someone who's still learning or, you know, junior to do that. So it definitely changes how I'm viewing it. And I hate that I'm using the human terms here 'cause it's not a human, but I I have to I'm effectively acting as a manager and understanding who I'm what I'm working with intelligence wise to best dictate what stories I want to give it. yeah.
Jeffrey Sherman: Well the the chat prompt makes it more like that, especially in this post COVID world where I would probably talk to most mostly I would s type chat with my fellow programmers anyway. And so I'm it's a chat and then I look
Isaac Askew: Yeah. And so and it a chat
Jeffrey Sherman: at code. So it very
much feels like
Isaac Askew: That's a good point. Yeah.
Jeffrey Sherman: similar to how I d it you okay. So your workflow you seem somewhat ten percent you think more productive on average. And do you think you're shipping fewer boats?
Isaac Askew: I do. I think interestingly, and I as like a kind of an ironic effect of this is because I can't trust what I'm talking to because of their mood, I actually double check more than I normally would. Because with my code, I I went through it and I'm pretty sure I got it right and you know the code and the flows look right. With it, I'm like, I don't know it's a you know, a coin toss. I don't know if you're delivered crap today. And so I I do a lot more quality control locally, where I I think we talked about in the prior podcast where I like a lot more end-to-end testing. So I spin up all my services, each one of them locally, make them talk to each other from a local environment. So have a really we have a really good local dev here. So all of the services can be spun up locally. and then I walk through it and have it like, you know, I'll take screenshots. I'll make sure that if it was like a database update that I can go check that row and that the thing I had it add is now a persisted column or data or whatever. and as a way to prove it. Because my main thing is I don't want to hand off something For a code review, that's trash, right? That's like you know, in the pre-AI days, that's just kind of a dick move, right? You you never even test this locally. This can't have worked. You know. And then we actually have a a proper many of
Jeffrey Sherman: I've I've had many of those, yeah.
Isaac Askew: those. Yeah. Right. So then when you want to pass it off to a proper QA, we have a QA team here too, who would test it in the staging environment, right? So my because I know I can't really trust this thing a hundred percent. Whereas I might actually trust a human more because I'm you know, different people take have different quality, you know, the some people take the roles more seriously than others and I know
Jeffrey Sherman: Mm-hmm.
Isaac Askew: who who I can trust more than others when it comes to I'm going away for two weeks, you're responsible for my stuff, you know. so yeah, it it makes me I I think I think the quality's been better because I you know, I take my own screenshots and add to the story, I'm like I can prove it the flow worked here and I'll just double check my work 'cause I'm like worried that it's done something crazy.
Jeffrey Sherman: Well you also have more time 'cause you're not writing code.
Isaac Askew: True. Yep. And while other stories are being completed I can be QA in the first one and kinda back to back.
Jeffrey Sherman: How do feel
Isaac Askew: Go ahead?
Jeffrey Sherman: I was gonna say how do you feel about that? 'Cause I know at my job, a lot of the programmers are feeling like it's not as fulfilling. Like they're like I am more productive and it's going better, but I don't enjoy the work as much.
Isaac Askew: A lot of the programmers are feeling like it's not as fulfilling. I am more productive and it's going better, but I don't enjoy the work as we This is tricky and I had I had I think a controversial answer when I was asked it from I was asked the similar question. Like my my do I enjoy my job now? You know, because it's changed so much. Am I just prompting more? Do I enjoy it? And for me it's yes. it doesn't take away the creativity for me. because I think it's it depends on the kind of work that you were doing. When I originally got hired at my current company, they wanted refactoring done, which can be done a lot you know, faster by AI. But a lot of things they wanted too, but I didn't realize they they did want it, but once I started building them I realized they did. was internal tools to make processes faster or to make self serve portals for other people who don't have to pull attention away from developers with questions. They can just figure it out or solve it themselves. So I ended up shadowing a lot of people early on to see how they were doing and their jobs and what roles could be automated. And that's completely separate from free feature development in your SaaS product. it shouldn't be, but the idea is like right now everyone gets concerned about tech debt or building new features or the stability of the system. You know, where do we go from here? How do we scale it? but part of the tech debt can be a bit of maybe nodge debt or something from from operations teams that are like constantly asking questions or needing things solved. Like
Jeffrey Sherman: Mm.
Isaac Askew: there's people asking questions in channels and I actually use AI to summarize how often do we get engineers pulled in to answer these questions and can we categorize them as root cause, you know, unique root causes. That way we can either fix them or build something for the operations seem to use to answer their own question and not pull away devel developer time.
Jeffrey Sherman: Right, operational debt.
Isaac Askew: And so I was kind of doing that pre-AI.
Jeffrey Sherman: operational debt instead of tech debt. It's where your processes are not
Isaac Askew: Yeah, I guess a better that's a better term for it. So for for me, yes, it's taken away like I was always back into focused to begin with, and to me there's not like a creative way to solve back in pro like many creative ways to solve back end problems. You can, like everyone can solve them differently, but there's one that's going to be the most efficient. Right? In school, they would say, you know, solve the problem and there might be ten different ways to solve it, but one of them might not solve at scale. And maybe you don't have to worry about scale because it's a thing that only ever has like thirty elements. And so somebody could solve it that in a way that took them longer but was like could handle load but didn't matter, you know?
Jeffrey Sherman: Right.
Isaac Askew:
There's all these different creative ways to actually solve it. But for now, like, okay, I can tell the AI, it can give me like probably the most efficient architecture or tell me why yeah I need to stick with my current one and do this instead, you know. With the front end it's a bit different. I could see that completely killing your flow. because it automates Some of the like it i i it gives a lot of slop on the the way UIUX performs. I feel like I feel like UI UX people are probably gonna be more coveted because it I think it does kind of a terrible job at understanding the flow and like why things are visually different. i in my recent experience especially where like it kept putting the button in stupid spots. So I'm like, that's doesn't even match Figma. What are you doing? You know, and there's some some there's some flow feels that are still very much human dependent. But I know a lot of front end engineers that were not really happy about it. They felt like it this kind of like, you know, automated away their joy. but again, my main thing what I love to build is internal tools. And what AI has given me is the backlog of three hundred internal tools that I wanted to make is now so much easier to deliver. And that's creative things for me. Like I I I I don't and someone could convince me, could throw them on the podcast next time. Someone could convince me that this has killed creativity for the most of people. But I think it just kind of again, it pivots creativity. I feel like if you can't find another creative thing to solve, that's on you. Because yeah, it automated the things out. I used to love refactoring. And now it can do it so much better than I can and catch so many things that I wouldn't call it. That's my favorite thing is to refactor.
Jeffrey Sherman: Yes.
Isaac Askew: So now I'm okay, cool, I'm just gonna bummer, but I'm gonna
Jeffrey Sherman: Yeah.
Isaac Askew: Build some internal tools. There's a lot of stuff here that we have that people just hate doing. It's a manual process. It's just so it's crap. Or a bunch of old things, old UIs that just look like they're geo cities, they never got updated. Now we can update them, we can see what's being clicked, what's being used, make them simpler for people. Like there's so many backlogged things that I can actually justify doing now because they take a day or half a day. so I'm still feeling super creative currently. But I g I I Get I get the roles for people who actually wanted to do the the refactoring or actually type the the the back end solution or whatever.
Jeffrey Sherman: Yeah, one analogy that goes in my head is like cargo ships these days are big container ships and they're ugly and they're built for how do we get the maximum number of containers on there and load it in easy and loaded out easy and th they're just big ugly ships. Like there's no beauty in a
Isaac Askew: Mm-hmm.
Jeffrey Sherman: in a cargo container ship. But it's really, really effective and that's what it's there for and
Isaac Askew: But it was really, really effective and that's what it's there for and that's probably where we are going and it's honestly where we should be going with software is with health containers. Yeah. But like the bespoke shift that we needed a
Jeffrey Sherman: that's probably where we are going and it's honestly where we should be going with software is let's have containers. Well, we have containers. But like the bespoke ship that where you needed a harbormaster and I can't remember the name that but there used to be a job. somebody's job was to look at a ship and figure out the most efficient way to load the ship and pack it as tightly as you could.
Isaac Askew: can't remember the name, but but there used be a job. somebody's was to look at a ship and figure efficient way to and pack it as tightly as you could. And that was a highly paid, respected job. Yeah, I could imagine. One extra
Jeffrey Sherman: And that was a highly paid, highly respected job and gone.
Isaac Askew: container could be a ton of product of money hand. Well well yeah.
Jeffrey Sherman: Well, they didn't have containers, they were just loading cargo. And cargo came in all shapes and sizes. But yeah, so it's like
Isaac Askew: Yeah. Understanding how to tetra sit in is like an optimized means one extra container or one extra bit of cargo, I mean.
Jeffrey Sherman: Right, everything you get in the ship is more money and so that was a and it's just gone. Like all the longshoremen are gone and that wasn't skilled labor, but also the the skilled labor of the person who could look at a ship and figure out the best way to pack it.
Isaac Askew: Yeah. And yeah, I get that. I can see that if that was like if you really enjoyed that job and there's something that can make it more efficient now. It's one of those things like I'm more back in focus, I learned to be full stack. The front end stuff is still harder for AI, but the back end stuff, I I don't necessarily mind it finding a a more optimal solution. If it it finds a query that it I I wrote that can it can make more efficient and it does it better than I can, I'll move towards other creative things to try to solve for people. I just like solving problems and so if I can solve different problems that's fine.
Jeffrey Sherman: I that's where I am too. the thing to point out though is it didn't make it more efficient. It changed the container ship isn't a more efficient
Isaac Askew: It didn't make it more efficient. It changed the containership isn't a more efficient
Jeffrey Sherman: in terms of actual packing cargo in. There's a lot of dead space in them where you you've got a container and if the container's not full then it's not full and you just move along. But it it changed the efficiency to how do we get the how do we load and unload the ship as fast as possible? And that became the efficiency piece. Whereas before it was how
Isaac Askew: Yeah. Whereas before it was how
Jeffrey Sherman: Do we pack the most stuff into the ship?
Isaac Askew: do we pack more stuff
Jeffrey Sherman: And so it changed what we were optimizing
Isaac Askew: Yeah, and so it changed the
Jeffrey Sherman: for.
Isaac Askew: All right.
Jeffrey Sherman: any other deep thoughts on how AI has changed your workflow?
Isaac Askew: Those are it for now, but I'm sure when I'm asked next week any other questions I'll I'll
Jeffrey Sherman: Yeah.
Isaac Askew: any any other kind of weird things that pop up I'll I'll summarize for for next episode.
Jeffrey Sherman: Have you felt the urge to do a rewrite and just see if you can one-shot a rewrite with with Fable or something?
Isaac Askew: No.
Jeffrey Sherman: Awesome. Thank you all for listening. I'm Jeffrey Sherman.
Isaac Askew: And I'm Isaac Askew, and this is Never Rewrite.