Full Transcript for Control Before Thrust: Making AI Work with Josh Colter

I was reading a little bit about the history and something kind of popped out to me that seems relevant for today. So the the equivalent in the early 1900s at the turn of the century for AI was the the race for who would Who would get into the air. and you had really two competing schools of thought. And so the the the kind of leader at the time was this guy named Langley. He was the head of the I believe Air not Air and Space. Langley was he was the the director of the Smithsonian. He was well funded. he was backed by Teddy Roosevelt. in the presidential cabinet of the time, Teddy Roosevelt wasn't president yet. But anyway, Langley's bet was flight would be like a chauffeur. So it'd be like imagine jumping into your car, telling it where you want it to go and it takes you there. And so that was very Hunter Harris: Mm-hmm. Josh Colter: much how they were approaching flight at the time. And that was in contrast to the Wright brothers, who, of course, as you know, are were bicycle shop operators. And so their view fell into this camp of what what was called at the time airmen. And so they believed that flying would require some level of skill. You would need to know how to operate it. And and kind of the big breakthrough that The Wright brothers are known for was their control mechanisms, like the warping wing that allowed them to not just get up in the air but also land a plane safely. the the reason Hunter Harris: Yeah. Josh Colter: all that was interesting to me is that it felt very much akin to what AI is is doing right now. So it feels like there's there's two schools of thought, which is one is AI is very dangerous, but also a chauffeur. Hunter Harris: Mm. Josh Colter: It's gonna take you, you know, We'll we'll tell you where we're gonna take you. you put in the prompts. but largely it is a a mechanism in the flight analogy that is going to to automatically take you someplace versus an airman, which I buy into, which is, you know, we need to learn how to operate this stuff. We need to learn understand how it works, but then also build tools around it and be able to operate it safely ourselves. And So yeah, I just I I think that's something that I see playing out in the both media narrative and in my own use of AI. Hunter Harris: Absolutely. In terms of like making the metaphor work, the the usage of like I I think the terminology you used was control gear that allowed you to steer specifically feels like a very apt comparison here where in one, you know, you just kind of sit back and let the vibes take you, and the other, you know, you have systems and apparat apparatuses, apparate things for controlling the system and And safely operating the system. Josh Colter: Right. Hunter Harris: So digging into these two a little bit, I guess let's start a little bit with the chauffeur side. Like, I mean, we we have kind of like high level shown what a chauffeur might look like. But like if if I wanted to be a really good chauffeur, what does that look

like at the moment? Like if I wanted to go hog wild with that sort of paradox. Okay. Josh Colter: That's interesting. Just going off the cuff, I guess the idea of a chauffeur being a thing that takes you where you want to go. I guess it reminds me of my early use of AI, where I would ask Claude to build something for me or open AI. I was kind of bouncing back and forth between whatever whatever model was newest of the month. But yeah, Hunter Harris: As one does Josh Colter: yeah. you know, when I first when I first acquired cardboard and I was trying to refactor. I probably spent four or five months where it felt really productive, felt like I was prompting Hunter Harris: Mm-hmm. Josh Colter: the models, the models were coming back, they were kind of making it seem like there was progress. But then when I looked at my my actual Git history and what was deployed, there just there was there really wasn't much. There wasn't much that was super valuable to my user base that was going out. Hunter Harris: Mm-hmm. Josh Colter: And so to to me that's that's a bit of like the chauffeur mentality that I have felt in the past. And I think I'm seeing more people really struggle with because y again, it feels very productive to prompt the models, to feel like, you know, you know, Claude is telling you, that's a really great point, Hunter. I'm I'm gonna update. Yeah, nice job. Hunter Harris: Yeah. Very incisive. Josh Colter: Yes, yes. you know, but then not really I ultimately was not making the progress I wanted to make with improvements to the product. So I think that's I think that's one ap aspect to being a chauffeur. I also think that, you know, sometimes like it's okay to have a chauffeur model. you know, if I'm booking a flight somewhere, I don't necessarily want to build my own infrastructure to do that. Like I just want to be able Hunter Harris: Mm-hmm. Josh Colter: to say, figure this out for me. Right. I think there are some things that it's okay. you know, I have a car, but I also use Uber. You know, sometimes I just want somebody to drive me somewhere. And so I I don't necessarily think that one school of thought is necessarily completely right or wrong, but I do think the primary narrative is chauffeur. And I think the people that I find the most interesting online are the people who are building it as airmen. You know, they're trying to figure out or air women. you know, they are figuring out how Hunter Harris: Mm-hmm. Air people. Josh Colter: air people, thank you. Yeah, how do I how do I build something interesting but understand also what I'm building? It's not just slop, it's not vibe coded, but they don't understand the infrastructure, but rather they're able to build something that's very sound, control it, pilot it, land it safely, and and then share that with the world afterwards. Hunter Harris: Absolutely. I mean, when you were talking about that, I thought of Jeff Patton's what is it output, outcomes, and results, right? it sounds a lot like the chauffeur model lends itself very, very much to fantastic output, but not necessarily fantastic outcomes, right? it's a lot more focused on well, kind of wha what's that terminology? Like sound. Thunder or something like that, you know, sizzle as opposed to stake. so I guess Josh Colter: Yeah. Well, I I I think to go along with that, maybe I might I might add that the chauffeur mentality of the day was very much focused on building faster engines that would get up to get an aircraft off the ground. So there's a big point of because if you think about the three points of flight, you need thrust, lift, and control. And so Hunter Harris: Right. Josh Colter: the chauffeur camp was, you know. using thrust. How do we how do we launch this thing more aggressively? and then, you know, trust that the lift with the wings will work. And then we'll figure out control last. And kind of what the what the Wright brothers did in an interesting way for the airmen race was they thought about control first. And so as I look at the progress of of AI, what I see in the model releases is is better thrust, better throughput. And so I benefit Hunter Harris: Largely Josh Colter: from that aggressively as a solo builder. But then I I think I see a lot of people that just throw more robots at a problem and that hoping that that compute will eventually figure it out. And I think there's a more interesting, you know, the Wright brothers when they launched their plane, they ultimately constructed a very light engine that effectively did the job because they'd figured out the control and the lift aspects. And so I think when you when you understand the the file structures and the directions and and behind what you're trying to get AI to do, then you don't need as much throughput to get outsized results on the back end. So I think it's I think it's an important difference between the two and also a big difference in you see in how people are approaching AI and and the build challenge. Hunter Harris: I think that's another interesting kind of like parallel, metaphorically speaking too, where I mean, talking about how organizations focus on speed is always very timely for the US software market, right? and the concept that if you just get enough speed you can compensate for everything else. Josh Colter: Ha ha ha ha. Hunter Harris: Who cares if there is a brick wall in your way? If you can pick up enough speed, maybe you'll just punch through it. But brick walls tend to remain being made of brick. And Josh Colter: Yeah. Yeah. Hunter Harris: a lot of organizations end up coming to rapid and unceremonious stops at the same time. Josh Colter: so my my dad still works in IT. He's kind of like Hunter Harris: Mm-hmm. Josh Colter: an old school dude. He's like an old school Fortran, kind of working on a big infrastructure migration. But we we were on a trip together recently, and he's like, Yeah, my Hunter Harris: Mm-hmm. Mm-hmm. Josh Colter: boss came in and was like, We've

got to figure out this whole AI thing. And keep in mind, this is a multi-billion dollar company. Hunter Harris: Mm-hmm. Josh Colter: Lots of resources, you know, hundred-year-old firm. And i i it's so it was funny that's like we've got to figure out AI. Now AI is important, I think, of course, because that's what I've baked, you know, banked my career on. but my dad's like, I don't I'm not quite sure. Like, I'm dealing with AS/400 stuff, right? Like I this is not in Git, this is really not even in GitHub yet. Like, how am I going to use AI? on this code base. And so we spent some time just like, okay, let's start from where you at. Map out your process. Hunter Harris: Mm-hmm. Josh Colter: Show me where where does code exist or processes or things like that. Let's just like get everything visible, you know, make the work visible. Hunter Harris: Mm-hmm. Josh Colter: And then from there, let's figure out where we could just stort sort of like leverage AI to ask questions or to improve process. Like we're not gonna, we're not gonna just like go from where we're at where he was at today to fully AI native, you know, the the robots are doing everything. so he he kind of we got done with this, let's call it an hour working session. He went back into the office that that next week and he's like, all right, I need this, this, and this. Like here's how I I I need to be able to like you know, I need to be able to get some some way for AI to be exposed to the code base to ask it questions. So here are the permissions and controls that I need in place. and so I I I ran into him a couple of weeks later and I was like, Dad, how'd it go? He says, Well, I'm I'm like the AI expert now at work, which is funny. Hunter Harris: Mm-hmm. Josh Colter: I mean, he would admit it's funny because he's he's still learning as we all are, but you know, just just by virtue of like slowing down within his firm, Mapping out the current process, asking the questions that a lot of times people are maybe a little afraid to ask because they don't want to appear Hunter Harris: Mm-hmm. Josh Colter: stupid. And just putting some of those like basic building blocks in place, he had carved out a nice little niche for himself. And and so it's funny. He was like, Man, Josh, I wish I had another 20 years left in my career. Cause this is like this is getting fun. And he's he's ready to Hunter Harris: It's really fine. Josh Colter: retire any day. But I'm like, Yeah, dad, like this is this is pretty cool stuff for sure. Hunter Harris: Yeah, I mean like that's kind of part of why we sign up for this sort of stuff, right? Like it's Josh Colter: Absolutely. Hunter Harris: it's god, it's such a cool time to be a builder right now. so I I guess I want to dig into that a little bit there. Like, so in terms of that like one-hour working session, like how are you collaborating? Because I think it's so important that you mentioned specifically about getting everything visible. Because you know, AI is really, really powerful. And I think I had a brief like 15-minute podcast episode about this a number of episodes back, but like how we've used AI has really changed, right? There's still people that are stuck in that sort of like prompt age. Of like, I think you said AS400, right? Like just go to ChatGPT and be like, hey ChatGPT, I want an AS400 hard. How do I AS400 hard? And just kind of pray a little bit versus like what you're laying out is obviously importantly like the right way to do it, which is you have to surface things so that you can surface those things to AI, because if AI doesn't have that surface to it, it doesn't necessarily have that information in its training, it doesn't necessarily have access to that. Additionally, even if it does have related information in its training, it doesn't necessarily have like it has like the general picture, not like your specific picture. So so much of this is, you know, taking advantage of the biases and everything to get it to understand. Understand is a strong word. Get it to be biased by what you've created and what you have and what your processes are and what your workflows are. Because there's nothing worse than having something be like, hey, you need to do this 12-step process. Get deep into that and find out like, hey, this thing is a perfect process for a completely different company that doesn't exist here. Josh Colter: Mm-hmm. Mm-hmm. Yeah. I guess the cool thing about AI is that it reads every word you write now. Right. So I think Hunter Harris: Mm-hmm. Josh Colter: you and I were joking around in the prep call about what we're we've been in our careers long enough to remember the day where we would spec stuff out and then hand it off to an engineering team and then the engineer would not read anything in the spec. or they would skim Hunter Harris: Right. Josh Colter: it and then they would come ask a question about the thing that's in the document. And so you just learn. Hunter Harris: And you're right. Josh Colter: Yeah, this document I I've always argued documentation is helpful for thinking things through, but I think there's a difference between Hunter Harris: Mm-hmm. Josh Colter: writing to kind of process something and learn myself versus writing to influence or instruct someone else. And those two different modes. The the nice thing about working with robots now is that they read every single word that I write. I think the downside though is that some of the tools that I had adopted in my career, like story mapping or other mapping related Hunter Harris: Mm-hmm. Josh Colter: tools, were aimed at helping me collaborate with other human beings. And Hunter Harris: Right. Josh Colter: so there's this saying within Agile that's like a story is a is really a ticket to a conversation. Right. That's Hunter Harris: Mm-hmm. Josh Colter: what that's what that card was about. The problem I have is that you know, you and I, Hunter and I and Josh, can be working on a problem together and

have a conversation, and then we can look at a card, which is a summary, and go, yeah, I remember Hunter and I talked about that thing and we're on the same page about it. But the robot, who's now our third collaborator, was not in that meeting. But it reads a transcript Hunter Harris: Mm. Exactly. Josh Colter: of every single word that was in that meeting. And so I think that's where this kind of interesting mash up exists right now for me personally, which Hunter Harris: Mm-hmm. Josh Colter: is like I've taken spec driven development and I've combined that with story mapping. So by that I mean Hunter Harris: Mm-hmm. Josh Colter: I've got story map that allows me as a human or as I collaborate with other humans, I can kind of step back and see the big picture at a glance and Hunter Harris: Mm-hmm. Josh Colter: understand. And you know, as you know, story mapping is written from the standpoint of what is the behavior change that I want to introduce. Hunter Harris: Right. Josh Colter: And then what are the steps that I need the consumer of my software to take, whether that's another human or an external system? there's there's a consumer of the code that I'm building. And so what are the what's kind of the progression that I that I want to take them through to get to an outcome? So it kind of Hunter Harris: Mm-hmm. Josh Colter: forces me to put myself in the user's shoes to frame it as such and to make sure I'm delivering value. But then, you know, the problem I was running into is if I just give that to a robot. There's like a thousand different ways a robot could take to get to that outcome. And Hunter Harris: Right. Josh Colter: so it could technically build something in a way that isn't aligned with how I want to see it, but achieves an end goal, and it still looks to me as you know a prompter of AI, like a failure. And so Hunter Harris: Right. Josh Colter: what I had to do then was reconstruct. Within a map, a card is is storing markdown, which is the native language of Hunter Harris: Mm-hmm. Josh Colter: AI. And so it's it's handling those five hundred or sometimes even multi-thousand word specs, depending on what that card calls for. And so a robot can read what is native to the robot, but then I can see what is most helpful for me as a human. And I I think it helps me avoid a trap that I'm seeing a lot with business analysts or product managers that are doing a lot of AI work, which is, you know, once you start leaning into specs, you start writing these five thousand word specs that look great, but like I as a human being can't read Hunter Harris: Mm-hmm. Josh Colter: as quickly as a machine can consume those specs. And so it can feel like everything's buttoned down until we get to the build process and then something pops up that I missed because I couldn't see it in Hunter Harris: Right. Here be dragons effectively because no one's actually popped the hood and read it. Josh Colter: Yeah. Yeah. Yeah. Hunter Harris: yeah, I mean that is a

very interesting conflux overall. And I I really like the idea of using like visual tools here to I mean like map out for your comprehension but also be able to express that depth into like the details of what the AI is doing because at least for my own workflow, I mean I don't storyboard like you do exactly, because at least for me I am very spec driven, but I am very spec driven. Like I Josh Colter: Mm-hmm. Hunter Harris: I was talking to one of my clients today about how I was using AI and you know with the new models releasing and whatnot. I really want to try out Opus 5.5. So I sat down Josh Colter: Yeah. Yeah. Hunter Harris: and I worked with 5.5, I think for about 12 hours on Sunday, just working on one single spec because I needed to just iron out all the details, right? Like there's just something really important about making sure that you have that level of fidelity there because. There are decisions you can compromise on, but anytime you compromise on those decisions, it or anytime you compromise on the wrong decisions, and discovering which decisions are the important ones and which ones are not is tough. But anytime Josh Colter: Mm-hmm. Hunter Harris: you compromise on the wrong decisions, you end up paying for it for the most part. So I guess go ahead. Josh Colter: Yeah. Yeah. Yeah. I just I did a similar, I guess one anecdote if you wanted to riff off of that. Hunter Harris: Sure. Josh Colter: so I just launched dark mode for cardboard this week, Hunter Harris: Mm-hmm. Josh Colter: which is cool to me because part of the reason I got there is I was looking through what the most requested features for Miro. Miro is kind Hunter Harris: Mm-hmm. Josh Colter: of the largest alternative to cardboard in the space. They are dominant, you know, and they also just got acquired or in the process of getting acquired by Bending Spoons. And so you know, for for six years users have been requesting dark mode in Miro. And Hunter Harris: Mm-hmm. Josh Colter: and so it's interesting because it's like, man, I use a lot of tools that have dark mode. What's what's so tough? Hunter Harris: Talents therefore. Josh Colter: What's so tough about this, right? But like FigJam doesn't have it, Miro doesn't have it. And it turns out that there's this like hidden trade-off that you're talking about within whiteboards that makes it really hard, which is most tools that do dark mode. There's a theme for light mode. There's a theme for dark mode. You flip a switch Hunter Harris: Right. Josh Colter: and you get whatever is shown to you. With a whiteboard, there's this third element, which is like you and I are collaborating on a board. And so you might set A card color as yet this yellow means this thing for your organization. And so what I can't do is have one, two different experiences, because you and I need to be able to look at a board and understand that this color means these things. So that kind of rules Hunter Harris: Mm-hmm. Josh Colter: out the the flip

switch binary light mode, dark mode. And then secondly, once you've once you've set that background or that card color. those are those are there's certain variables around that that also need to adapt to that lighter or dark mode, like Hunter Harris: Mm-hmm. Josh Colter: the lines on a board or you know, different different extra elements. And so turned out to be like a pretty pretty like interesting build on my part, but The way I did it is that I first of all tasked Grok and Claude with doing an analysis of Figma, Miro, all the other players in the space. What Hunter Harris: Uh-huh. Josh Colter: do they have? What's worked? What hasn't? So it's interesting because FigJam doesn't have it, but Figma as a tool does have dark mode. And they've been extensively. Hunter Harris: I've never used Dark Mode in checkbook. Josh Colter: Yeah, they've written extensively about it. And so I was able to kind of leverage a compare and contrast lessons learned from their build. How did they do it? I think they, if I remember correctly, they had like 40 engineers dedicate a week to it. but they outlined a pretty smooth Hunter Harris: Yeah, that sounds about right. Josh Colter: a pretty smooth process of like here is how they went about introducing that to their tool. So I I created a map and cardboard, it's a mind map. And then I used because cardboard speaks native AI through Markdown, I pointed the robots at that map and I said, okay, now I want you Claude Code to analyze my current code base and I want you to look at what we need to build. And then I want you to build a story map for me. And so it generated Hunter Harris: Mm-hmm. Josh Colter: 20 or so cards on a story map. And so I started kind of shuffling those around and ultimately chunk the work into three different releases. Hunter Harris: Mm-hmm. Josh Colter: And then I I you know, kind of I did I do a lot of the Ryan Singer Shape Up type work. So I'll open up a card, Hunter Harris: Mm-hmm. Josh Colter: type a comment, ask a question, and then I'll point an LLM at the card and have it respond to the question. So there's Hunter Harris: Love it. Josh Colter: there's a chat history there. so I think I I ended up with like 29 or so cards when all was said and done. And this was Sunday night. I mean, I I literally decided Sunday afternoon, I should work on this thing. two hours later, I had a completely fleshed out story map and had release one built that night. I think I had two releases go out Sunday night. And then Monday I just powered through, like did a marathon day. And by the end of Monday, I had it stood up for our team. So I was able to kind of start dog fooding it. And as you know with with releases like dark mode, the devil's in the details. There's a lot of like edge cases Hunter Harris: Course. Josh Colter: of what does this look like in this setting? And and so you know I I spent a good chunk of like Tuesday morning just figuring out where those edge cases were. Again, using cardboard Hunter Harris: Mm-hmm. Josh Colter: to queue up all the work in the comments on the cards, tracking where work was at on the card. So it becomes like my control surface where I'm able to orchestrate Hunter Harris: Right. Josh Colter: everything. And then I released it. And even like today, I have a whole I have separate release map and that includes I was playing around with Claude Opus five point five before this, having it create videos of the functionality. So it was spinning up local Hunter Harris: yeah, love that. Josh Colter: Yeah, spinning up local environments of like light mode versus dark mode side by side. What does this look like? Help me write the LinkedIn post, help me write the Twitter thread, help me identify Hunter Harris: Mm-hmm. Josh Colter: twenty different threads where people have complained about dark mode on these whiteboard tools so that I know where to go post. three days and it's like we're we're done. Hunter Harris: I love it. Cause I I worked on a similar project years ago where basically doing something else doing something similar, not exactly the same, but doing something similar took a whole team of engineers something like six months to do. I mean some Some decisions made that more difficult than it needed to be. And to to be clear, you should not spend six months on adding dark mode to your thing. your Josh Colter: Yeah. Crap. Yeah. Hunter Harris: your timeline's a lot better. But I mean I I love the emphasis that you're putting on dog fooding and essentially like owner QA here, because I feel like, you know, we we've talked quite a bit now about like how you go about specing and how you go about making sure you're building the right thing, but man, do I like behavior driven development and making sure that you rigorously test what you're releasing and making sure that it is actually what you think it is is so important still. And r realistically AI helps you a ton with that. Like very specifically, to your example with the videos, right? Like I love having like a whole UI flow and then pointing clawed or pointing Codex at it and being like, hey, go step through these 15 web pages, take a screenshot of each, send me the 15 web pages, and Josh Colter: Mm-hmm. Hunter Harris: like I'll come, I'll go get a cup of coffee, I'll come back, and I'll have my 15 web pages, and I'll be like, great, great, great, great, great font screwed up. That thing's off in the corner in the wrong way, this thing's wrong, great, great, great, great, great. Repeat, repeat, repeat, repeat. and you know, like ideally you one-shot it, but it you never really one-shot it, right? Because there's all this like surprise stuff in there. so we're getting close to time. Do you want to plug your pluggables? Josh Colter: Pluggables. Yeah. Well, I would I would love, you know, especially since there's so many builders that are are listening and following you. You know, anyone who wants to go jam on the product. I've got a free account for anybody who signs up through the website. free first three boards are free. So try it out. You can plug those boards into your own MCP. So if you're a Claude or a Codex user or whatever your LLM of choice is. connect it to a board, have it generate some cards based on what you're working on. And then like like I said, I'm a solo builder, so I'm the one responding to the support tickets or questions. So would love to would love to connect with any folks that are interested in this space and have valuable feedback to give. Hunter Harris: I dig it. Love that man. Like solo voters are where the future's at right now. Well, Josh, thank you very much for joining us. I've really appreciated what you've talked about here. I feel like there's a ton of gold here for people to adopt and learn and build better. And for everyone watching, thank you very much for joining us. love this conversation. I hope you've learned too. And I hope y'all you I hope y'all have a fantastic day. Peace. Josh Colter: See.