Fullstendig transkripsjon av Beyond Chat: Phoenix Tests, Ruby Agents & the AI Tipping Point

building the everything to suit them, right? 00:03:57.199 --> 00:03:58.400 Like Yeah. 00:03:58.879 --> 00:04:08.240 I'll never forget, uh I forget on the Ruby Rogues we had somebody where, you know, they had built their own, like, you know, their own like high performance throughput HTTP server. 00:04:08.400 --> 00:04:12.319 And I was like, well, why would why would you build this with all the other options out there? 00:04:12.400 --> 00:04:13.759 And it was just like, well, I could. 00:04:14.000 --> 00:04:14.240 Right. 00:04:14.479 --> 00:04:17.839 And it's like I know how it works, and it works great for my use case. 00:04:18.000 --> 00:04:22.240 And uh, you know, I feel like that's like almost the Rubyest way. 00:04:22.399 --> 00:04:33.759 It's like you know, like you get like these, like, you know, all the meta programming, and you can like create like kind of your own language out of Ruby, and that's like the benefit of Ruby, right? 00:04:33.839 --> 00:04:38.319 Is like if you want to make your own DSL, like honestly, it it's very easy. 00:04:39.040 --> 00:04:42.480 Make it look how you want it to read, and it's honestly that easy. 00:04:42.720 --> 00:04:44.240 Yeah, yeah, I totally agree. 00:04:44.319 --> 00:04:48.240 And I think that's so that is part of our journey in developing Phoenix. 00:04:48.560 --> 00:04:59.360 We started last year or sometime with Q3, and um, you know, my development team, they're not they're not all Rubyists, you know, they're they're kind of generalist engineers. 00:04:59.439 --> 00:05:07.759 Um I run a company that does a lot of services, uh software engineering services, teams gotta be able to hack in in multiple languages. 00:05:07.920 --> 00:05:12.480 And so, okay, so they're looking at the tool set and saying, yeah, it's mostly Python. 00:05:12.560 --> 00:05:21.519 Like if you wanna, you know, if you want to start to integrate with the APIs, like the LLM APIs and language at the time that we wanted to work with it. 00:05:21.680 --> 00:05:34.480 You needed to use Python, or the other option was like, okay, well, there are these ports from Python tools to to Ruby tools, which I think is you know a similar story to the the HTTP server, right? 00:05:34.560 --> 00:05:40.399 And it's like, okay, well, that's good, but I also know, you know, we've been in this Ruby community for a long time. 00:05:40.480 --> 00:05:42.560 Like some people do it just has to be Ruby. 00:05:42.720 --> 00:05:58.319 And so we want to know, like, okay, well, is it is it a tool that's that's actually like serving a purpose and you know and well maintained, or is it just a guy alone somewhere saying, you know what, I I refuse to code in Python, so I will rewrite everything in Ruby before I even do anything. 00:05:58.639 --> 00:06:06.240 And uh so, you know, so in the beginning we did a lot of Python, and uh today the application is much more Ruby and Ruby on Rails. 00:06:06.639 --> 00:06:16.639 And what we learned, you know, courtesy of Scott Werner, among uh you know a number of other people, is that okay, now there are starting to be some some tools that are built from scratch from Ruby. 00:06:16.720 --> 00:06:26.000 They're not built, they're not porting anything, they're doing something net new, in that case, the Gentec architecting and orchestration, and uh and they've got a life of their own. 00:06:26.079 --> 00:06:28.879 And that, I think, as you've mentioned, is kind of a tipping point. 00:06:28.959 --> 00:06:32.160 It's like, oh, okay, this is this is where we're headed next. 00:06:32.480 --> 00:06:32.959 Right. 00:06:33.279 --> 00:06:43.120 Yeah, and I mean I I feel like at the last uh Ruby Kayegi, Matt's even said it best, like, you know, the the language of Ruby is like almost built for LLMs, right? 00:06:43.199 --> 00:06:53.519 Like the way that it reads is like uh so much like a sentence that like you know, you could almost just like write a Ruby program so that it is sentences. 00:06:53.759 --> 00:06:53.920 Yeah. 00:06:54.319 --> 00:06:58.800 Or maybe get like some more, you know, more advantages out of there, like uh, you know, markdown to Ruby. 00:06:59.120 --> 00:07:00.480 I think that's entirely possible. 00:07:00.639 --> 00:07:01.040 Yeah. 00:07:01.360 --> 00:07:22.000 Somebody mentioned uh that they did they thought that Ruby wasn't kind of curious about this, that Ruby was not well suited for I guess well suited for the kind of uh code generation that happens today with LLMs and a chatbot, which I've got some opinions on we can get into. 00:07:22.319 --> 00:07:44.959 But like that style of development today is uh like Ruby not a good candidate for that because it's dynamically interpreted, it's you know it's strongly typed, and this person's contention was like, look, TypeScript, Python, and uh, you know, and then the Java.net kind of suite are going to be the languages uh that continue to prosper because LLMs can easily spit out code based on them. 00:07:45.199 --> 00:07:47.199 I'm curious what your thoughts are on that. 00:07:47.920 --> 00:07:49.439 I mean there are ways around that. 00:07:49.519 --> 00:07:51.680 I I honestly agree. 00:07:53.120 --> 00:08:01.439 Currently, I have this project I'm like kind of working on the side to like fine-tune an LLM specifically for Ruby code generation because of this reason. 00:08:01.680 --> 00:08:03.519 And you know, it's it's challenging. 00:08:03.600 --> 00:08:04.480 Like, I will say. 00:08:04.720 --> 00:08:08.959 And the the biggest the biggest issue is like these bigger models, they're good, right? 00:08:09.120 --> 00:08:17.680 Like they're honestly good enough at Ruby uh today that like if using like I don't know, cursor or even anthropic code is pretty good. 00:08:19.519 --> 00:08:26.560 They can generate like if if you want to just spin up a Rails app and work on features, like Claude Code is great, like you know. 00:08:27.839 --> 00:08:30.639 Yeah, like expensive, right? 00:08:31.040 --> 00:08:52.720 I

heard actually the the author of uh you know anthropic's uh Claud Code, uh they had him on Latent Space podcasts, and he was basically saying, you know, listen, like it may seem like you know, fifty dollars a day or a hundred dollars a day may be expensive, but like you have to think about like what it enables the developer to do, right? 00:08:52.799 --> 00:09:02.960 Like if developers are expensive on their own, and if you can enable them to get even like a fifty percent boost in productivity, like that's very valuable, right? 00:09:03.039 --> 00:09:09.440 Uh because their output is even more so than the these code generation aspects of things, right? 00:09:09.519 --> 00:09:18.480 So like it'll be interesting to see, like, you know, yes, it it's expensive, and so like the free models are honestly not great for Ruby code generation, right? 00:09:18.559 --> 00:09:29.440 And so like, yeah, if you wanted to run something on your laptop, right, like currently, yes, Ruby is not like ideal for code generation in the same ways, right? 00:09:29.759 --> 00:09:30.399 Yeah. 00:09:30.639 --> 00:09:33.919 But, you know, like it is the tipping point, right? 00:09:34.000 --> 00:09:38.159 Like, it's not like so bad that it's like preventative, right? 00:09:38.240 --> 00:09:44.480 Like you can still, even if you don't know Ruby, you can still use these free models, right, to learn Ruby. 00:09:44.639 --> 00:09:56.879 Like it'll still generate enough code, you know, enough quality too, where like it's not gonna send you down necessarily the wrong uh you know pathway, uh, although that is still an issue, right? 00:09:56.960 --> 00:10:04.159 Like I'm curious what your thoughts are there, like uh because I know like it's almost like I feel like it's like relearning how to use a computer, right? 00:10:04.240 --> 00:10:23.279 Like we're almost like relearning to like how to use a computer, how to program, like how to do all these things, because like there are ways of like kind of massaging the uh you know the chats or experience of building software with these new things that we're kind of still like readjusting to, right? 00:10:23.440 --> 00:10:40.879 And I know I've had I've had issues like you know, and maybe that I'm like a little bit out there because like I have a very specific I've read a lot of software books, so like my thinking process is around what I've read and what I've experienced and like you know all the things that I've made. 00:10:40.960 --> 00:10:43.840 And so I'm like biased, right, to like whatever it's generating. 00:10:43.919 --> 00:10:51.600 So like I have a harder time like assisting the path for the LLM to travel down rather than it just like taking its own path. 00:10:51.759 --> 00:11:00.480 So like there are like these kind of like two, I feel like buckets where like, well, do you want to just like produce something or do you want to produce something very specific? 00:11:00.639 --> 00:11:03.759 And I feel like right? 00:11:03.919 --> 00:11:09.600 If you want something very specific, it's like much harder to get that than just like making something happen, right? 00:11:09.840 --> 00:11:10.480 Yeah. 00:11:10.960 --> 00:11:26.159 I uh I I did uh I just did a talk a couple weeks ago in uh at an Earth conference, and it's uh and what I talked about was the cognitive burden placed on development teams because of generative AI. 00:11:26.480 --> 00:11:36.879 And uh and it's interesting that the um that they said on latent space, hey, you know, even if it's a 50% increase, that's really worth the the money. 00:11:37.039 --> 00:11:43.679 I don't disagree with that, but it's right in line with one of the stats that I pulled, which was a Microsoft study. 00:11:43.840 --> 00:11:48.320 Uh of course owns GitHub and can measure pull requests and all these different things. 00:11:48.399 --> 00:11:58.320 Um and they said that yeah, they're seeing a 55.8% increase in what they're calling productivity, which is really just code generation and pull requests and acceptance. 00:11:58.559 --> 00:12:02.799 Accenture did a separate study, which is interesting, and they were measuring uh code compilation. 00:12:02.960 --> 00:12:12.799 And so, okay, that leaves Ruby out, but for a lot of applications, it's like, okay, well, the amount of times that you're compiling code means that you are testing it more, you're releasing it for some definition of production. 00:12:13.039 --> 00:12:40.240 So, you know, what I was trying to try to hammer home in the talk was if you're if you are looking, like if every single day you simply have code in a wheat surf or a wheelserve clone or cursor, you have this amount of code, and you are reading it, messing with it, you're putting it into 50% more code than you've ever seen in your whole life, and then you make it every single day. 00:12:40.399 --> 00:12:44.639 And the decisions that come up from that are, I think, fascinating. 00:12:44.960 --> 00:12:50.159 Because just like you said, you've got a really specific idea of where you want the code to go. 00:12:50.320 --> 00:12:51.039 I do too. 00:12:51.200 --> 00:12:54.799 But that means that you and I are gonna do it a little bit slower than the next person. 00:12:54.879 --> 00:13:01.840 And I'm not sure, like maybe we're really confident, you seem like a confident guy, I think I am too, that hey, I'm doing it my way and it's gonna work. 00:13:02.159 --> 00:13:02.399 Right? 00:13:02.480 --> 00:13:04.799 But the next person next to me might be moving a little faster. 00:13:04.879 --> 00:13:08.159 And if I was a mid-level engineer, I might be like, well, I don't know. 00:13:08.320 --> 00:13:11.919 I mean, these people around me seem like they're going pretty quick, and I'm not going so quick. 00:13:12.080 --> 00:13:13.120 Maybe it's okay. 00:13:13.840 --> 00:13:21.600 You know, the same as doing what I expect, stand up locally, stand up and server working. 00:13:22.159 --> 00:13:23.440 Maybe it's okay to do this.

00:13:24.480 --> 00:13:32.320 Everybody has looked at a pull request they were supposed to review and immediately saw that it was AI generated. 00:13:32.480 --> 00:13:33.519 And every hand went up. 00:13:33.600 --> 00:13:40.159 And I'm I mean, me too, you know, and I work with some really smart people, but I can tell when I look at it, I'm like, oh, this was generated by AI. 00:13:40.320 --> 00:13:40.639 Right? 00:13:40.720 --> 00:13:44.639 Like some of this is you know straight from the code generated. 00:13:44.960 --> 00:13:47.840 And uh, and I I'm not saying a good or bad thing about that, right? 00:13:47.919 --> 00:13:51.440 Because if they're not a very developer, maybe you're happy that it's generated by AI. 00:13:51.840 --> 00:13:52.879 I'm curious there. 00:13:53.840 --> 00:13:59.279 Like, what what are your tells that it is code like generated from an LLM? 00:13:59.440 --> 00:14:04.320 Like, are there are there very specific examples, or is it like kind of just like a gut feeling? 00:14:04.720 --> 00:14:10.480 Well, if there's a bunch of comments in the code, you can tell right away. 00:14:10.960 --> 00:14:12.720 They're always trying to put in a bunch of comments. 00:14:12.799 --> 00:14:16.240 It doesn't I it always drove me nuts to begin with. 00:14:16.960 --> 00:14:21.840 Beyond that, I think what you end up seeing, I mean so it depends, right? 00:14:22.080 --> 00:14:27.200 Uh there can be really insane things if really the people don't know what they're doing. 00:14:27.360 --> 00:14:36.639 Like you see, like when when AI is sitting on top of a an application that already is not really making a lot of sense, it will do insane things. 00:14:36.799 --> 00:14:56.159 Like we saw once we're doing a consultant gig, we saw a commit come through that had a try-catch block where the catch was like rescuing from a certain error and then modifying the database record um to like to account for the fact that whatever it was looking for was not there. 00:14:56.320 --> 00:15:00.080 And so it's like a you know, like like generating a migration, right? 00:15:00.159 --> 00:15:02.960 Because uh, you know, it hit an exception. 00:15:03.039 --> 00:15:04.320 So there could be some crazy stuff. 00:15:04.480 --> 00:15:21.759 But generally speaking, I think, you know, there can be a there's a certain amount of verbosity, I think, and there's a certain amount of um you know editing in in sort of more files than maybe is necessary, and you start to see like, oh, I think that there's I think this is AI is at work here. 00:15:22.240 --> 00:15:22.639 Yeah. 00:15:22.799 --> 00:15:24.559 Yeah, it's funny you mentioned comments. 00:15:24.720 --> 00:15:33.279 It's kind of funny because like there's so many tokens wasted on comments, especially when it's like a descriptive comment for a very self-explanatory line. 00:15:33.440 --> 00:15:33.519 Yeah. 00:15:33.759 --> 00:15:34.879 Like just one, right? 00:15:34.960 --> 00:15:41.039 Like, oh, we're gonna update the blah blah blah table, and like it's like table like update. 00:15:41.360 --> 00:15:43.120 Yeah, it's out of date, yeah, right. 00:15:44.159 --> 00:15:45.600 Yeah, when that happens. 00:15:45.840 --> 00:15:50.879 And I'll call it out, you know, but um I'm always just like, you know, it's it's a weird thing. 00:15:50.960 --> 00:15:55.279 It's like, well, I'm gonna call this out and have this person change it, but I know they didn't really write it. 00:15:55.600 --> 00:16:02.559 Uh so it's not a comment about them, but it's kind of it's kind of more about their vigilance in accepting code and not in how they're writing it. 00:16:02.799 --> 00:16:04.080 Because I don't think they would write that. 00:16:05.039 --> 00:16:05.679 Right. 00:16:06.080 --> 00:16:07.360 Which is a kind of funny thing. 00:16:07.440 --> 00:16:14.799 And you know, so you made these different decisions that, as you've already said, you know, we're kind of still grappling with how that works on a day-to-day basis. 00:16:15.039 --> 00:16:21.519 Yeah, you know, it makes me think I'm thinking about your latest like Phoenix post on like reactive AI, right? 00:16:21.600 --> 00:16:34.080 Like where uh I feel like we're kind of at the cornerstone of like truly implementing uh you know AI actions as like a reactive component to various things in the day, right? 00:16:34.159 --> 00:16:35.919 And to the development process. 00:16:36.080 --> 00:16:42.399 Uh right now it's like very like you know, instructional where you're like directing it, doing trying to do something. 00:16:42.559 --> 00:16:47.120 And there are more and more things where it's like, oh, like GitHub actions that run on something, right? 00:16:47.360 --> 00:16:47.759 Yeah. 00:16:48.080 --> 00:16:53.360 But uh they're not like it's still kind of like, oh, it's gonna be a GitHub action and it'll create a pull request. 00:16:53.519 --> 00:16:58.000 And then you're like, you have to still have to, you know, get in front of it and like do something with that, right? 00:16:58.080 --> 00:17:01.600 And like, yeah, well, it's gonna produce way faster than you can review it, right? 00:17:01.679 --> 00:17:03.039 Like oh yeah, absolutely. 00:17:03.279 --> 00:17:09.359 In fact, that our our first customer, Phoenix's first customer was ex got exactly that. 00:17:09.519 --> 00:17:18.799 We're like, because we're thinking like everybody else was like, hey, you know, this thing can do a task, in this case, generate some tests and uh and open a pull request for their repository. 00:17:18.880 --> 00:17:22.000 Well, you're right, it can open a thousand pull requests, which nobody wants. 00:17:22.160 --> 00:17:29.759 But even if it doesn't, you know, it's kind of like this interloper that just gives you a thing that's like, oh hey, these tests work. 00:17:29.839 --> 00:17:33.359 It's like, well, yeah, but now I gotta stop what I'm doing and look at it. 00:17:33.440 --> 00:17:37.519 Uh and it's like having another member of the team that's like

PandaBots, it's a little slower. 00:17:37.759 --> 00:17:40.880 You know, and it so it's like it didn't fail. 00:17:41.200 --> 00:17:45.119 The customer liked it, but we could see by how fast PRs were getting accepted. 00:17:45.279 --> 00:17:46.319 This was not the way. 00:17:46.480 --> 00:17:53.920 And I've come kind of full circle around that, and I like, you know, your term for reactive actions is is exactly right. 00:17:54.480 --> 00:18:02.880 First generation AI, the cursors, clients, bolts the lovables of the world. 00:18:03.039 --> 00:18:04.079 These are these are great tools. 00:18:04.160 --> 00:18:05.279 I mean, we use some of these tools. 00:18:05.440 --> 00:18:07.279 We use bolt to create Phoenix. 00:18:08.640 --> 00:18:12.079 I'm not saying anything bad about them, but I'm saying this is first generation. 00:18:12.240 --> 00:18:22.880 You have a chat, you have a chat with a thing, you tell it what to do, you give it some instructions, it does it you know, anywhere between halfway good and 80% good, and then you have it go back again. 00:18:23.599 --> 00:18:28.400 I think that that is I think those are not actually very long for the world. 00:18:28.480 --> 00:18:29.440 So I could be wrong. 00:18:29.680 --> 00:18:32.559 But what uh what we aimed to do was take that out. 00:18:32.720 --> 00:18:35.519 Phoenix doesn't have a chat interface, it already has its commands. 00:18:35.680 --> 00:18:49.039 Its commands are to like lovingly support your code base with tests, with bug reports, uh, you know, with suggestions or highlighting of risks, um, and to do this in an automated fashion, right? 00:18:49.519 --> 00:18:56.960 You know, it's gonna spin up immediately, generate a full suite of tests, and that's gonna follow you as your as your software evolves. 00:18:57.200 --> 00:19:00.559 And there's nothing stopping us from doing that, right? 00:19:00.960 --> 00:19:12.319 What it does take is a serious amount of automation and sort of well-crafted, well-designed um integration between a bunch of different endpoints. 00:19:12.720 --> 00:19:14.319 So that completely different. 00:19:14.720 --> 00:19:20.799 That's funny you mention uh dependabot, because that that's kind of like how how I imagine all these things to start, right? 00:19:20.880 --> 00:19:24.480 Like it's just like annoying you with like a barrage of like updates. 00:19:24.559 --> 00:19:25.759 Hey, this thing is like out of date. 00:19:25.839 --> 00:19:27.599 And it's like, well, like merge it then. 00:19:27.680 --> 00:19:30.960 Like, I don't know, like I don't have a test though, so like don't, right? 00:19:31.039 --> 00:19:36.000 Like you end up like trying to configure it just right, and you spend all this time doing that. 00:19:36.079 --> 00:19:40.640 Uh, and then you finally get it right, and it's like you still end up with a pull request that hangs, yeah. 00:19:40.799 --> 00:19:44.160 Because you still gotta test it, and like that takes time manually, right? 00:19:44.240 --> 00:19:47.119 Like verifying that things work and like some side problems. 00:19:48.160 --> 00:19:52.799 Like, I like I just got the we use Camal to uh deploy uh part of our application. 00:19:52.880 --> 00:19:54.160 I think Camal is cool. 00:19:54.480 --> 00:19:59.119 I just got a dependabut update, it's like, oh hey, it's a minor minor point release bug. 00:19:59.279 --> 00:20:05.440 I'm like, oh because I you know the test is manual, I have to actually deploy it a bunch of times and make sure it works. 00:20:06.319 --> 00:20:10.799 And I of course it's good software, but I'm like, oh, this is like another thing in my day. 00:20:11.119 --> 00:20:15.519 I will say I I have a little side project called now reading.dev. 00:20:15.839 --> 00:20:18.720 Uh it keeps track of my reading list so I could just throw stuff at it. 00:20:18.880 --> 00:20:24.079 And like I use Camal 2, and it was I was like, oh, bundle update, you know, minor, right? 00:20:24.160 --> 00:20:25.279 Like that should be safe. 00:20:25.359 --> 00:20:32.559 And I go and try and Camal deploy, and it's like, oh, you know, Postgres fails because you can't like restart update now or something like that. 00:20:32.720 --> 00:20:39.039 Like it was like a small like configuration option that like no longer is there in a minor version update, right? 00:20:39.119 --> 00:20:42.640 And like, oh if I had this autopilot, like I just couldn't deploy then. 00:20:43.519 --> 00:20:44.319 Yeah, yeah, yeah. 00:20:44.559 --> 00:20:45.200 Like right. 00:20:45.440 --> 00:20:50.960 Yeah, we're all just like deploy is automated, like it was just it would just be alerted, right? 00:20:51.039 --> 00:20:52.240 Like, yeah, right. 00:20:52.400 --> 00:20:59.839 Yeah, we're all just living in this, like, we live in this incredible amount of trust, like interdependent trust with semantic versioning. 00:21:00.079 --> 00:21:02.400 Where we could just say, okay, the minor version is gonna work. 00:21:02.559 --> 00:21:06.240 Because for most people it is, but it's not gonna work for you, and uh, and then you're screwed. 00:21:06.799 --> 00:21:10.400 Yeah, and that all this computer use stuff, like that seems a little more promising. 00:21:10.559 --> 00:21:18.240 At least it could like spot check things for you and be like, hey, like I tried to deploy this thing, and I see like, oh, your homepage doesn't load, right? 00:21:18.319 --> 00:21:19.279 Like, yeah. 00:21:19.359 --> 00:21:25.680 Yeah, that's uh you know, that'll get that'll that might get handy, but like I feel like it's still not quite there. 00:21:25.839 --> 00:21:26.640 I don't know. 00:21:27.039 --> 00:21:28.000 No, I think you're right. 00:21:28.240 --> 00:21:29.920 But you know, I think that it's coming. 00:21:30.000 --> 00:21:37.039 It's uh there's we just we're stuck in this place where we just want it now, and like it's it's not there, but people use it like it's now, right? 00:21:37.119 --> 00:21:38.880 Like, yeah, it's true. 00:21:39.119 -->

00:21:53.599 But I'm uh I like I have to say I'm a little like I'm a little disappointed in the software community writ large because I I feel like most of us are not thinking beyond what like lay people are thinking about AI. 00:21:53.839 --> 00:21:59.839 It was what three years ago now that ChatGBT kind of like over the world and open AI, the first thing they said was like, hey. 00:22:00.240 --> 00:22:01.680 Hey, this is just like one use case. 00:22:01.759 --> 00:22:02.480 This is not the thing. 00:22:02.559 --> 00:22:04.000 The chatbot is not the thing. 00:22:04.160 --> 00:22:04.319 Right. 00:22:04.559 --> 00:22:12.319 But it's three years later, we're all still treating it like, oh, well, if you've got a chatbot who will code you to a landing page, we gotta give you like$25 million. 00:22:13.039 --> 00:22:17.359 Um and uh and I think that's that's fine for the permit, you know. 00:22:17.440 --> 00:22:20.480 I think for us, you know, we should hold ourselves to a little bit of a higher standard. 00:22:20.799 --> 00:22:22.960 Yeah, I I don't disagree with you there. 00:22:23.039 --> 00:22:27.599 I've been trying to escape the chatbot as a input. 00:22:28.240 --> 00:22:28.880 Yeah. 00:22:29.359 --> 00:22:31.680 As like the definitive user experience. 00:22:31.839 --> 00:22:43.920 Because it's so easy, to be honest, if you have a software product, like a chat makes complete sense if you were just like trying to service somebody that wants something solved from you, right? 00:22:44.000 --> 00:22:56.799 Like yeah, just throw them in a chat and like chat ops, which is fun, which is funny, like chat ops didn't work before, but now that AI has you know conglomerated the idea of it, it's like now back and like everybody's like, oh yeah, chat ops. 00:22:57.359 --> 00:22:59.920 It's like oh, like what didn't work about it before? 00:23:02.000 --> 00:23:04.319 And you know, we're like relearning that aspect of things. 00:23:04.480 --> 00:23:16.480 But uh yeah, I'm definitely like I'm liking this more of like reactive approach of seeing like okay, where can you just like throw an LLM at just like observing something and like start there and see like what it observes? 00:23:16.720 --> 00:23:21.359 And like from from past experience, like it's both good and bad, right? 00:23:21.440 --> 00:23:29.839 Like it's like a whole new learning experience where like uh you know if you if you're trying to like categorize what it's doing, like it's honestly not that great at that. 00:23:30.000 --> 00:23:42.319 Like it can't it can be good, but it'll like it'll just make stuff up, and like if you have very specific things you're trying to like have it like bucket, it'll just like you know, it'll get it like 80% of the time. 00:23:42.480 --> 00:23:48.480 And then like, well, what do you do with the other 20% of the buckets that are just like you know, what is that? 00:23:48.960 --> 00:23:51.599 Yeah, right, yeah, and then you're going back through it manually. 00:23:52.000 --> 00:23:52.240 Right. 00:23:52.319 --> 00:23:55.200 But I I feel like there's more and more like observational stuff. 00:23:55.279 --> 00:24:06.480 So I'm curious, like, uh where do you see kind of like the observations heading to the point where and like where's the point where we can actually start acting on those observations, right? 00:24:06.559 --> 00:24:12.160 Like, how does somebody even know, like, okay, this is a better way than chat bot, right? 00:24:12.559 --> 00:24:13.680 Oh, I see what you're saying. 00:24:13.839 --> 00:24:22.160 I mean, what I see is you know, just stepping back to the the business perspective, like you have to be able to demonstrate a lot of value to people. 00:24:22.559 --> 00:24:29.519 And you know, the way that generative AI through chat demonstrates value is it spits out a lot of code. 00:24:29.680 --> 00:24:33.920 And in the case of Bolt, see Bolt is this fascinating example because Bolt is not really Bolt. 00:24:34.160 --> 00:24:35.200 Bolt is Stack Blitz. 00:24:35.440 --> 00:24:53.680 Stack Blitz has been around for 10 years, and Eric Simon built Stack Blitz to develop a tool that is production grade development environments in the browser, and they're working on it for 10 years and doing okay, they're selling it to the enterprise, but certainly not blowing anybody, uh, blowing anybody's doors in. 00:24:53.839 --> 00:24:58.079 And uh, they got this idea to wrap it around a chat bot. 00:24:58.240 --> 00:25:11.200 And the result was magical, not because they did something to a science and LLM, but because they already had this infrastructure that could make up, and that is a really cool experience. 00:25:11.279 --> 00:25:13.599 So people see that and they're like, that's the value I want. 00:25:13.759 --> 00:25:19.440 And I think that we're all just human beings at the end of the day, and we want to have that experience of like that thing. 00:25:19.680 --> 00:25:22.880 All of a sudden I have something that I didn't have before, right? 00:25:23.119 --> 00:25:32.799 And um, you know, for us, it's funny because we're we're these totally test-driven engineers, we write tests for everything, but we know that first of all, that's not the world. 00:25:32.960 --> 00:25:35.599 And so I know all there are limitations to what we can do. 00:25:35.759 --> 00:25:37.599 We can't possibly think of every end case. 00:25:37.680 --> 00:25:42.480 We can't possibly like test everything because that's why bugs creep into our system and we get things wrong. 00:25:42.720 --> 00:25:48.720 So what we want to be able to deliver is like I already I feel I feel the safety, right? 00:25:48.880 --> 00:26:00.480 Like within a week or week, or maybe even sooner than that, not only do I have these tools but this or these tests, but this this software has alerted me to issues that would have been in production and are now not. 00:26:01.200 --> 00:26:06.880 You know, I think that's that's something where we can say, oh, okay,

okay, redesign belief I have the safety methods here. 00:26:07.680 --> 00:26:12.559 Yeah, I will say there's there's an incredible amount of value for what you've built at Phoenix. 00:26:12.640 --> 00:26:29.039 And I I say that because having personally tried to do TDD with all these varying code editing tools, um yeah, you know, like cur even cursor cloud code, like open AIs, they'll help, they'll get tests created, but it's like you start looking and it's like everything's mocked. 00:26:29.119 --> 00:26:30.960 Like, yeah, nothing's real, you know? 00:26:31.119 --> 00:26:34.400 And so, like, well, it's just testing mocks, like, yeah, I could do that. 00:26:34.480 --> 00:26:36.079 Like, I could get anything to pass, you know? 00:26:36.160 --> 00:26:37.519 Like, oh yeah, yeah, yeah. 00:26:37.680 --> 00:26:42.400 And so like it'll like hallucinate factories, it'll it'll do some really crazy stuff, yeah. 00:26:42.720 --> 00:26:49.039 Yeah, and you can have it and it'll it'll get on a loop and like you know, eventually fix itself, but like only to a certain extent. 00:26:49.200 --> 00:26:57.920 And uh sometimes it gets into a loop and you've just wasted hours of it trying to like recreate a database helper or like something like that, right? 00:26:58.000 --> 00:27:00.240 And like you're like, well, we already had one of those. 00:27:00.400 --> 00:27:00.880 Yeah, right. 00:27:01.279 --> 00:27:05.759 The context got too big and now like you've forgotten about like all the stuff that exists. 00:27:05.839 --> 00:27:22.640 Like well, that's you know, that's an interesting point with um you know, one of the reasons that that uh I think that the LLM stumble on uh on some of the tasks we give it is because the context wind, like they're they are they are loading on too much context. 00:27:23.440 --> 00:27:31.119 Friend of the show, Landon Gray, had a post about this not too long ago where he was like, if you're focused on the size of the context window, you're focused on the wrong thing. 00:27:31.279 --> 00:27:35.359 And I think he's absolutely right, because it is all about the specific context you give it. 00:27:35.519 --> 00:27:49.200 And it's still today the best use of AI when you're generating code, and I'm sure you've already experienced this, is to ask it to do very small, very specific things, or else it's going to do something it's gonna, or else it's gonna make a bunch of assumptions and do something you don't want. 00:27:49.359 --> 00:27:52.960 And so, you know, loading in the proper context is not easy. 00:27:53.200 --> 00:27:58.079 And even when I try to, I'm like, look at this file, then look at this file, and then do this, it doesn't do it. 00:27:58.160 --> 00:28:03.839 Um, you know, and that's using G41, it's using the latest and greatest, and it is better, but it's still not there. 00:28:04.480 --> 00:28:04.880 Right. 00:28:05.519 --> 00:28:18.960 Yeah, you know, I've been experimenting a little bit with sub-layer amongst other things, of trying to because like you make a good point of like, you know, the the context window being an issue, but also not being the right focus area, right? 00:28:19.039 --> 00:28:26.079 Like uh it's almost like we're trying to teach it how we want to do things and how we do things, and yeah. 00:28:26.480 --> 00:28:29.920 The source code isn't necessarily right the only source of that. 00:28:30.000 --> 00:28:34.319 It's also like our brains and like our utility and like how we do things right. 00:28:34.480 --> 00:28:45.839 And so like every person's unique and that's like an unfortunate thing, but it's also like uh you it all the LMs because they're not like you know the same thing every time, right? 00:28:46.000 --> 00:28:48.720 Like uh that's a positive for that, right? 00:28:49.039 --> 00:28:49.440 Yeah. 00:28:51.680 --> 00:28:56.160 Huge hole I see missing is kind of like that learnability aspect of things, right? 00:28:56.240 --> 00:29:02.720 Like something that you could just attach on to your experience that just like is watching you and your behavior with it, right? 00:29:02.799 --> 00:29:10.160 Like, and it can inject like you know, that behavior into the context as you're using it. 00:29:10.400 --> 00:29:12.640 Yeah, and like that's hard to do, right? 00:29:12.720 --> 00:29:17.119 Like, yeah, it's hard to even watch it, let alone persistent, right? 00:29:17.279 --> 00:29:17.519 Right. 00:29:17.759 --> 00:29:22.559 So that's uh and so I've been messing around with like some sub layer like actions, right? 00:29:22.640 --> 00:29:29.839 Because they they can watch files and and I feel like this is actually something I've built so many times is just like a log, right? 00:29:29.920 --> 00:29:33.200 Like what has been done, just like just keeping track of the log. 00:29:33.359 --> 00:29:34.400 I haven't actually used it, right? 00:29:34.480 --> 00:29:49.839 Like I don't know what to do with it now that I have it, but like, you know, I I keep building them and like eventually be like, you know, if I could just like find how to like introduce this into however I'm using it, like there's value there, but like I don't know if that's the right way to do it, right? 00:29:49.920 --> 00:29:58.240 Like, but like I feel like there's something there where it's like, okay, like your use of it is gonna like train it somehow at some point. 00:29:58.480 --> 00:29:58.720 Yeah. 00:29:58.960 --> 00:30:00.160 Do you think I'm off there? 00:30:00.400 --> 00:30:05.119 No, I don't I'm actually really interested to see this now because I think that there's I do think there's something there. 00:30:05.279 --> 00:30:07.680 And I think you know, we've had this. 00:30:07.839 --> 00:30:18.240 Yeah, I mean, I guess I haven't heard it this year, but uh you know, last year people were always talking about training LLMs, and uh yeah, to a large extent that was confusing. 00:30:19.279 --> 00:30:22.480 Maybe maybe we're just using like too dangerous of a word training, right?

00:30:22.559 --> 00:30:25.039 Like that's very specific to this world, right? 00:30:25.119 --> 00:30:35.440 Like it's almost like not like we're training necessarily, but like providing some you know backdrop to experience, right? 00:30:35.519 --> 00:30:49.680 Like AIs are becoming like we've made these chatbots, and it's like now, okay, well, they're all the same chatbots, and now people are trying to like augment the context to like change the experience, but really we're like at that experience, right? 00:30:49.839 --> 00:30:57.039 Like, and like how okay, how do we rework it to like change the experience to a more personal level? 00:30:57.200 --> 00:30:58.400 And like, yeah, I don't know. 00:30:58.480 --> 00:31:00.960 There's like that's it's really uh really interesting to me. 00:31:01.200 --> 00:31:09.680 Yeah, it is, and it's also you know what we've found is you know when we try to you you try to bolt things on to the LLM, right? 00:31:09.839 --> 00:31:13.279 Either additional context or additional instructions. 00:31:13.440 --> 00:31:18.000 And the thing is that these LLMs are changing really fast because that's where most of the money is going. 00:31:18.160 --> 00:31:34.720 And uh and you really we found that you really have to keep a lot of loose coupling, like you really have to structure your code in such a way that the things that you're building around, like especially the like the blast radius of your LLM, that stuff has to be able to just be you know deprecated completely or changed out really fast. 00:31:34.880 --> 00:31:40.400 We were on our second agentic agent orchestration library, and I'm pretty sure my team's gonna tell me. 00:31:40.559 --> 00:31:47.200 And it's just you know, these things outlive their usefulness really quickly, and uh so you gotta be able to move and change really quickly, too. 00:31:47.759 --> 00:31:51.440 Yeah, I feel like that's not unlike all software, right? 00:31:51.519 --> 00:31:56.400 Like, yeah, you know, a lot a lot of people even hit those limits with Rails, right? 00:31:56.480 --> 00:32:00.079 Like Rails is a great like kickoff utility like framework. 00:32:00.240 --> 00:32:06.079 Like you have something on the web that you want to accomplish, and like Rails is awesome for that, right? 00:32:06.160 --> 00:32:22.160 Like, and you can build an incredible business out of that, and like you know, at some point you may see yourself being like, well, you know, Rails has out, you know, our our like business processes have like outgrown what Rails had offered us originally, right? 00:32:22.240 --> 00:32:27.680 And so like people do like find something else, even within the Ruby space, too, right? 00:32:27.759 --> 00:32:31.359 Like just building something themselves that fits the business better. 00:32:31.440 --> 00:32:34.640 You know, Stripe is a famous like example of this, right? 00:32:34.720 --> 00:32:38.880 Where like they are probably one of the biggest Ruby shops that's not Rails, right? 00:32:38.960 --> 00:32:40.400 Like, yeah, right. 00:32:40.799 --> 00:32:51.680 And like I feel like that's a similar thing with all these like AI frameworks or you know, things that happen is like people are now like outgrowing them to the to the custom point, right? 00:32:51.839 --> 00:33:00.319 Where like we now have custom AI like software that we're like focused on how do we build that best, and like that's always gonna change, right? 00:33:00.400 --> 00:33:04.960 In the same way that like software, you know, frameworks and usage changes too. 00:33:05.200 --> 00:33:08.319 Curious if like that normalizes in the same way though, right? 00:33:08.400 --> 00:33:15.839 Like, we're at such like a a hockey stick, like in the graph of advancement, right? 00:33:15.920 --> 00:33:21.680 Like, seeing all these like coding agents and stuff like that that are like coming out, like it's difficult. 00:33:21.759 --> 00:33:32.160 How do you gauge like alright, this makes sense to pursue, and this doesn't, because we know like it, you know, it's probably not gonna be worth it, right? 00:33:32.240 --> 00:33:34.720 Like, it's gonna be gone like in a month, right? 00:33:35.519 --> 00:33:35.839 Right. 00:33:36.480 --> 00:33:49.359 I spent a little while on the sideline and my personal opinion is dug in start building start you know to sort of take advantage of the new tools as they come out. 00:33:50.799 --> 00:33:57.440 You don't know how you know the next big thing is, unless you already have worked out everything before. 00:33:58.720 --> 00:34:08.719 So that's been my take is to get our get our team just deepen the weeds of of AI, specifically how AI automation. 00:34:09.360 --> 00:34:19.679 I think like everything else you know, so it's repeatedly test kind of the use cases that are important to do with different all times, different prompt, whatever. 00:34:19.840 --> 00:34:23.039 Um, and you know, and be ready to shift and shift quickly. 00:34:23.199 --> 00:34:32.800 But I think one thing is that um you know you really have to build in a way that can take advantage of what's there without sort of obfuscating what's there. 00:34:32.880 --> 00:34:40.239 And that's I think another big advantage for the huge AI generation tools that are out there right now, because it's just a thin wrapper. 00:34:41.199 --> 00:34:48.159 So you kind of don't need to, and some companies are doing some really cool innovation, but you kind of can let the LLM lead the way. 00:34:48.400 --> 00:34:54.320 That's that's we're still kind of keeping that as a core concept because again, it's where billions of dollars is going. 00:34:54.400 --> 00:34:57.199 So it would be foolish to just say, well, we're gonna build separately from that. 00:34:57.280 --> 00:35:02.000 You know, we want to build with it, but make sure that we're always able to take advantage of those innovations. 00:35:02.880 --> 00:35:04.320 Yeah, I agree with you there. 00:35:04.480 --> 00:35:09.360 It's uh, you know, you

definitely have to chase it, which is kind of exciting to be in the industry, right? 00:35:09.440 --> 00:35:14.480 Like uh you gotta see what works for you and what doesn't because it yeah, you never know what could stick for you. 00:35:14.639 --> 00:35:22.159 And and so I'm curious, like I'd like to transition us into like you know, the Ruby space specifically for like all this tooling, right? 00:35:22.239 --> 00:35:30.079 Like because there's a lot of talk, like you know, Langchain has like their whole stack that just like is kind of like the Rails for AI. 00:35:30.559 --> 00:35:38.480 Has been for a while, like where you oh just go and like build stuff and it locks you into that Langchain experience and like people outgrow in the same way Rails do. 00:35:38.719 --> 00:35:39.360 Yeah, yeah. 00:35:39.920 --> 00:35:50.079 What is what are like some of the Rails equivalents that you see uh that are kind of like setting in for some of this, like hey, if you want to build with AI stuff, like this is a great starting point. 00:35:50.320 --> 00:35:51.440 Oh, that's a good question. 00:35:51.599 --> 00:35:56.800 Well, the easiest way to be wrong on a tech podcast is to tell you what the next Rails is gonna be. 00:35:57.039 --> 00:36:01.840 But uh There are no right answers on this show, are you kidding? 00:36:02.480 --> 00:36:03.920 Okay, yeah, yeah. 00:36:04.000 --> 00:36:09.679 I think I'd say that um, you know, I actually I I like Scott's Sub Layer a lot. 00:36:09.840 --> 00:36:12.800 I would love it if that was the next library that we moved to. 00:36:12.960 --> 00:36:19.840 I know it's a bigger shift because right now we've used Crew AI for about three or four months now, which is open source. 00:36:20.000 --> 00:36:25.840 I think they have um I think they have like a premium tier that is more full service, but we haven't used it. 00:36:26.000 --> 00:36:38.719 But you know, you start to see some of the cracks in these tools, and it's just because um I think, I mean, I I guess I'd like to think that most people aren't using them as an extensive use case as we have. 00:36:38.880 --> 00:36:48.719 Now, there are some and some that are far more complex, but I'd say that we're probably on the 80 to 90 percent, you know, more complex use cases when it came to uh Langchain. 00:36:48.880 --> 00:36:55.440 We still use Langchain to evaluate and like find out how much we're spending uh every day, which is alarming. 00:36:55.760 --> 00:37:02.800 But um but we don't use the orchestration anymore because we think it's um well, it's just it became very difficult to set up. 00:37:02.880 --> 00:37:06.960 It be it wasn't really fitting, uh again, like you said, it wasn't really fitting our use case. 00:37:07.199 --> 00:37:09.440 I would like to move move ourselves to Sublayer. 00:37:09.599 --> 00:37:14.480 Uh Scott said he would uh come and port it over for free, so when he does that, we'll move for sure. 00:37:14.639 --> 00:37:20.880 And uh I guess I'm not sure beyond that, like what the next big thing will be. 00:37:21.039 --> 00:37:21.519 What do you think? 00:37:21.760 --> 00:37:24.719 Yeah, I've definitely used Sublayer uh obviously heavily. 00:37:25.199 --> 00:37:32.960 I don't know if that's just because that's what first came out or not, but it does solve most of the use cases I've seen for building with AI in a very rubis way. 00:37:33.199 --> 00:37:38.159 Maybe if I was new to Ruby, there would be other things that might be better, uh, because I don't have to know as much Ruby. 00:37:38.320 --> 00:37:38.880 I don't know. 00:37:39.119 --> 00:37:39.360 Hmm. 00:37:39.440 --> 00:37:39.679 Yeah. 00:37:39.840 --> 00:37:43.119 I I don't know if that's the same use case for uh Langchain though, right? 00:37:43.280 --> 00:37:47.360 I feel like m many people maybe don't know Langchain either or Python. 00:37:47.679 --> 00:37:48.239 Yeah, right. 00:37:48.400 --> 00:37:49.440 You know, going into it. 00:37:49.599 --> 00:37:52.400 So um Yeah, it's hard to say. 00:37:52.480 --> 00:38:02.880 I I feel like, you know, Shopify just announced their roast framework, which looks pretty promising as like a similar command line workflow orchestration system. 00:38:03.679 --> 00:38:13.119 I feel like there's still like a lot of like people missing the boat on like benchmarking and evaluation of like what it's doing. 00:38:13.360 --> 00:38:14.480 And yeah. 00:38:16.480 --> 00:38:19.199 For that reason, I'm very hesitant with any of them. 00:38:20.559 --> 00:38:20.960 I see. 00:38:21.199 --> 00:38:23.280 You mean the tools themselves are not benchmarked? 00:38:23.840 --> 00:38:24.079 Right. 00:38:24.159 --> 00:38:35.199 The tools themselves, like I feel like if you want to build like a quality AI system, like evaluating what it's doing is like one of the highest priorities, right? 00:38:35.280 --> 00:38:48.960 Like if you're not testing it in a uh like an automated fashion, at least some way, and like scoring and judging it uh and having that feedback loop, it's very difficult to like get any like long-standing quality use out of it from a business. 00:38:49.440 --> 00:38:51.119 What's the score that you're looking for? 00:38:51.360 --> 00:38:53.519 I that's up to you to make, right? 00:38:53.599 --> 00:38:59.440 Like, but like having tooling in place to like make those up uh are kind of like really important. 00:38:59.519 --> 00:39:20.559 So like you know, there's lots of vendors that offer this service as like a sideways product, and then it always creates this issue of well, then you have all these tools that then have to connect to it, uh, right, and so like you're then uh dumping all of your logs at right these services, and then having to like massage that workflow, right? 00:39:20.639 --> 00:39:25.599 And like, so how do you then get the feedback mechanisms right from your app into those? 00:39:25.760 --> 00:39:34.480 And then it becomes this whole like massive orchestration where you're trying to like just work

with vendors all the time uh to solve something that should be built into the tools, to be honest. 00:39:34.639 --> 00:39:40.639 And I feel like it's often like missed as like a framework, it should almost be like the first choice, right? 00:39:40.719 --> 00:39:51.679 Like, here is the uh evaluation system or or metrics builder for a an AI framework, and then starting from there, then all of the features start getting built into that. 00:39:51.920 --> 00:39:58.800 And I feel like Sublayer provides all the tooling to make that happen, it's just not like automatically there, right? 00:39:58.880 --> 00:40:02.880 Like you kind of have to like make your own system out of it, which isn't bad. 00:40:02.960 --> 00:40:07.440 Uh it's just like more modular and more like you know, you have to do it. 00:40:07.599 --> 00:40:09.679 You have to hand it more of an investment, yeah. 00:40:09.840 --> 00:40:16.239 Right, which isn't, you know, uh for for Rubyus is actually nice because we like to build things ourselves and of our own, right? 00:40:16.320 --> 00:40:18.239 Like, so like it's I'm split, right? 00:40:18.320 --> 00:40:22.559 Like, because I would like to have these things just automatically work for me. 00:40:22.800 --> 00:40:23.039 Yeah. 00:40:23.280 --> 00:40:24.880 And like and defined, to be honest. 00:40:25.039 --> 00:40:26.800 Like Rails is very like opinionated. 00:40:26.880 --> 00:40:30.320 I would like an opinionated, like, this is how you do it. 00:40:30.639 --> 00:40:31.599 Yeah, I can see. 00:40:31.760 --> 00:40:32.320 Yeah, I can see. 00:40:32.800 --> 00:40:36.639 But having the flexibility is also important in an AI framework, right? 00:40:36.719 --> 00:40:41.920 So the active agents library by Justin Bone is like super promising too, right? 00:40:42.000 --> 00:40:55.039 Like it's built into the Rails ecosystem, so like you get all of the familiarity, and it's like plug and play, and like that the aspects feel like testability is something that you could just use Rails for, right? 00:40:55.119 --> 00:40:57.679 And be like, okay, I expected to do this thing. 00:40:57.840 --> 00:41:07.119 I feel like though, with a lot of these like agentic stuff that's coming out, it's like, well, how do you like then do the integration test, right? 00:41:07.280 --> 00:41:15.280 Like, there's so many variables in that workflow, like it's not like you're just like input to output, it's like right. 00:41:15.360 --> 00:41:24.880 Yeah, just I'm thinking about VCR trying to Yeah, right, like input and then side effects, and then right, like, well, how are you measuring those side effects, right? 00:41:24.960 --> 00:41:26.880 Like, and like that's very challenging. 00:41:27.039 --> 00:41:31.840 And I feel like even some of these uh valuation vendors like have a hard time capturing that, right? 00:41:31.920 --> 00:41:37.920 Like, how do you measure the side effects within your business processes, especially as you start to hook everything up to it? 00:41:38.239 --> 00:41:39.199 Yeah, I know. 00:41:39.519 --> 00:41:47.440 I think you know the active agents one is something that's exciting to me too, and we'll have to have them on the show to uh active agent folks to I would like to try it. 00:41:47.679 --> 00:41:57.199 It's um like the the Rails part of our application is still a little bit distant from the agents, but it's getting kind of closer and closer every day. 00:41:57.440 --> 00:42:00.800 So it could be an exciting way to you know kick the tires. 00:42:01.599 --> 00:42:08.400 Yeah, I would say next to those, like, you know, Obi Fernandez, he's killing it like the whole Ray framework is awesome. 00:42:09.039 --> 00:42:27.440 I've used that like just I really like the descriptive, it has its own DSL, so like I really like the descriptive nature of like the objects that you build with it as like a okay, this thing has a bunch of steps in it, even that you can hook up to each other, and like you declare the prompts very specifically within the class structure. 00:42:27.599 --> 00:42:32.559 Yeah, there's a lot of parts of it that I really that like really snap in play too. 00:42:32.880 --> 00:42:35.760 Well, so tell me a little bit a little bit about it, because I don't know the Ray. 00:42:36.159 --> 00:42:47.679 Yeah, so the the Ray Library R A I X, it's very declarative in like okay, you're gonna have, you know, let's say you're generating like an you're parsing an invoice, right? 00:42:47.760 --> 00:43:07.679 And you want to do something with that invoice, like you make like some kind of like you know, invoice, you know, prompt and declare it as like a class, and it's like actionable, and you can say, okay, like it's gonna do this prompt, and then it's gonna use the output from that to do this one, and so you can chain a bunch of stuff together, which is nice. 00:43:07.760 --> 00:43:19.840 Uh, and it has like a lot of the the niceties where you can like attach arguments to the prompt and declare that in and like declare your tools that it can use, and so you can abstract kind of like an agent aspect. 00:43:20.000 --> 00:43:23.519 I don't know, I don't think it's actually called an agent, but like Yeah. 00:43:23.679 --> 00:43:31.760 You can you can create these objects kind of like just regular Ruby objects that have these attributes that can be reactive based on what the input is. 00:43:32.000 --> 00:43:32.320 Okay. 00:43:32.800 --> 00:43:40.639 And it it's a really nice like utility to to work through, and I like the er the ergonomics of like it almost has these like then steps. 00:43:40.880 --> 00:43:44.559 So like after it has a response, then you can do something like that. 00:43:44.639 --> 00:43:46.800 Uh yeah, and the chain ability aspect, yeah. 00:43:46.880 --> 00:43:53.039 I really like it's like active record style, like where you where you can chain a bunch of methods off of the same thing and use them together. 00:43:53.280 --> 00:43:53.519 That's cool. 00:43:53.599 --> 00:43:54.480 I'm

gonna check that out. 00:43:54.719 --> 00:43:59.199 Yeah, but again, like I'm just experimenting, like I haven't built a product on it, so like I don't know. 00:43:59.360 --> 00:43:59.679 You know, a lot. 00:44:00.320 --> 00:44:03.840 term how that pans out, but you know he obi's building quite a lot. 00:44:03.920 --> 00:44:10.880 So like yeah I trust the framework survives, especially as Shopify starts to embrace it, uh which they have. 00:44:10.960 --> 00:44:12.719 So that's that's they have yeah. 00:44:12.880 --> 00:44:17.679 Yeah I had a uh had a talk with Obi and a couple of the other guys at Shopify not too long ago. 00:44:17.840 --> 00:44:41.679 We were just kind of comparing notes on you know how you know structuring our programs and structuring the applications and they just it's so funny because they're just like you know we've said well we have had different experiences with Claude versus at that time I think probably ChatGPT 4.0 and you know one of them at Shopify was like yeah we've got you know we're talking to Claude and and ChatGPT about like what they're doing for Ruby. 00:44:41.760 --> 00:44:44.719 I'm like that sounds nice when they're not taking my phone calls. 00:44:44.880 --> 00:44:54.800 Yeah but you know if you're the biggest Rails application in um in the world you know you can get them on the phone and uh maybe so I'm like yeah you know they're gonna advocate for us please that's awesome. 00:44:54.960 --> 00:44:58.480 Yeah I uh I'm excited to see what comes out of Andrew Kane this year. 00:44:58.559 --> 00:45:09.760 Yeah he he's just like keeps pumping stuff out for the Ruby world specific specifically the stuff that I follow for fine-tuning I keep waiting for the day where I I could just like uh switch over a bunch of stuff I'm doing. 00:45:10.000 --> 00:45:38.719 Yeah it's a it's I know just use Ruby uh right now I have these stupid like you know python scripts that I'm calling shelling out to basically which is not fun but you know work with what you got yeah exactly I'm curious like uh because you mentioned you know you were using Python for some things like is that still the case like how does that how is that transition working or integrating with with Ruby like what are the challenges you have left there? 00:45:40.079 --> 00:46:26.320 Well so basically this was a this was a Python app for about the first three months uh until I was kind of like they were deep in the weeds you know they're working on giving proper context on you know on getting the the proper automations in place um just just for the agent and right at that point it was just a CLI and and I think at some point yeah we realized hey you know we're gonna need a we're gonna need an app web app that shows what we're doing and so like over a weekend I was like alright we're you know Rails eight like let's go and you know it spun something up and the whole point was like hey this is just a show this just shows what the LLM does and so I it was Ruby but it

was you know it was a Ruby application but really it was just Ruby it was just a showcase point. 00:46:26.480 --> 00:46:39.280 And then it started to become like oh actually yeah we're moving away from anything like any kind of interaction on the command line like we want to we want to have this thing working end to end and continuously working. 00:46:39.440 --> 00:46:53.679 And so then that continuous part became well it's Rails to GitHub and pro possibly GitHub actions and there's some Docker in there and then you know there's the uh the CLI at the other end. 00:46:54.079 --> 00:46:54.559 Yeah. 00:46:54.880 --> 00:47:18.159 And this just and like when we saw that and uh we can get into this on the next episode like we did a lot of like we did a lot of things that probably a few years ago would have been considered big upfront design and today because AI just like spits out like ERDs and you know and like mermaid charts in like two seconds it's like now like oh we've got like really established stuff and it doesn't matter if it changes. 00:47:18.400 --> 00:47:27.920 So then like when we started we looked at this thing is it yeah mermaid's good oh yeah I mean because even making changes you just be like oh move this node over here like and it just does it like perfect. 00:47:28.320 --> 00:47:28.960 It's fascinating. 00:47:29.039 --> 00:47:31.760 I like the first time I saw it I was like what did you do? 00:47:32.000 --> 00:47:34.719 You spent like a week doing this and then you know it was like no it took like five minutes. 00:47:34.880 --> 00:48:00.559 I'm like all right awesome let's keep doing it then but yeah so like when we realized that it was like oh actually there's like all of these moving pieces and we didn't want to write a bunch of Python and manage that right so it became then all of a sudden that little you know UI application became like a focal point for the I'm trying to use a word other than orchestration the like the managing the integration points across everything. 00:48:00.719 --> 00:48:10.719 And uh and so now the CLI is still really important and that's written in Python but it's becoming a smaller and smaller part of the application not because we're making it smaller but because the rest of the application is growing. 00:48:10.960 --> 00:48:17.360 Yeah that's exciting to hear it makes me wonder because like crew AI has a heavy Rails for something. 00:48:17.440 --> 00:48:19.440 I will have to have them on to find out what that is. 00:48:19.679 --> 00:48:51.920 Yeah I'm I'm curious like how the the lines start to blend right because uh I feel like every company is like especially that got in early python had to be at the forefront right yeah yeah a lot of progress but now yeah now that it's stabilizing like Python's not great for web yeah I know it's not a lot of missing pieces you know even the Django like it's funny I'm friends with a lot of Django folks and they're like you know they try and look to Rails like oh how can we like integrate that it's like every day like how you know how do you do this in Rails like in Python but from Rails. 00:48:52.960 --> 00:48:59.039 Yeah I know I've I've experienced that too and um it do it does get really complicated. 00:48:59.119 --> 00:49:09.760 Yeah I would never start uh a web application like I've built web applications in yeah C sharp it's it's kind of annoying I've done it in Java it's it's fine. 00:49:09.920 --> 00:49:16.559 But yeah Ruby still feels like the best it's it's hands down and the web's not going anywhere I don't think anytime soon. 00:49:16.800 --> 00:49:19.039 So we're gonna keep building for the web. 00:49:19.360 --> 00:49:24.400 Yeah totally well I think we're we're hitting our limit here on time. 00:49:24.639 --> 00:49:26.800 Otherwise we're gonna keep talking all day long. 00:49:27.039 --> 00:49:29.760 I know to be honest this is good though. 00:49:29.920 --> 00:49:50.719 Yeah I mean uh if you're out there listening you know we have a lot lined up here uh we're gonna start having on guests uh every week if we can uh you know we're gonna explore the depths of everything that's happening in the AI community uh and you know try and focus on Ruby as much as we can yeah uh you know as things progress. 00:49:51.440 --> 00:50:18.239 A lot happening and you know it should be exposed and find out more about you know how people are using it because like everybody's using it a little bit different and I'm always you know kind of impressed anytime I I see somebody talking about what they're working on like whoa like yeah you can do that you know and like uh we gotta get more of that you know we gotta get more of that and we have to make it less shocking when we hear about a new cool thing right like you know you come here and you're on the ground floor and then you know and then everybody knows. 00:50:18.400 --> 00:50:18.559 Right. 00:50:18.719 --> 00:50:19.920 That we can all be on the same page. 00:50:20.079 --> 00:50:23.679 Yeah totally all right Joe was it was great talking to you today. 00:50:23.840 --> 00:51:34.880 Uh yeah you too I feel like maybe we should close out uh you know just like sharing one thing that we found interesting recently uh you know in the AI space uh that you're pretty excited about uh um all right you sit you first I I can go first I've been messing around with uh code agents specifically Hugging Face has this thing called small agents and there is a a deep learning course on building these code agents with their hugging face small agents super cool it is in python but it like really eye-opening kind of what you can do when you empower an LLM with uh its own like ability to code and make use of coding in general so I highly recommend checking this out it's uh it's on deep learning one of Andrew Ng's courses building with code agents with hugging face small agents very cool I actually um I wanted to get that uh I wanted to highlight again that that Ruby OpenAI official project that I'm having trouble finding because I loved the woman who presented was like a no nonsense Rubyist and now I'm trying to find what it is. 00:51:35.519 --> 00:51:36.480 Give me a moment here. 00:51:36.639 --> 00:53:05.599 There we are yeah so I just I liked the the openai ruby it's openai slash openai dash Ruby it's good software it makes a lot of uh it makes a lot of use of sorbet which is a little contentious around deaf methods but you know in a real way um it tries to to give you some static typing or some benefits of static typing it's uh it's insanely fast it's it's a really good place to start if you're trying like most people you're just starting out you're trying uh you're trying to integrate with uh you know ai in an interesting way that's not through just a a chat this is a good a great place to start yeah that's awesome I'm I'm happy to see that come up too yeah well this has been really great Joe we gotta do more and more of this yeah we're branding ourselves the Ruby AI podcast we'll see how that sticks I feel like we're in for a lot of fun times ahead of us I know you'll be at RailsConf coming up here I will be at Railsconf I'll I will also be there awesome so if you're out there listening come and see us we'd love to talk about AI and and Ruby and uh who knows maybe we'll we'll have a show uh I don't wanna you're speaking so I I don't want to like put any pressure on you also do a show yeah I think it'll be fun we could we could right uh I feel like that could be a lot of fun yeah but yeah stay tuned and we'll be releasing more and more yeah stay tuned all right thanks everyone we'll talk to you soon