النص 🔗
WEBVTT
00:00:00.000 --> 00:00:03.040
Hey, it's Jason, host of the Code with Jason podcast.
00:00:03.200 --> 00:00:04.160
You're a developer.
00:00:04.320 --> 00:00:06.000
You like to listen to podcasts.
00:00:06.080 --> 00:00:07.440
You're listening to one right now.
00:00:07.599 --> 00:00:13.359
Maybe you like to read blogs and subscribe to email newsletters and stuff like that.
00:00:13.519 --> 00:00:14.640
Keep in touch.
00:00:15.759 --> 00:00:23.199
Email newsletters are a really nice way to keep on top of what's going on in the programming world.
00:00:23.920 --> 00:00:26.559
Except they're actually not.
00:00:26.960 --> 00:00:36.159
I don't know about you, but the last thing that I want to do after a long day of staring at the screen is sit there and stare at the screen some more.
00:00:36.320 --> 00:00:40.000
That's why I started a different kind of newsletter.
00:00:40.079 --> 00:00:43.439
It's a snail mail programming newsletter.
00:00:43.600 --> 00:00:51.359
That's right, I send an actual envelope in the mail containing a paper newsletter that you can hold in your hands.
00:00:51.520 --> 00:00:58.960
You can read it on your living room couch, at your kitchen table, in your bed, or in someone else's bed.
00:00:59.039 --> 00:01:01.920
And when they say, What are you doing in my bed?
00:01:02.079 --> 00:01:04.560
You can say, I'm reading Jason's newsletter.
00:01:04.640 --> 00:01:05.840
What does it look like?
00:01:06.319 --> 00:01:12.319
You might wonder what you might find in this snail mail programming newsletter.
00:01:12.640 --> 00:01:20.159
You can read about all kinds of programming topics like object-oriented programming, testing, DevOps, AI.
00:01:20.400 --> 00:01:23.519
Most of it's pretty technology agnostic.
00:01:23.920 --> 00:01:41.680
You can also read about other non-programming topics like philosophy, evolutionary theory, business, marketing, economics, psychology, music, cooking, history, geology, language, culture, robotics, and farming.
00:01:42.000 --> 00:01:45.920
The name of the newsletter is Nonsense Monthly.
00:01:46.239 --> 00:01:50.239
Here's what some of my readers are saying about it.
00:01:50.560 --> 00:01:56.799
Helmut Kobler from Los Angeles says thanks much for sending the newsletter.
00:01:57.040 --> 00:02:00.640
I got it about a week ago and read it on my sofa.
00:02:00.799 --> 00:02:06.159
It was a totally different experience than reading it on my computer or iPad.
00:02:06.400 --> 00:02:12.000
It felt more relaxed, more meaningful, something special and out of the ordinary.
00:02:12.639 --> 00:02:18.560
I'm sure that's what you were going for, so just wanted to let you know that you succeeded.
00:02:18.879 --> 00:02:20.400
Looking forward to more.
00:02:21.680 --> 00:02:31.120
Drew Bragg from Philadelphia says Nonsense Monthly is the only newsletter I deliberately set aside time to read.
00:02:31.520 --> 00:02:43.599
I read a lot of great newsletters, but there's just something about receiving a piece
Load full transcript
of mail, physically opening it, and sitting down to read it on paper that is just so awesome.
00:02:44.319 --> 00:02:45.599
Feels like a lost luxury.
00:02:58.080 --> 00:03:02.560
Something about holding a physical piece of paper that just feels good.
00:03:03.360 --> 00:03:04.159
Thank you for this.
00:03:04.400 --> 00:03:05.680
Can't wait for the next one.
00:03:06.400 --> 00:03:16.000
Dear listener, if you would like to get letters in the mail from yours truly every month, you can go sign up at nonsensemonthly.com.
00:03:16.719 --> 00:03:19.599
That's nonsensemonthly.com.
00:03:20.719 --> 00:03:21.919
I'll say it one more time.
00:03:22.159 --> 00:03:25.120
Nonsensemonthly.com.
00:03:26.319 --> 00:03:30.000
And now without further ado, here is today's episode.
00:03:45.360 --> 00:03:48.080
Hey, today I'm here with Ernesto Tag Worker.
00:03:48.400 --> 00:03:49.360
Ernesto, welcome.
00:03:49.680 --> 00:03:52.960
Hey Jason, great to be here and chat with you again.
00:03:53.360 --> 00:04:00.879
Yeah, uh you're coming close to setting a record, I think, of of most times appeared on the Code with Jason podcast.
00:04:01.039 --> 00:04:02.800
Um I'll have to go back and count.
00:04:02.879 --> 00:04:12.000
Um, but for anybody who hasn't heard you on the numerous uh podcasts that you've been on before, um, can you tell us a bit about yourself?
00:04:12.719 --> 00:04:17.199
Yeah, um I'm founder and CTO at fastruby.io.
00:04:17.680 --> 00:04:25.439
Um we do a lot of talk a lot of work with technical debt and Rails upgrades and performance optimization.
00:04:25.680 --> 00:04:38.000
And FastRuby itself is a productized service by Ombu Labs, and that's our consulting side, and we do a lot more work with AI custom models for our clients.
00:04:38.160 --> 00:04:47.199
And uh I'm happy to talk about anything related to like technical debt remediation and the intersection with AI and agents and all that.
00:04:47.600 --> 00:04:55.279
Yeah, and today I was interested to talk with you about the ways that you're using AI in connection with your Rails upgrade work.
00:04:55.360 --> 00:04:58.959
Um that's something you and I have talked about offline before.
00:04:59.120 --> 00:05:15.040
Um and it makes a lot of sense to me because Rails upgrades, it's the kind of thing, correct me if I'm wrong, but there's a certain amount of grunt work to it and a certain amount of probably just like researching and finding out, finding out like, okay, what's this difference?
00:05:15.120 --> 00:05:16.800
What do I need to do exactly?
00:05:16.959 --> 00:05:20.800
Blah, blah, blah, ripe for AI automation.
00:05:21.040 --> 00:05:23.360
Um, but anyway, tell me about that.
00:05:24.399 --> 00:05:25.279
Yeah, yeah.
00:05:25.439 --> 00:05:44.319
There are a lot of things in an upgrade project that are you know very repetitive, and for that, AI is great because you can automate part of it, or you can have Claude, uh, you know, or you can develop skills for Claude or something like that that will help you uh do some of the repetitive work.
00:05:44.639 --> 00:05:55.600
And then there's also like the other side where you know projects will like monkey patch Rails and extend like private APIs and stuff like that.
00:05:55.759 --> 00:06:03.920
And then you kind of have to go dig deeper and try to figure out like what why is this active record callback different?
00:06:04.160 --> 00:06:07.920
And why am I seeing this failure in CI?
00:06:08.319 --> 00:06:17.279
Um, so yeah, there's like a lot of like debugging and investigation when it comes to like an upgrade project.
00:06:17.439 --> 00:06:23.519
And what we do is work a lot with monoliths that have been around for more than 10 years.
00:06:23.680 --> 00:06:34.160
So there's a lot has happened in those 10 years, and we come in and we try to figure out how to get them to like Rails 8 and Ruby you know 3.4 or 3.5.
00:06:34.879 --> 00:06:35.199
Okay.
00:06:35.759 --> 00:06:37.519
And how did you start?
00:06:37.680 --> 00:06:47.759
Um, like okay, so at some point in the past, there was zero AI involved in your upgrade projects because AI didn't exist in the way it does now.
00:06:48.000 --> 00:06:55.839
Um when when you first started applying AI, what were what were some of those ways that you applied it, if you remember?
00:06:57.680 --> 00:07:02.879
Yeah, we've always had this drive to like automate part of the process.
00:07:03.120 --> 00:07:11.839
We wrote a gem called Rails Upgrade many, many years ago to basically help us with the strong parameters migration.
00:07:12.160 --> 00:07:16.560
And you know, a lot of the work could be done with scripts and transformations.
00:07:17.040 --> 00:07:26.160
Um but then when AI came and you know became like so popular, we saw it as uh well, actually as a risk.
00:07:26.319 --> 00:07:30.000
You know, we do 80% of our revenue comes from upgrades.
00:07:30.240 --> 00:07:36.720
So someone could develop like an AI agent that basically replaces fastRuby.io.
00:07:37.600 --> 00:07:53.279
So about two, three years ago, we decided to like turn that into an opportunity and start building our own AI tooling to be the be the service that does use AI to make the upgrades easier.
00:07:53.439 --> 00:08:20.319
Um and about two months ago, we launched our AI enhanced Rails Upgrade Roadmap, um, which basically takes uh project source code and does a bunch of calculations behind the scenes, uses a lot of data sources to say, okay, this project has 17 deprecation warnings that you're gonna have to fix for the next jump, but also for future jumps.
00:08:20.480 --> 00:08:21.920
And it gives you a plan.
00:08:22.079 --> 00:08:36.720
If you have an application that's running like Rails 2, 3, and you want to take it all the way to Rails 8, you can plug in your GitHub repo, submit it, and then you're gonna get an action plan to you know that's tailored to your application.
00:08:37.120 --> 00:08:52.639
And that's one of the tools that that we just released,
and we have more coming down the pipeline uh to make it easier for you know clients that that might not be ready to hire us, but you know, they might want to do it themselves.
00:08:52.720 --> 00:09:00.639
So we want to be you know driving innovation there and coming up with like more skills for Claude to make it easier for teams to upgrade.
00:09:00.960 --> 00:09:02.960
And can you refresh my memory a little bit?
00:09:03.120 --> 00:09:05.840
Like the pre-AI format of it.
00:09:06.159 --> 00:09:11.200
Would you do some kind of like discovery process and put together a roadmap and stuff like that?
00:09:11.440 --> 00:09:15.919
Which which components of the process were already there before AI?
00:09:16.000 --> 00:09:22.320
Is this is this kind of automating and augmenting something that was already there or adding something new or both?
00:09:23.120 --> 00:09:25.840
Yeah, it's kind of like gluing it all together.
00:09:26.000 --> 00:09:29.120
Um, you know, there are so many different resources out there.
00:09:29.200 --> 00:09:36.240
Like you could use um there's Rails diff, um, which basically shows you the diff between versions of Rails.
00:09:36.480 --> 00:09:43.360
Um there's Railsbump.org is an open source project that was started many, many years ago.
00:09:43.440 --> 00:09:47.600
Uh it was called Ready for Rails, and it was like ready for Rails4.
00:09:48.000 --> 00:09:56.480
Uh and then Manual, this engineer in Germany, took it over and integrated it with like GitHub Actions and stuff.
00:09:56.720 --> 00:10:02.799
And then you know, he kind of gave it to us because he was like done maintaining it, and now we're running that.
00:10:02.879 --> 00:10:15.679
And Rails Bump basically takes your gemfile.lock and it looks at all the dependencies you have and it tells you like which versions are not going to be compatible with your future versions of Rails that you need to upgrade to.
00:10:16.080 --> 00:10:23.440
Um and then you know, there are like RubaCop cops that you could use for guiding your upgrade project.
00:10:23.600 --> 00:10:28.480
So there's definitely a lot of tooling, or there was a lot of tooling even before AI.
00:10:28.720 --> 00:10:46.320
And now that with AI, I think uh you know it's definitely like useful, but I don't think you can like submit like your entire application to JetGPT and be like, give me an action plan that is like accurate to my project and my dependencies.
00:10:46.480 --> 00:10:59.200
Uh not saying you know, you might be able to do something like that, but then there are also like you know privacy implications, and I don't know if you want to be feeding all that data to JetGPT, or you want to be very careful if you do that.
00:10:59.759 --> 00:11:07.919
And I imagine that the upgrade process certainly still involves a non-trivial amount of human judgment.
00:11:08.080 --> 00:11:09.600
Would I be right in that?
00:11:11.279 --> 00:11:11.759
Yeah.
00:11:12.000 --> 00:11:14.960
Yeah, there you know, there are
some gotcha.
00:11:15.440 --> 00:11:18.960
Like um we usually try to go like one minor version at a time.
00:11:19.120 --> 00:11:24.240
Uh but you know, Rails 5.2 to 6.1, we would usually say, let's just do one jump.
00:11:24.399 --> 00:11:34.879
No reason to do 5.2 to 6.0 and then to 6.1 because for big projects is like a lot of QA time and um just go one jump there.
00:11:35.279 --> 00:11:45.840
Um but I I think most of the human judgment and upgrade project is basically like estimating like how long is it going to take you to upgrade from X to Y?
00:11:46.159 --> 00:11:48.559
And we still get a lot of questions about that.
00:11:48.799 --> 00:11:55.120
Companies still need to get budgets approved for those projects, so so that's when they come to us.
00:11:55.519 --> 00:12:22.879
And one of the things we used to do before the AI roadmap is um basically do everything manually, get access to the source code, get access to the dependencies and forks and all that, and then basically compare an action plan, a generic action plan to our client's source code and dependencies, and then pick and choose which action steps that we needed to include in the plan.
00:12:23.279 --> 00:12:32.000
And then at the same time, we needed to add like a blind estimation process based on our you know 50,000 hours of doing upgrades for clients.
00:12:32.159 --> 00:12:39.919
We would look at all the data and all the metadata to say, okay, this is gonna take between five and seven developer weeks.
00:12:40.240 --> 00:12:53.759
And right now that we don't have the estimates in the AI enhanced roadmap, and I don't think we would want to add them because they're it's just so tricky to estimate, and I wouldn't give that to an AI right now.
00:12:54.320 --> 00:12:54.960
Right.
00:12:55.360 --> 00:13:07.840
Um there's okay, so there's there's this interesting there there's so many um aspects of AI that are so interesting to think about.
00:13:08.159 --> 00:13:14.960
Um and and one is people view AI as being as being a speeder-upper.
00:13:15.440 --> 00:13:21.759
Um like I can do the same thing in half the time or an eighth of the time or something like that.
00:13:22.000 --> 00:13:24.879
Um but I don't exactly look at it like that.
00:13:25.039 --> 00:13:27.120
Like sometimes that's the case, certainly.
00:13:27.360 --> 00:13:38.080
Um, but most of the time the way that AI speeds me up is like there are just certain like categories of work that I kind of just don't do anymore.
00:13:38.320 --> 00:13:44.480
Um like for example, I've been doing some side projects recently in Python and in Rust.
00:13:45.039 --> 00:13:50.960
Um my Python it I my my Python knowledge is not particularly deep.
00:13:51.279 --> 00:14:02.080
Um I could write a Python prop program from scratch with no AI, but anything non-trivial, I'd have to go look up a lot of syntax.
00:14:02.320 --> 00:14:04.799
Um and then Rust, I don't know at all.
00:14:04.960 --> 00:14:14.720
I can't I couldn't even do a hello
world in Rust, but with clawed code, I've written a whole complex program in Rust because I never touched the code at all.
00:14:15.120 --> 00:14:18.559
Um by the way, this is a throwaway experimental project.
00:14:18.639 --> 00:14:20.799
I would never do that for a production project.
00:14:21.200 --> 00:14:32.080
Um anyway, I don't exactly look at it as being a speeder-upper, it's just like uh it it takes away these things that were barriers before and stuff like that.
00:14:32.240 --> 00:14:34.559
Um so I'm curious how you look at it.
00:14:34.639 --> 00:14:39.679
I I imagine it's not exactly just doing the same exact work faster.
00:14:39.919 --> 00:14:43.279
I imagine it's different somehow, but I'm I'm not sure how.
00:14:43.440 --> 00:14:45.039
Can you speak to that a bit?
00:14:46.159 --> 00:14:59.919
Yeah, and I love that you bring this topic up because there's a lot of hype there, you know, and someone who's like, yeah, it's gonna be five times faster, they're probably full of shit.
00:15:00.159 --> 00:15:14.320
I'm sorry, pardon my French, but if you come out and say, Oh yeah, our teams are five times faster, I would ask, like, yeah, what data are you using to come up with that?
00:15:14.720 --> 00:15:16.080
Well, and just what does that mean?
00:15:16.240 --> 00:15:19.039
What what does five times faster mean to them?
00:15:19.679 --> 00:15:20.720
Yeah, yeah.
00:15:20.879 --> 00:15:33.120
And I see, you know, there are studies against that the hype, and it's like, yeah, it is making us faster, it is speeding us up, but it's not 5x, it might be like closer to like 1.5 or 2x.
00:15:33.759 --> 00:15:45.360
Um, what I do, my main use use case for for AI or or clot or cursor uh or whatever, chat GPT is to get me unblocked.
00:15:45.440 --> 00:15:58.720
You know, sometimes when I find something and I have no idea or no motivation to do the thing, I'll just ask ChatGPT like, oh, I need to do this thing and I don't know anything about it.
00:15:58.879 --> 00:16:01.279
Can you just point me in the right direction?
00:16:01.600 --> 00:16:08.080
And then I find that you know it's it's fixing my blockers faster.
00:16:08.240 --> 00:16:18.320
Instead of like me Googling and plugging in keywords that I don't even know the right keywords, uh Chat GPT will usually give me a better idea of where to start.
00:16:19.600 --> 00:16:22.480
Yeah, I want to unpack that a little bit.
00:16:22.639 --> 00:16:26.000
Um, you said the thing about motivation or something.
00:16:26.240 --> 00:16:29.759
Um, that is definitely definitely something for me.
00:16:30.000 --> 00:16:42.159
So like I've referred to AI before as like a mental lubricant, like it it it or maybe even mental WD40, which is not a lubricant, it's like an unsticker, you know.
00:16:42.480 --> 00:16:52.240
Um so when there's a task, sometimes, you know, I'm just in like a mental fog or I'm like tired or whatever.
00:16:52.399 --> 00:17:06.000
And I don't know about you, but sometimes I'll just like
look at something somebody wrote in Slack and I like reread it three times, and I'm just like, yeah, I'm looking at the words, but they have no meaning to me, and I just like can't think.
00:17:06.720 --> 00:17:09.359
Um and so I I do this a lot.
00:17:09.440 --> 00:17:11.119
This is a pattern I use a lot now.
00:17:11.359 --> 00:17:23.279
I will take an entire Slack conversation and paste it into um AI and just say, like, hey, what are these people talking about?
00:17:23.599 --> 00:17:28.240
And like you you see this conversation, I'm Jason.
00:17:28.480 --> 00:17:31.440
So like help me figure out what I need to do.
00:17:32.079 --> 00:17:40.640
Um like I don't even have the brain power to formulate a smart question, so I just like paste in the context and I'm just like help me.
00:17:40.960 --> 00:17:42.319
It's as vague as that.
00:17:42.480 --> 00:17:53.359
And then it helps me like get a little bit unblocked, and it's like, oh, okay, I see that they're they're talking about this, so maybe my next step can be this, and it helps me move forward a little bit.
00:17:53.519 --> 00:17:55.759
Does any of that resonate with you at all?
00:17:56.799 --> 00:17:58.160
Yeah, yeah, for sure.
00:17:58.319 --> 00:18:16.799
And you know, we're constantly bombarded with notifications and we're constantly dis distracted, and sometimes you're trying to do some code, and then you get a text message from your wife or you know, uh alert from Slack where you know it's like an emergency, but it doesn't really pertain to you.
00:18:16.880 --> 00:18:19.359
Like you don't really need to look into that.
00:18:19.599 --> 00:18:36.319
What I will do sometimes when I'm like struggling to focus is we'll feed whatever error I found in the console to JetGPT or Copilot or whatever IDE uh code editor I'm using and being like, oh, I did this thing and I got this error.
00:18:36.559 --> 00:18:45.920
And then that will give me like a couple ideas of like, oh, it could be this, you know, Postgres is not running, so you need to like start Postgres uh or something like that.
00:18:46.000 --> 00:18:57.839
That's like, oh yeah, duh, that's obvious, and maybe it would have taken me like two minutes to find that, and but if you just plug it in in seconds, it's like okay, cool, here, try these things and this will fix it.
00:18:58.240 --> 00:18:59.279
Yeah, it's interesting.
00:18:59.440 --> 00:19:14.559
It's it's an application of an idea that I've been using for decades at this point, which is like um always try to operate at the lowest RPMs possible.
00:19:15.599 --> 00:19:28.160
So you know, with the analogy there, of course, is like uh you can you can drive a car in first gear and have the pedal to the metal and only go like 40 miles an hour or whatever.
00:19:28.319 --> 00:19:29.839
That's not realistic.
00:19:29.920 --> 00:19:34.559
You wouldn't go 40 miles an hour in first gear, but anyway, side note, okay.
00:19:34.640 --> 00:19:35.039
Sorry.
00:19:35.279 --> 00:19:36.880
Side
note to my side note.
00:19:37.279 --> 00:19:42.799
One time I went to Las Vegas and I did that thing where you could drive a Ferrari and it was insane.
00:19:43.279 --> 00:19:43.759
I was there.
00:19:45.119 --> 00:19:46.960
Well okay, I don't remember.
00:19:47.200 --> 00:19:50.880
I've done that several times, and this was yeah.
00:19:51.119 --> 00:19:57.599
Anyway, um in s in like second gear you could go like 40, third gear you could go like seventy.
00:19:58.079 --> 00:19:59.279
It was insane.
00:20:00.240 --> 00:20:00.640
Yeah.
00:20:00.960 --> 00:20:05.759
Um anyway, uh uh what was I even saying?
00:20:06.000 --> 00:20:08.720
Oh yeah, operating at low low RPMs.
00:20:09.039 --> 00:20:09.359
Okay.
00:20:10.160 --> 00:20:20.960
So it it's w when I like get an error message or something, what I've noticed a lot of people do is like they'll scrutinize the error message and be like, okay, what is this?
00:20:21.119 --> 00:20:21.920
What does it mean?
00:20:22.079 --> 00:20:25.359
And I'm like, dude, like don't don't think about it.
00:20:25.519 --> 00:20:29.839
Just copy paste it into Google and see what it says.
00:20:30.000 --> 00:20:42.160
Like obviously read it once in case it's something obvious, but it's if it's not something obvious, don't think about it, just paste it into Google and and use Google as the mental lubricant.
00:20:42.400 --> 00:20:46.079
Um and 90% of the time that'll get you through it.
00:20:46.319 --> 00:20:53.759
Same exact idea with AI, when there's something, error message, whatever, read it once to see if it's something obvious.
00:20:53.920 --> 00:20:56.640
If it's not, just paste it into AI.
00:20:56.799 --> 00:21:01.759
You don't even need to type anything more, you just paste it and hit enter, and that's all.
00:21:01.920 --> 00:21:06.880
Um and 90% of the time it'll get you get you past that.
00:21:07.119 --> 00:21:10.799
So again, AI is not always about speeding me up.
00:21:10.960 --> 00:21:14.319
Sometimes it's about doing the same work at lower RPMs.
00:21:14.559 --> 00:21:19.039
That way I don't run out of juice before the day ends.
00:21:19.279 --> 00:21:23.119
Um I can keep actually working the entire day.
00:21:24.240 --> 00:21:24.880
Yeah.
00:21:25.359 --> 00:21:30.319
Yeah, I see it also as uh rubber ducking on steroids, you know.
00:21:30.400 --> 00:21:42.160
Uh just the process of feeding the prompt to the AI is like, you know, useful to you because you're trying to connect the dots, and then maybe the answer comes to you even before you send the prompt.
00:21:42.400 --> 00:21:51.200
But if it doesn't, then the prompt will probably get you there faster than if you were just um you know rubber ducking 100%.
00:21:52.160 --> 00:22:05.359
Yeah, and and I gotta say this also like AI is is so much more powerful when it's combined with like common sense and stuff like that.
00:22:05.680 --> 00:22:10.880
Um I I've said this before, AI is kind of like a multiplier.
00:22:11.200 --> 00:22:19.440
Like, take
somebody who's already really effective, and AI will make them like uh ten times as effective.
00:22:19.759 --> 00:22:27.519
But take somebody who's not very effective, give them AI, and it might not really make a difference.
00:22:28.160 --> 00:22:41.920
Um so there's there's things like when you're when you're trying to fix a bug, um AI does not obviate the need to clearly articulate exactly what the bug is.
00:22:42.480 --> 00:22:56.799
It's crazy how much people will see a bug and then start to try to fix the bug without even knowing um without even having articulated or understood what the problem actually is.
00:22:57.440 --> 00:23:06.240
And so like using AI has to be coupled with these like common sense practices that you would do anyway.
00:23:06.960 --> 00:23:14.559
That doesn't have anything to do with Rails upgrades or anything, but um side note that that applies to pretty much anything.
00:23:15.359 --> 00:23:19.839
Yeah, no, and we see this a lot in on the Umble Labs side.
00:23:20.079 --> 00:23:28.079
We are working with teams to basically get them to use AI tooling effectively and securely, you know.
00:23:28.480 --> 00:23:35.359
Sometimes uh teams, you know, are not leveraging those tools as as as much as us.
00:23:35.519 --> 00:23:36.480
They should.
00:23:36.720 --> 00:23:40.880
And part of it is that they're not wiring them correctly.
00:23:41.200 --> 00:23:44.559
And I'd love to talk about this.
00:23:44.720 --> 00:23:53.200
And you might uh you know want eventually to add like an MCP server to uh Saturn CI.
00:23:53.440 --> 00:24:13.200
You know, um I know that every service out there, like New Relic and Honey Badger, I all the services that we use in production have or will have soon an MCP server that you can feed data back to your um models or like your AI agents.
00:24:13.519 --> 00:24:22.720
Um I think the common mistake that I see is that teams start using cursor or claude, but they don't set it up correctly.
00:24:22.880 --> 00:24:33.519
So one of the things we recently published is an assessment for these teams to basically come to us and we'll give you an idea of where you're falling short.
00:24:33.680 --> 00:24:37.839
We're calling this a maturity model for engineering teams.
00:24:38.400 --> 00:24:47.680
And we can look at all the tooling and all the practices your engineering teams are using and tell you where you're falling short.
00:24:47.839 --> 00:25:03.119
It's like, oh well, you have RuboCop and you have Reek and you have RubyCritic for this Rails code base, and you have code quality standards and complexity standards that you hold your humans accountable when they miss that mark.
00:25:03.599 --> 00:25:08.319
But have you configured Claude to feed off this data?
00:25:08.559 --> 00:25:25.519
Like have you do you have agents that feed off APM data in production, or do you have agents running that will look at CI and it will consume like the failure that happened in your feature branch and give you a suggestion right away in like minutes?
00:25:25.680 --> 00:25:33.200
You're like, oh, I wrote this feature, I submitted the pull request, and I realize it broke something in the accounting module.
00:25:33.440 --> 00:25:44.319
Then CI can quickly report back to an agent, and then that agent could like co-pilot could come in and say, hey, this is great, but it's gonna break this other feature.
00:25:44.480 --> 00:25:50.400
So here's a suggestion to make it work for your feature and for the existing features.
00:25:50.480 --> 00:25:53.680
Um have you thought about that for Saturn CI?
00:25:54.079 --> 00:25:55.839
Uh I've definitely thought about that.
00:25:56.079 --> 00:26:01.680
Um, but it is uh you said this off the air, it's it's it's kind of a long ways off.
00:26:01.839 --> 00:26:11.759
There's a lot of more basic stuff that is gonna make sense to do before that, um, including especially uh sales and marketing.
00:26:12.000 --> 00:26:21.519
Uh as of this recording, I have zero customers, so I'm not gonna add anything like that before I get uh some some customers using it.
00:26:21.680 --> 00:26:32.160
Um but you you touched on something which I find very interesting, and and we're working on this at my job at Cisco Meraki right now.
00:26:32.400 --> 00:26:34.480
Um and that's kind of the idea.
00:26:34.640 --> 00:26:36.559
Tell me if this is what you're getting at at all.
00:26:36.720 --> 00:26:40.720
Um it's something like the idea of applying AI to ops.
00:26:41.359 --> 00:26:57.599
Um, like for example, something that that we're working on is ingesting all of our deployment-related data, such as here's when certain deployments went out and stuff like that, here's when certain PRs were merged, that kind of stuff.
00:26:57.759 --> 00:27:05.759
Then you can answer questions like, hey, is is this particular change in production right now, or is it not, that kind of thing.
00:27:06.079 --> 00:27:19.279
Um and what you mentioned with like ingesting APM data and stuff like that, especially because all these things can be connected to one another, and then you have a really powerful ability to answer questions.
00:27:19.519 --> 00:27:31.279
And of course, if you put all that stuff in a database, you can have an LLM translate a natural language question into an SQL query and then get your answer.
00:27:31.440 --> 00:27:33.440
Is is that the kind of thing you were touching on?
00:27:33.519 --> 00:27:36.240
Or is that does that overlap with what you were thinking about?
00:27:37.440 --> 00:27:45.359
Yeah, yeah, and that's a really powerful idea to have an agent like answer questions about your code base.
00:27:45.920 --> 00:27:56.079
Um we're not quite there yet, uh, but we definitely are helping teams set up their tooling so they can answer questions like that.
00:27:56.319 --> 00:28:08.079
Um I can think of a use case that you could ask this chatbot or this agent, uh, yeah, is this change in production?
00:28:08.160 --> 00:28:22.319
But you could
also ask it to, you know, if it's properly wired, feeding off the APM new relic or something like that, you could say, Oh, I see uh performance regression in the accounting module.
00:28:22.559 --> 00:28:26.240
Uh now our response times are like 20% worse.
00:28:26.400 --> 00:28:34.160
Uh what changed in you know, since that performance degradation started happening?
00:28:34.400 --> 00:28:41.839
And then that chatbot could be like, oh, actually all these changes were shipped between this date and this date.
00:28:42.000 --> 00:28:46.559
And maybe these changes could impact that performance.
00:28:46.640 --> 00:28:51.680
Um yeah, that that would be super powerful if if you guys get to that point.
00:28:52.559 --> 00:28:53.039
Yeah.
00:28:53.279 --> 00:29:09.200
Yeah, I think it's just a general um good pattern of like taking your your taking any any database and layering an LLM on top of it so that you can ask questions in natural language of your data.
00:29:09.359 --> 00:29:22.160
Speaking of Saturn CI, I've done that with Saturn CI uh to ask questions like how many uh test runs were um were run in the last 24 hours or something like that.
00:29:22.559 --> 00:29:24.319
Um and it's really interesting.
00:29:24.559 --> 00:29:37.680
It's it's it's one of those magical computer moments, you know, to be able to just ask the question and then it gives you the answer and it like shows you how it got the answer, and it's like, oh yeah, that's actually correct.
00:29:37.839 --> 00:29:40.319
That's amazing that that figured that out.
00:29:41.519 --> 00:29:47.359
Um okay, so you mentioned, I think you mentioned, uh, MCP.
00:29:47.759 --> 00:29:50.880
This is something that I personally am behind on.
00:29:50.960 --> 00:29:53.680
I don't understand it as well as I would like to.
00:29:53.839 --> 00:30:02.079
I've asked a couple other guests in the past, like, hey, what the heck is MCP exactly, and like how are you using it and stuff like that?
00:30:02.240 --> 00:30:05.279
Um, and I'd like to ask you the same thing.
00:30:05.519 --> 00:30:09.440
Um how how are you using MCP in in your work?
00:30:09.680 --> 00:30:18.559
And I don't want to put you on the on the spot, but if you're able to kind of explain what it is and stuff like that, just anything you're able to share.
00:30:20.559 --> 00:30:24.799
Yeah, I again I'm not an AI expert.
00:30:24.960 --> 00:30:30.799
I I'm also still learning as I go, but the way I see it, it's just a protocol.
00:30:30.960 --> 00:30:38.079
You know, MCP is a protocol, and then you can implement it in any way you want, but you know, following the protocol.
00:30:38.720 --> 00:30:40.720
Model context protocol, right?
00:30:41.519 --> 00:30:42.079
Right.
00:30:42.319 --> 00:30:50.720
And then um basically LLMs can feed off this data and use it in their context.
00:30:50.960 --> 00:31:17.200
Um so I that's as far as I know, and I understand like the potential behind that, but I also understand like the security implications there, you know, because sometimes uh what we've seen is that some clients want to use AI and they want to integrate everything, but not everyone should have access to everything, right?
00:31:18.079 --> 00:31:32.640
So then it becomes a question of like, can you build agents that are only accessed by DevOps or by only by engineering teams, and it won't leak sensitive information.
00:31:33.039 --> 00:31:51.599
Um, and that's some sometimes that's where we come in and we try to help our clients figure out like, yes, this is great, and you want to start using AI, and you want to have an MCP server, and you want to have an agent that integrates with all these different MCP servers to solve a specific problem.
00:31:52.079 --> 00:32:11.759
Uh, but you have to be very careful about who's going to be using this agent because you know, if you're gonna have all your infrastructure using AI, you don't want to leak data from like the sales department to like the engineering department with data that they should not be uh seeing.
00:32:12.400 --> 00:32:22.880
So I think it's super powerful, but it also raises the question of like security and privacy and making sure that people only see what they're supposed to see.
00:32:23.279 --> 00:32:24.400
Yeah, I'm curious.
00:32:24.640 --> 00:32:31.920
Have you experienced or witnessed any like bad AI moments?
00:32:32.079 --> 00:32:41.119
Like, I don't want to say necessarily something like disastrous, but like something where it's like, oh wow, we this is a good reminder, we need to be careful with this.
00:32:41.200 --> 00:32:42.960
Have you seen anything like that?
00:32:44.160 --> 00:32:47.759
Um no, I haven't seen a lot.
00:32:47.920 --> 00:33:02.559
I mean, that I can talk about publicly, uh, but I have seen um infrastructures that are not properly configured and access that's not properly set up.
00:33:02.799 --> 00:33:16.960
So then when we get clients that want to build something on top of that, we raise questions about hey, have you looked at the roles and the departments and the access of the data to like the current infrastructure?
00:33:17.119 --> 00:33:38.079
And then that kind of is a blocker for us to work with a client because before we build an AI agent for them or a model, predictive model, or something like that, we need to ask those questions because we need to be like responsible about the work we do, and we don't want to build on top of something that's already not properly set up.
00:33:38.400 --> 00:33:51.759
Um on the upgrade side, I could tell you many stories about big corporations, you know, public companies that are running really, really old versions of Rails in production.
00:33:52.000 --> 00:34:03.039
But you know, of course, because of NDAs, I can't tell you names, but I can tell you there are many companies out there that are running really, really old versions of Rails and Ruby.
00:34:03.599 --> 00:34:34.719
And if any of your listeners are in that situation, I would recommend
they look at Rails LTS, which is a secure old version of Rails that this company in Germany maintains and backports security patches, and please pay them their yearly fee to use those versions so that you don't get you know hacked or you don't get you know ransom whatever.
00:34:34.960 --> 00:34:39.599
You don't you don't get some bad actor exploiting a known Rails security hole.
00:34:40.239 --> 00:34:43.840
Um I had an interesting sp experience with AI.
00:34:44.400 --> 00:34:52.000
Uh all right, I I'm gonna obfuscate this in order to uh be because I shouldn't be too specific.
00:34:52.239 --> 00:34:59.360
Um but there's a developer uh a who I'm familiar with who I don't trust all that much.
00:34:59.840 --> 00:35:04.880
Um and I was looking at uh some of his code that he wrote.
00:35:05.119 --> 00:35:20.079
I was looking at a at a PR that he created, and I was looking at the code and I'm like, hmm, maybe you wrote this, but I think a lot of this was probably written by AI, but I couldn't be sure.
00:35:20.880 --> 00:35:30.480
Um and and it was an interesting moment because it's like if you look at a PR and it's obviously created by AI, that's one thing.
00:35:30.800 --> 00:35:34.480
If you look at it and it's obviously written by a human, that's another.
00:35:35.199 --> 00:35:54.239
But if you're looking at code from again, a person you don't really trust and it looks like it was written by AI, I don't want to approve that because I I can't make a safe assumption that this person has vetted this code, you know?
00:35:54.559 --> 00:36:12.320
If if I'm looking at a PR of yours, Ernesto, I'm gonna naturally assume that like, okay, I don't understand why this particular part is here, but I think it's a pretty safe bet that Ernesto like didn't just recklessly put some random stuff in here.
00:36:12.559 --> 00:36:16.079
Um so I'm gonna I'm gonna not worry about that.
00:36:16.480 --> 00:36:24.079
Um on the other hand, if if there's something that's totally AI generated, I'm gonna scrutinize liter literally everything.
00:36:24.800 --> 00:36:33.519
Um but when it comes from a person and some of it might be AI generated, it creates kind of a tricky situation.
00:36:34.079 --> 00:36:37.920
Um have you thought about that or have you have you seen anything like that?
00:36:38.480 --> 00:36:58.000
Yeah, it's funny that you mentioned that because this reminded me of one of our clients that basically reached out to us to add like a senior Rails engineer to a team of non-Rails engineers generating Rails code.
00:36:58.239 --> 00:36:58.719
Oh wow.
00:36:59.039 --> 00:37:03.599
So they could tell that there was something off there.
00:37:04.000 --> 00:37:27.760
And that's the problem sometimes when you have, you know, I I I started with Java, I love Java, but if you have like a Java engineer use Claude or an AI tool to generate code for Rails, and then you start building features that like that, then the code sometimes starts to look like non-rails code.
00:37:27.840 --> 00:37:36.559
It's Ruby, it works on Rails, but it's not really the way you know convention over configuration is supposed to work.
00:37:36.880 --> 00:37:45.360
It works, but it does feel fishy and it does sometimes make it hard to maintain, and it's also like hard to explain.
00:37:45.519 --> 00:37:52.719
Like, yes, this is Rails code, but that's not usually the way a senior Rails engineer would write it.
00:37:52.880 --> 00:37:54.000
And why are you doing it?
00:37:54.159 --> 00:38:03.280
And it's like, oh, okay, because you don't have a ton of experience with Rails, but you do have a ton of experience with Java, and this is how in Java it would look.
00:38:03.679 --> 00:38:17.920
Um so what this client did is we brought they brought us in to basically guide this team into you know writing more Rails-y code, even using AI.
00:38:18.079 --> 00:38:19.360
So I think that's the danger.
00:38:19.519 --> 00:38:43.519
It's like, oh, we're using Ruby, we're using Rails, we're using AI, and then you're creating an application that is gonna work, but then when someone else comes along and it's a human uh that has a ton of Rails experience, they're gonna be a little bit confused, and they're gonna be like, I if everything is in the wrong place, it works, but it's not the way a Rails application usually is written.
00:38:43.679 --> 00:38:51.119
So that then there is that real danger, and this is like a real story from a client that I think we all can learn from.
00:38:51.679 --> 00:38:53.199
Yeah, interesting.
00:38:53.519 --> 00:39:25.840
Um I've seen that same phenomenon where somebody comes from a different community and they're writing Ruby code, and it it has a little bit of a funny flavor to it because they're kind of taking the culture or style or whatever from that other community and applying it to this same this this language where those same things maybe don't make complete sense, or they're just uh they they go against the style conventions, culture, that kind of stuff.
00:39:26.480 --> 00:39:30.960
Um and with AI generated code, I'm sure you've seen this.
00:39:31.039 --> 00:39:38.239
My my number one frustration with AI generated code is that it writes a lot of what I call speculative code.
00:39:39.119 --> 00:39:59.760
Um so I I I think of programming a lot in terms of bets, um, where it's like I'm I don't know, for example, I I'm refactoring this piece of code because I'm making a bet that after I clean this after I make the investment to clean up this area of code, uh it's likely.
00:40:00.159 --> 00:40:04.639
To pay off at some point in the future and have a positive return on investment.
00:40:05.360 --> 00:40:22.239
I know that out of every 10 bets I make, I'll probably lose six of them and win four of them, but I'm betting that my entire investment portfolio on balance is going to have a positive return on investment, even though I'm not winning every single bet.
00:40:23.840 --> 00:40:27.440
And there's a big
difference between investment and speculation.
00:40:28.000 --> 00:40:41.280
Speculation is when you're just like, well, I uh I I'm guessing that maybe the value of this thing will go up in the future, so I'm gonna buy this uh in FT or whatever.
00:40:41.519 --> 00:40:51.199
Um and then uh you know notoriously speculate speculation is very risky and you can just lose a whole bunch of money that way.
00:40:52.239 --> 00:40:57.199
So speculative code is code that's written without a good justification.
00:40:58.719 --> 00:41:05.440
Um you know, uh uh Stephen Baker calls it getting a case of the Midas Wells.
00:41:05.840 --> 00:41:11.360
Like you're in an area of code and you're like, well, I might as well add this because maybe we'll need it someday.
00:41:11.519 --> 00:41:13.199
That's speculative coding.
00:41:13.920 --> 00:41:24.079
And so code written by AI is just rife with these, like, I don't know, a conditional for like if this variable is set, then do such and such.
00:41:24.400 --> 00:41:33.440
It's like, well, okay, but that variable is like never ever not gonna be set, and we have no reason to believe it would never it would ever be not set.
00:41:33.679 --> 00:41:40.159
So like this code is like all liability and no asset, and it's just so frustrating.
00:41:40.400 --> 00:41:49.679
And so if you have AI write any non-trivial amount of code, you're gonna have to strip so much of the speculative code out of there.
00:41:50.079 --> 00:42:01.519
And to me, it just seems like such a nightmare to like have to review PRs by people who are using AI generation as their main way of writing code.
00:42:01.920 --> 00:42:11.039
Um I don't have like a super uh coherent point or anything with that, but had have you noticed that phenomenon yourself?
00:42:12.079 --> 00:42:19.360
Yeah, I think that's where this is what makes me believe that you know options are always there.
00:42:19.840 --> 00:42:43.519
AI is gonna be even better and more present, but it's just like so hard to have all the context, and you'll sometimes get context from the business, and it's not gonna be in a Jira story or a bug report or an epic because it would just blow up the description of that story.
00:42:43.760 --> 00:42:51.679
But when you're in a company and you're building on a feature, you have a ton of context, and the more time you're in a company, the more context.
00:42:51.840 --> 00:43:01.920
And you know that if they're like, well, we want you to add this feature to export to JSON, and then you're like, okay, you ask the right questions.
00:43:02.000 --> 00:43:04.400
You're like, okay, Jason, what about XML?
00:43:04.480 --> 00:43:06.320
Is this like gonna be a thing in the future?
00:43:06.480 --> 00:43:08.079
Is it gonna be CSV?
00:43:08.320 --> 00:43:26.480
And then they're like, no, no, just JSON, you know, and maybe the product manager just tells you that to keep the scope small, but then I don't know, at the same time, you do know that this company does work with Microsoft or some other corporation that loves XML.
00:43:26.639 --> 00:43:51.599
So then you're like, okay, I am gonna write this code in a way that there's a formatter, and then later, if I want to have like a different formatter, I just have like a clear contract of this class called whatever formatter export, and then it'll do it for me first with JSON, but then it is eventually like easily extensible to XML.
00:43:51.920 --> 00:44:02.400
So I guess to sum it up, I think the context of a story or a bug report is definitely useful for AI and it's gonna solve that problem.
00:44:02.719 --> 00:44:12.159
But the way it's solved, it's gonna be like very much tied to like the human driving the AI tool and the context they have.
00:44:13.440 --> 00:44:16.719
Um okay, we're we're getting close to time.
00:44:17.360 --> 00:44:35.760
Um so I want to um want to wrap up soon, but uh before we go, um can you can you tell us again just about even as long as I've known you and as many times as we've talked, um I have trouble keeping straight like Ambu Labs and Fast Ruby and all that stuff?
00:44:35.920 --> 00:44:39.440
Um can you share those things again and and what they are?
00:44:39.599 --> 00:44:49.039
And like if somebody is interested in working with you, um what's what's the way that makes the most sense to engage with you on that?
00:44:49.199 --> 00:44:51.039
Um how should they contact you?
00:44:51.199 --> 00:44:53.599
What what's gonna happen if if they contact you?
00:44:53.679 --> 00:44:55.360
How does that relationship begin?
00:44:55.599 --> 00:44:59.039
Anything you you want to share on that kind of stuff?
00:44:59.840 --> 00:45:00.639
Oh yeah, sure.
00:45:00.719 --> 00:45:01.280
Thank you.
00:45:01.519 --> 00:45:22.000
Uh if uh anybody's interested in Ruby Rails or tech debt remediation beyond upgrades, um they can just go to fastruby.io and send us a message there or follow us on Mastodon or Blue Sky or LinkedIn.
00:45:22.079 --> 00:45:29.360
Uh we're always sharing interesting content uh for anybody to kind of DIY their tech debt remediation.
00:45:30.079 --> 00:45:45.039
And if anybody's interested in AI and the way that engineering teams are using AI, um go to ombulabs.ai and that's where we publish most of our AI-related content.
00:45:45.119 --> 00:45:55.519
And some of it might include might include include Ruby, but uh there I think we talk a little bit more about like Python and Kubernetes and machine learning models.
00:45:55.599 --> 00:46:02.239
Um and again, they can follow us in social media and we'd be happy to connect and see how we can help.
00:46:02.559 --> 00:46:02.880
All right.
00:46:02.960 --> 00:46:05.360
Well, we'll put all that stuff in the show notes.
00:46:05.440 --> 00:46:07.760
And Ernesto, thanks so much for coming on the show.
00:46:08.000 --> 00:46:13.760
Yeah, thanks so much, and uh, I'm yeah, happy that we got to connect and catch up.
00:46:14.000 --> 00:46:14.480
Definitely.
00:46:14.639 --> 00:46:15.440
Thank you.