תעתיק 🔗
WEBVTT
00:00:00.000 --> 00:00:09.599
I remember talking to a prospect and I asked them how often they released and they said once a quarter and I asked if they wanted to ship more and they said no.
00:00:09.759 --> 00:00:12.080
Which now just seems absolutely jaw-dropping.
00:00:12.400 --> 00:00:21.679
You know, Anthropic just came out with a blog and they talked about how they have moved from 2,500 changes to 30,000 changes in a day.
00:00:21.839 --> 00:00:26.480
It is just a very different pace that needs a different sort of workflow for production.
00:00:26.640 --> 00:00:28.480
If a deploy goes wrong at 2 a.m.
00:00:28.640 --> 00:00:31.519
and it's rolled back within a second, did it really go wrong?
00:00:32.640 --> 00:00:39.280
The AI Native Dev is a podcast for developers and engineering leads at the cutting edge of AI and agentic coding.
00:00:39.439 --> 00:00:49.679
Join your hosts, Guy Fajani, and meet Simon Maple every week as we chat with the most exciting voices in AI and tackle the biggest questions facing developers today.
00:00:50.159 --> 00:00:52.479
This is the AI Native Dev.
00:00:57.039 --> 00:00:59.920
AI DevCon is coming back to London.
00:01:00.079 --> 00:01:02.240
Our first London event sold out in June.
00:01:02.399 --> 00:01:14.079
After our second New York event this November, we're heading back to the brewery Moorgate in 2027 with a bigger lineup and two full days of talks from the people actually building with agents.
00:01:14.239 --> 00:01:23.920
You'll hear honest takes on where the industry's at, real stories from teams adopting AI, the wins as well as the messy bits, and where they think things are heading next.
00:01:24.079 --> 00:01:25.680
It's on June 7th and 8th.
00:01:25.760 --> 00:01:28.640
And don't tell anyone, but we've got a really great deal for you.
00:01:28.799 --> 00:01:32.400
Scan this QR code to grab an early bird ticket for just£100.
00:01:33.040 --> 00:01:34.400
See you in London.
00:01:42.719 --> 00:01:53.280
Today we'll talk about what is even a release and how do you kind of manage these uh kind of insane pace of application deliveries that we have.
00:01:53.359 --> 00:02:10.560
Uh and we have a guest who's probably one of the best equipped uh people to deal with that, talking about sort of delivery and you know, dynamic delivery and such uh in of applications for quite a few years and also kind of a good friend, Edith Harbaugh, uh who is the uh founder and CEO of LaunchDarkly.
00:02:10.639 --> 00:02:13.199
Uh Edith, thanks for joining us here.
00:02:13.840 --> 00:02:16.639
Yeah, it's great to be here in the virtual studio.
00:02:17.039 --> 00:02:36.080
So I guess for for starters, I'd love to kind of get your view a little bit of what does it mean to do this type of feature flag controls, release management, give us a little bit the right terminology uh at you know a pace that seemed or like makes the cloud era pace feel like a snail.
00:02:37.840 --> 00:02:39.680
Yeah, so it's been a journey.
Load full transcript
00:02:39.840 --> 00:02:46.159
Um so when we started the company in 2014, it was when cloud was still kind of a new idea.
00:02:46.400 --> 00:02:49.759
You know, a lot of people were actually still shipping on packet software.
00:02:49.919 --> 00:02:57.759
Uh I remember talking to a prospect and I asked them how often they released, and they said once a quarter.
00:02:58.000 --> 00:03:00.960
And I asked if they wanted to ship more, and they said no.
00:03:01.360 --> 00:03:07.120
Uh, which now just seems absolutely jaw-dropping of that they thought once a quarter was more than sufficient.
00:03:07.199 --> 00:03:07.599
Yep.
00:03:07.840 --> 00:03:18.400
Um now uh, you know, Anthropic just came out with a blog and they talked about how they have moved from 2,500 changes to 30,000 changes in a day.
00:03:18.560 --> 00:03:19.199
Yeah.
00:03:19.439 --> 00:03:31.520
Like in a single day, um, which uh is inspiring and amazing, but it's just a very different pace that needs a different sort of workflow for production.
00:03:32.159 --> 00:03:33.680
Yeah, yeah, no doubt.
00:03:34.000 --> 00:03:42.159
But how so so how do you like how do you think about sort of feature flag in this reality or or about kind of managing behavior?
00:03:42.319 --> 00:03:48.479
Because I feel today there's definitely an ethos of ship it, you can monitor the sort of the reality of it.
00:03:48.639 --> 00:03:51.039
So if there's a problem, you can roll back and you can iterate.
00:03:51.199 --> 00:03:55.199
Clearly, all these things require the right infrastructure to be able to do as well.
00:03:55.439 --> 00:04:11.919
Um but uh but then when you think about you know this type of pace, oftentimes the the decision of whether to roll back and all that is takes longer than the um the the time kind of you you already want to evolve your next feature, right?
00:04:12.000 --> 00:04:13.840
You already want to build the next one.
00:04:14.400 --> 00:04:19.279
Yeah, I mean, and that and that's really so that's really what Launch Darkly and feature flags can help with.
00:04:19.360 --> 00:04:22.720
And let me be clear, uh, you don't need Launch Darkly to use feature flags.
00:04:22.800 --> 00:04:25.040
Um this is a technique, we're a vendor.
00:04:25.199 --> 00:04:27.920
Um I would like to think that we make it a lot easier.
00:04:28.079 --> 00:04:31.439
Um, but everything I'm talking about you can do on your own.
00:04:31.680 --> 00:04:40.720
Um so that the the concepts you just talked about are a rollback, and to do that effectively, you do need feature flags.
00:04:40.879 --> 00:04:57.519
You know, so if you're looking at uh even if you're not anthropic scale, but if you're looking at, let's say, 10 or 100 changes a day, um, you do need a way where you can quickly detect if something is going right or wrong and move it back if it's going wrong.
00:04:57.839 --> 00:05:08.160
And that's where uh one, you could do it on your own if you had feature flags, but too launch darkly could really help you because you can have that as automated.
00:05:08.720 --> 00:05:15.759
So if uh if it's going bad in the in the field, you could within seconds say, hey, this isn't working, and have it automatically rolled back.
00:05:16.000 --> 00:05:21.839
And this is what AI is very good at because uh you don't need a human in that loop at all.
00:05:22.000 --> 00:05:28.160
Uh you don't need somebody to be sitting at a dashboard looking at it, it could just be a very autonomous rollback.
00:05:28.399 --> 00:05:39.920
And then you multiply that times 10, 100, 1000, and it's a very good fail-safe if you're shipping a lot of code to production and you want not to have to babysit every single change.
00:05:40.160 --> 00:05:40.720
Yeah.
00:05:41.120 --> 00:05:48.160
So how so so there's the technical aspect of figuring out uh how to sort of you know do the flag and sort of apply the change on it.
00:05:48.319 --> 00:05:54.879
Uh, but you know, I hearing this, my my kind of bigger question is how do you decide that you need to roll back?
00:05:54.959 --> 00:06:00.319
And if you do that agentically as well, what would you sort of say, we're sort of looking right now at sort of state of the art.
00:06:00.399 --> 00:06:05.600
Maybe we'll talk a little bit about the journey, but if you're really good at it, how do you how do you make those decisions?
00:06:06.000 --> 00:06:25.120
Um so a lot of the work we're doing right now is to make that very automated, um, so that you can send us whatever telemetry you have, whether it be error rate, whether it be um uh elevated uh operational metrics, whether it be something as fuzzy as just uptick in tickets.
00:06:25.439 --> 00:06:28.480
It's a good it's a good question though, about like what is it that you should feature flag?
00:06:28.560 --> 00:06:32.720
Because like some things get you in trouble if you start kind of getting a little bit too dependent on feature flags.
00:06:32.879 --> 00:06:48.639
You want to roll them out gradually, but you end up creating inconsistent experiences, or when you decide to roll back, you kind of have, I don't know, data structure problems, or you maybe are letting down some user who has now kind of formed some attachment to the to the capability you just launched.
00:06:48.879 --> 00:06:58.879
I guess what would you sort of systems aside, like uh from from the you know, probably have more exposure to sort of correct and incorrect feature flags than uh than almost anyone.
00:06:59.040 --> 00:07:03.839
Like what's a how would you define like a good feature flag uh criteria?
00:07:04.720 --> 00:07:12.079
Um well so leaving aside launch darkly as a vendor, uh I would say a good one is always one that you understand.
00:07:12.720 --> 00:07:17.600
Um if you have put in one and you don't understand what it does, that's where you get into a lot of trouble.
00:07:17.839 --> 00:07:40.959
Um so I've seen people who have still flags, a flag that was once very important, but nobody understands
what it does anymore, and it actually uh accidentally got triggered, or uh uh the worst case I heard is um one uh one company that had something to shed a lot of load, and one feature flag, if you toggled it, it would actually prevent you from uploading any new files.
00:07:41.600 --> 00:07:44.720
Uh and it would actually get toggled and nobody knew what it was.
00:07:44.959 --> 00:07:49.120
So the number one uh I'd say guideline is just make sure you understand that.
00:07:49.279 --> 00:07:53.040
And then you can still have a human in the loop uh to make those judgment calls.
00:07:53.519 --> 00:07:57.519
It's it's so interesting to think about um uh judgment, right?
00:07:57.600 --> 00:08:06.000
I feel uh when I look at how we build a TESL and I look at the pace, alignment becomes this really kind of interesting uh challenge, right?
00:08:06.079 --> 00:08:21.439
Because you have a bunch of people, they're individually like they're individually empowered and uh you know just sort of so uh equipped, you know, like they have such powerful tools at their disposal, so that each of them operates like I don't know, a directorship, right, of sort of of people.
00:08:22.240 --> 00:08:44.480
But uh keeping them uh aligned as they make many, many of these decisions is uh is is is hard because you know it takes the same amount of time to talk to their you know colleague and sort of you know describe what it is that they're they're uh they're building, and it's still a pain or like a product discomfort if uh if if they're misaligned.
00:08:44.799 --> 00:09:02.559
Like how how do you keep like I I'm trying to think about uh sort of deploying capabilities with feature flags, and so you add to that kind of matrix or to the to the already kind of existing risk of uh I'm not sure what my colleague has shipped, right?
00:09:02.639 --> 00:09:14.480
And we're trying to kind of reconcile this as we as we fly the plane, you add a potentially very long list of feature flags that further customize and sort of tweak behavior, and you do that.
00:09:14.879 --> 00:09:17.440
I guess if you the when does this go wrong?
00:09:17.600 --> 00:09:25.759
You know, when uh is there a I don't know if there's magic, like I guess I'm sort of yearning for these sort of magic numbers of like never have more than a hundred feature flags over the eye.
00:09:25.840 --> 00:09:33.840
Like, I guess what's the what are sort of guidelines to say, you know, until this point it's helpful and from here on it's kind of detrimental?
00:09:34.559 --> 00:09:39.039
Uh again, I don't want to give too much of a vendor pitch, which means I'm about to give one.
00:09:39.279 --> 00:09:45.600
Um I I think the guideline is as many as you can safely manage uh in a system.
00:09:46.320 --> 00:09:52.799
Um so a lot of small startups, I talk to them and they they say, hey, we've rolled our own feature flagging system.
00:09:52.960 --> 00:09:53.840
I say, wonderful.
00:09:54.080 --> 00:09:54.720
That's great.
00:09:54.879 --> 00:10:02.000
Like if you have like 10 to 20 or 40 and they're well understood within your org, you know, you don't you don't really need a vendor.
00:10:02.399 --> 00:10:13.440
Um it's saying when you get to a scope when you have multiple teams who are perhaps distributed, uh, you start to need more of a framework about managing them.
00:10:13.759 --> 00:10:23.759
Uh you might also have dependent flags, which is pretty common where you have perhaps a flag that's uh handling five other subsystems where you want them to move in parallel.
00:10:24.559 --> 00:10:29.120
Uh and you also want to be really dil diligent about cleanup.
00:10:29.440 --> 00:10:36.960
Uh so where I've seen people get into trouble is if they've just been adding features for every release.
00:10:37.200 --> 00:10:45.279
And there's always what I say the boring part of doing the dishes and going back and really examining whether a flag can be used or not.
00:10:46.080 --> 00:10:59.919
So I guess related to that a little bit, um, I think um I guess what I was also kind of wondering is whether the use of feature flags, and especially as you kind of grow them, depends.
00:11:00.000 --> 00:11:05.919
Uh I mean you sort of said like how much you understand them, but also about how well can you clarify the outcome, right?
00:11:06.000 --> 00:11:10.399
And and and in that sense, like I've I I really sort of think a little bit about agents.
00:11:10.559 --> 00:11:29.200
There's this um uh comparable that I often uh kind of run into, which is uh, you know, in DevOps we were talking about all these like great traits of like, you know, you need to instrument your systems and you need to have good lineage and you need to you know manage your data and have kind of you know understandable kind of uh the uh architecture, information architecture.
00:11:29.360 --> 00:11:32.240
And then if you did all of those, agents can actually help you run faster.
00:11:32.399 --> 00:11:39.120
And agents can also help you kind of overcome a bunch of uh uh of uh pains that you had and kind of get your house in order.
00:11:39.279 --> 00:11:46.399
But if you if you had a bunch of that infrastructure, those are actually now multiplied, like the agents really benefit from sort of having that data and that access.
00:11:46.559 --> 00:11:47.039
And I guess so.
00:11:47.120 --> 00:11:52.720
I guess sort of similarly, I wonder if with with the feature flags, you know, one of the questions is like what's your judgment?
00:11:52.799 --> 00:11:55.679
You know, what is the uh the thing that you measure?
00:11:55.919 --> 00:11:57.919
So I guess how much does that factor in?
00:11:58.000 --> 00:12:02.879
Like we're in this world where a lot of things are ill-defined, you know, of what correct behavior is.
00:12:04.000 --> 00:12:05.039
Like, does that get in the way?
00:12:05.120 --> 00:12:07.200
I'm also kind of wondering about what you're sort of seeing in the wild.
00:12:07.279 --> 00:12:16.000
There's
there's kind of established businesses, but when you're building kind of newer capabilities even within those established businesses, they don't necessarily know what good looks like, right?
00:12:16.080 --> 00:12:20.000
Like what is it that the feature flag should aspire to achieve?
00:12:20.879 --> 00:12:24.639
Well, that that was actually, I think, about four or five questions wrapped into one.
00:12:25.519 --> 00:12:27.360
There's a I've got a tendency of doing that.
00:12:27.519 --> 00:12:28.080
Apologies.
00:12:28.799 --> 00:12:30.480
Well, no, it was a really good question.
00:12:30.639 --> 00:12:32.960
Um it's it's something that I've been thinking a lot about.
00:12:33.120 --> 00:12:44.320
Uh, you know, so I originally got it, I originally founded Launch Darkly because I'd been an engineer and then a product manager, and I had built things that just did not work.
00:12:45.039 --> 00:12:56.879
You know, in terms of uh they worked at an engineering level, but then they got out into the field and customers were sometimes just completely apathetic to them.
00:12:57.600 --> 00:13:04.159
You know, so and I say apathetic deliberately because it wasn't they loved or hated a feature, it was just that they did not care.
00:13:04.480 --> 00:13:05.600
They didn't care, yeah.
00:13:05.840 --> 00:13:06.799
They didn't care at all.
00:13:07.360 --> 00:13:08.320
I didn't do that.
00:13:08.399 --> 00:13:09.679
Um that's painful.
00:13:09.840 --> 00:13:16.000
Um the other part of an engineer is that you put out something and it turns out that it breaks in the field, which is also painful.
00:13:16.399 --> 00:13:26.799
Um, you know, that uh you always either over-engineer or under-engineer, depending on where the minimum viable product spectrum you are.
00:13:27.200 --> 00:13:30.320
Uh so sometimes you think a lot of people use it and nobody uses it.
00:13:30.480 --> 00:13:34.879
Sometimes you think you'll have very light workload and suddenly it's through the roof.
00:13:35.120 --> 00:13:40.639
Um so I think in this world of agents, it forces you to think about that.
00:13:41.279 --> 00:13:46.240
Um, you know, so I think about a year ago was the heyday of the one-shotter.
00:13:47.200 --> 00:13:54.320
Um, you know, which we we even tried internally at Launch Sharkly about, like uh, you know, the one-shot of refactor this part of our code base.
00:13:54.639 --> 00:13:55.279
Yeah.
00:13:55.919 --> 00:13:59.679
And that phrase works about as well as you think it might.
00:14:01.120 --> 00:14:02.480
Which is okay.
00:14:02.639 --> 00:14:09.519
Like to a human, what seems very clear, which you might give to a team of humans of refactor the code base, they go away and refactor it.
00:14:09.679 --> 00:14:11.600
To an agent, they just have no idea what that means.
00:14:12.159 --> 00:14:13.360
Refactor for what?
00:14:13.600 --> 00:14:15.279
Like, what does that mean?
00:14:15.519 --> 00:14:36.000
Um, so what we found in our own development is that the the dream
of the one-shot agent that goes away and does the work is more like us trying to decompose work into smaller and smaller and smaller and smaller um atomic units uh until we got to a point where we could point agents at different points, parts of the code base, and say, okay, here are your tasks.
00:14:36.080 --> 00:14:42.320
Like make sure that everything is commented out, make sure that an agent understands it, make sure that um we're looking for bloat.
00:14:42.639 --> 00:14:47.200
Uh so it was being very specific about what we meant by refactor.
00:14:47.440 --> 00:14:47.919
Yeah.
00:14:48.399 --> 00:14:51.840
This is like a defining defining what what you actually want.
00:14:51.919 --> 00:14:54.080
You know, you need to kind of express that.
00:14:54.159 --> 00:14:56.639
What good what what do you want and what good looks like.
00:14:56.720 --> 00:15:01.840
So you give the right instruction and you're able to kind of verify it as part of the kind of software building in this case.
00:15:02.240 --> 00:15:06.720
Yeah, and and and and even add on to that that agents do not know the concepts of good and bad.
00:15:06.879 --> 00:15:08.399
Like these are abstract words though.
00:15:08.639 --> 00:15:10.720
Yeah, no, though what what you think good looks like.
00:15:10.799 --> 00:15:16.159
You know, like what is uh what is a legitimate outcome, right, versus what is a failed outcome.
00:15:16.559 --> 00:15:28.960
Yeah, so like if you told uh an engineer like make sure your code looks good, there's usually a common understanding about like what it what that word good means, like you know, um, but an agent has no concept of it.
00:15:29.200 --> 00:15:35.759
So all of this work that we were doing into defining work for agents is actually really good for a feature flag.
00:15:36.000 --> 00:15:41.600
Because your your original question was, well, how do you define what success looks like?
00:15:41.679 --> 00:15:44.399
And it's like, well, you're defining it from the beginning for the agent.
00:15:44.559 --> 00:15:52.720
Like you're saying, hey, this part of code needs to be performant to this standard, um, it needs to, you know, load within this time.
00:15:52.879 --> 00:16:06.399
And then those are very useful metrics that you can then push into a feature flag for when it's actually out into the world of hey, all that stuff I told you up front about how I wanted this thing to behave, make sure it still behaves that way out of the real world.
00:16:06.720 --> 00:16:07.120
Yeah.
00:16:07.440 --> 00:16:08.960
I I think that makes a lot of sense.
00:16:09.039 --> 00:16:26.799
And I I I feel again, there's like a very common through line, I think, in everything that you deal with sort of delegating any type of work to agents, which is you have to do the hard part, which is describe what what success looks like, what failure looks like, uh, what are the guardrails that you want them to stay within?
00:16:26.879 --> 00:16:30.879
You know, like what are what are you know what are the lines, like what's disallowed, what's allowed.
00:16:31.039 --> 00:16:40.879
And so you need to do, you know, what we oftentimes avoid and just saying like we'll know it when we see it, you know, for for humans, you know, now we need to actually codify that uh in the system.
00:16:41.200 --> 00:16:45.360
Um how I guess kind of pushing that a little bit to the extreme.
00:16:46.159 --> 00:16:49.360
How how far would you let agents uh evolve?
00:16:49.440 --> 00:16:56.320
Like, you know, you're you're probably like the most advanced sort of user of uh of feature flags, or like I'm up there.
00:16:56.559 --> 00:16:59.440
Um how much would you let them just experiment?
00:16:59.519 --> 00:17:24.799
Like, hey agents, like like look at all my telemetry and how people are using it, listen to all the customer calls that I've had, you know, you know, like read all of my internal documentations, propose changes, roll them out if I'm being bold here, like you know, build them, roll them out under feature flags, see what the result is, uh come up, and you know, I'll be surprised and amazed at the uh at the great enhancements that you've sort of built for my for my product.
00:17:25.279 --> 00:17:27.440
Um I mean that's that's the dream.
00:17:27.680 --> 00:17:31.599
I'd say we are not quite there, but we're moving there.
00:17:32.000 --> 00:17:38.480
Um like uh we have a pretty strong internal blogging culture at Launch Darkly.
00:17:38.799 --> 00:17:45.519
Um so we have roughly 200 people in RD, and I encourage them to write, and I try to read them.
00:17:45.680 --> 00:17:54.640
And I was reading one this morning about how um they'd built a fleet of about 20 plus agents to work on a part of the code base.
00:17:55.440 --> 00:17:59.920
And then the work of the engineer actually turned out to be just reviewing all the PRs.
00:18:00.240 --> 00:18:05.920
Um so in my mind they had transitioned more to uh what used to be called like the classic architect role.
00:18:06.240 --> 00:18:06.319
Yeah.
00:18:06.480 --> 00:18:11.200
Um I'm old enough where like um you used to have the architects that would just do code review.
00:18:11.440 --> 00:18:20.480
Um so there was still a human in the loop, um, there was still some judgment, uh, but it just went much faster because somebody else was doing actual code writing.
00:18:20.640 --> 00:18:26.319
Um and then when this stuff is ready to go out to production, it already has guardrails pre-built in.
00:18:26.640 --> 00:18:31.599
Um, because that's part of the process is of before you change the code, figure out what the guardrails are.
00:18:31.680 --> 00:18:33.680
So it is much more autonomous.
00:18:33.759 --> 00:18:34.079
Yeah.
00:18:34.319 --> 00:18:41.039
Uh we've we've also done projects where uh we pointed an agent at a backlog of bugs.
00:18:41.680 --> 00:18:58.559
And I think this is just amazing, um, where you can point it at a backlog of bugs, and it could produce PRs, you know, for
sometimes months, if not years, of things that never bubble to the top of fixing, and just start releasing them with Gardros.
00:18:59.440 --> 00:19:08.559
And I love this because I think uh, as I said, I I came out of I started off in engineering and I remember just the tedium of sitting in a bug review.
00:19:09.279 --> 00:19:16.880
Like, did you ever see a bug review just like, hey, uh it doesn't work on this browser, there's this cosmetic flaw, is this worth fixing or not?
00:19:17.039 --> 00:19:20.319
And you just spend hours on that, like it's a tedium.
00:19:20.480 --> 00:19:20.799
Yeah.
00:19:20.960 --> 00:19:23.759
Uh versus an age that just like goes through all of it.
00:19:24.319 --> 00:19:24.720
Yeah, yeah.
00:19:26.400 --> 00:19:28.160
I don't know if you're seeing the same things in your own development.
00:19:28.480 --> 00:19:32.960
We're we're definitely sort of, you know, definitely many, many, many things are uh are building there.
00:19:33.039 --> 00:19:38.000
And so but I guess I I want to then sort of push a little bit on like why not sort of drive the autonomy.
00:19:38.079 --> 00:19:39.200
So like what's getting in the way?
00:19:39.279 --> 00:19:53.839
Because like my my I guess my my my view is that the feature flags and sort of this type of kind of runtime management should enable you actually to release stuff that is that is bad, you know, like that is uh but it's okay because you're like gradually exposing people to them.
00:19:54.000 --> 00:20:02.720
And you know, to the extent that you have enough of a user base, uh, and yes, there's probably some constraints around like some areas in which your tolerance for a mistake is very low.
00:20:02.960 --> 00:20:20.400
But like why why wouldn't you like why do you need the human in the loop if you have the sort of the feature flag uh capabilities with sort of you know tight management uh and are able to gauge like if you have these sort of two that you have these controls and you have like a pretty good understanding of outcomes?
00:20:20.960 --> 00:20:24.799
Oh, I mean we're we're depending on the feature, we we basically do that.
00:20:25.039 --> 00:20:25.279
Yeah.
00:20:25.519 --> 00:20:32.480
Um like if it's a if it's a bug that's coming in uh on UI, we can fix it within minutes.
00:20:33.279 --> 00:20:36.640
Um it's it's very little uh human there.
00:20:36.720 --> 00:20:40.640
Um for things with a bigger surface area, there is a lot more human judgment.
00:20:41.039 --> 00:20:41.599
Yeah.
00:20:41.920 --> 00:20:42.319
Yeah.
00:20:42.480 --> 00:20:51.039
Um But I'm curious, I'm still sort of, I guess what would make you unleash this, sort of say, hey, in this area of the product, autonomously improve my product.
00:20:52.559 --> 00:20:54.559
So I like that you're pushing me.
00:20:55.119 --> 00:20:59.839
I think the question of autonomously approve is pretty broad.
00:21:00.799 --> 00:21:03.359
Um so let me let me give you a real example.
00:21:03.519 --> 00:21:07.119
Like we have SOC uh compliance, we have HIPAA compliance.
00:21:07.359 --> 00:21:07.680
Yeah.
00:21:07.920 --> 00:21:15.279
Um so just say autonomously approve is so broad that I don't know if an agent would understand that right now.
00:21:16.640 --> 00:21:16.880
Yeah.
00:21:17.119 --> 00:21:21.839
Yeah, it's interesting to So I I think compliance is definitely a legit answer.
00:21:21.920 --> 00:21:30.079
It's like hopefully a temporary answer, as in, you know, we should learn how to sort of build compliance systems that still have sufficient guardrails that they can evolve autonomously.
00:21:30.240 --> 00:21:32.880
But I think today that is still a scary proposition.
00:21:32.960 --> 00:21:43.119
You have to be very confident in the sort of guardrails that you have uh to be able to attest to an auditor uh that that you're doing it in that fashion.
00:21:43.359 --> 00:21:49.759
Um it just uh I guess I'm sort of pushing on it just because I feel um uh there's like different parts of it, right?
00:21:49.839 --> 00:21:54.640
There's the judgment of uh of w what is a reasonable change.
00:21:54.799 --> 00:21:59.759
I think agents are actually already pretty good at that, especially if you're sort of scoping kind of the volume of change for a proposal.
00:22:00.559 --> 00:22:06.880
They don't need necessarily know that it's great, but they'll have good kind of betting stats over here.
00:22:07.839 --> 00:22:13.759
Then the implementation, okay, they're getting pretty, pretty good of that, you know, like being able to like build the capability.
00:22:14.000 --> 00:22:17.519
And then you know, when you release it, there isn't a taste element, right?
00:22:17.599 --> 00:22:19.039
Or like a curation element.
00:22:19.279 --> 00:22:25.119
And you know, one way I think about feature flags is just can they replace taste?
00:22:25.200 --> 00:22:35.200
You know, I don't know, like with with ads, with sort of coupons to supermarkets, with doing it, like I don't know, a lot of taste goes away in favor of data, right?
00:22:36.000 --> 00:22:38.640
Oh, I I I see the same future.
00:22:38.799 --> 00:22:41.680
I mean, um, I would like to get there.
00:22:42.319 --> 00:22:48.160
I I think also though, people like predictability.
00:22:49.119 --> 00:23:02.480
Um, so a real example is, you know, we have customers who are in education or airlines, and they do not want all of their interfaces to change on a moment-by-moment basis.
00:23:03.119 --> 00:23:07.759
They're like, you know, the airline clerk, and which who knows might might go away.
00:23:07.920 --> 00:23:13.839
You know, somebody who's sitting on an interface and is like, I want to push this button, know that the button is in the same place as it was yesterday.
00:23:14.000 --> 00:23:15.440
They don't want that rate of change.
00:23:16.160 --> 00:23:22.720
Um, however, there are things on the back end that can be constantly iterating much faster just to improve that front end.
00:23:23.279 --> 00:23:32.000
Um something we've been talking about internally is uh Launch Darkly, uh the UI is all API driven.
00:23:32.880 --> 00:23:40.960
Uh about uh making it even easier for people to just have more of a conversational interface with Launch Darkly.
00:23:41.279 --> 00:23:46.559
Uh where instead of using our uh, for lack of a better word, canned UIs, yeah.
00:23:46.720 --> 00:23:52.880
Uh that you could be building UIs on the fly with an agent every time you want to talk with us, which I think is great.
00:23:52.960 --> 00:23:56.880
But I think some people want the familiarity of, I know where everything is.
00:23:57.200 --> 00:23:57.440
Yeah.
00:23:57.759 --> 00:24:00.160
This part is like the slow change as a feature.
00:24:00.319 --> 00:24:05.279
Uh yeah, for sort of some parts of the system is the like stability, really, is the feature.
00:24:05.680 --> 00:24:07.519
So uh let me kind of shift gears a little bit.
00:24:07.599 --> 00:24:15.119
So like I I I'm I'm very excited by by the notion of like dynamic behavior uh and you know, how do you how do you control it?
00:24:15.279 --> 00:24:23.519
I have all sorts of thoughts that probably will sort of suck us down the sort of the wrong rabbit hole over here around personalized software and uh and the likes.
00:24:23.680 --> 00:24:29.680
Um how how do you think about this type of delivery versus or like as part of the software factory?
00:24:29.839 --> 00:24:32.240
Is it the same, is it the same thing?
00:24:32.400 --> 00:24:43.839
Like you build, and maybe I'll I'll I'll stabilize a little bit to say like let's assume the core of the software factory is this like tickets to PR, but then the sort of the breadth of it thinks about the SDLC as a whole.
00:24:44.079 --> 00:24:51.440
Like, how do you think about Future Flags as a whole, but also maybe like Launch Darkley's place, right, in in that flow?
00:24:52.160 --> 00:25:00.720
Yeah, so it's it's it's interesting the software factory metaphor has really taken off, I'd say, in the last uh nine months.
00:25:00.960 --> 00:25:05.519
Um I personally like it, though I know there's a little bit of backlash against it.
00:25:05.759 --> 00:25:10.160
Um I like it in that a factory seems more predictable.
00:25:10.319 --> 00:25:13.680
It seems like there are gates and steps and repeatability.
00:25:13.920 --> 00:25:21.759
Um I know that some engineers fear that it makes them obsolete or just more rote.
00:25:21.839 --> 00:25:30.799
Uh so I I in my own vision, I think it frees them to be more creative, knowing that there is stuff that they can depend on.
00:25:31.119 --> 00:25:44.319
Um so that the knowing, for example, that you could go, to extend the factory metaphor, that you could go somewhere and like heat treat your plastic so that it's usable is actually a good thing rather than having to create that heat treater every time.
00:25:44.640 --> 00:25:51.920
So when I think about Launch Starkly, uh we assume that there's a lot of stuff upstream from us about your how you produce code.
00:25:52.160 --> 00:25:56.640
Like we don't want to compete with uh a cloud a clot or uh, you know.
00:25:56.880 --> 00:25:59.920
And then we also assume that you have some sort of code review facility.
00:26:00.000 --> 00:26:03.039
Uh, we don't want to compete with uh a code rabbit, for example.
00:26:03.279 --> 00:26:18.720
What we're really good at is that last step of you've built something, you want to release it to the world, and you want to have guardrails around it, um, so that once it's out in the world, uh the things we talked about before, you can either um break up who it goes to.
00:26:18.880 --> 00:26:25.680
So, for example, if you want to do a phase release where it goes to people in London and then in Scotland and then uh to the rest of the world, you can do that.
00:26:25.920 --> 00:26:31.279
And also, so if something is suboptimal, you could roll it back.
00:26:31.519 --> 00:26:39.680
Uh also if you're getting a lot of positive feedback, you can feed that very quickly, perhaps even to a coding agent to enhance a feature.
00:26:39.839 --> 00:26:53.839
Um, so for example, if you'd built uh a simple workflow and it's getting a lot of use, maybe in the future an agent could say, hey, this is getting a lot of use, let's uh continue iterating on it autonomously without even a human in loop.
00:26:53.920 --> 00:26:55.200
So that's kind of the vision.
00:26:55.359 --> 00:27:23.279
Uh what we're doing a lot at LaunchDarkly right now as a vendor, and what I said before is vendor independent, is building up uh agentic controls where you can tune your system about what does and does not need it to be flagged, uh so that you can have a risk tolerance for this is what I want to control, this is what a change that could just go through, but it gets smarter over time, so it's just auto-magic.
00:27:23.599 --> 00:27:30.799
And you can use that as hey, if any of these measures takes a jump, uh start rolling back.
00:27:31.039 --> 00:27:36.960
And you could do that in an automated fashion without a human ever knowing something went wrong.
00:27:37.279 --> 00:27:42.240
Uh we call it internally the the two if a deploy goes wrong at 2 a.m.
00:27:42.400 --> 00:27:44.880
and it's rolled back within a second, did it really go wrong?
00:27:46.240 --> 00:27:59.200
Um and then on top of that, what we're adding to Launch Starkling uh as a vendor is a lot of automation around making it a lot easier to know when to feature flag.
00:27:59.680 --> 00:28:04.079
Um so we've been using our own system for 10 plus years.
00:28:04.319 --> 00:28:10.559
And there is, I would say, some amount of friction about like you have to wrap a flag, you have to send this telemetry.
00:28:10.720 --> 00:28:14.400
Um, but in an agentic world, the agent can do all that for you.
00:28:14.640 --> 00:28:23.599
Like you could say, hey, uh instead of having to look at, hey, I want to wrap this flag, let me think about how what signals to send to it, you just
say, agent, do this for me.
00:28:23.839 --> 00:28:25.119
And it's very automated.
00:28:25.599 --> 00:28:28.400
Hey everyone, hope you're enjoying the episode so far.
00:28:28.640 --> 00:28:36.720
Our team is working really hard behind the scenes to bring you the best guests so we can have the most informative conversations about agentic development.
00:28:36.960 --> 00:28:42.319
Whether that's talking about the latest tools, the most efficient workflows, or defining best practices.
00:28:42.480 --> 00:28:46.079
But for whatever reason, many of you have yet to subscribe to the channel.
00:28:46.319 --> 00:28:53.119
If you're enjoying the podcast and want us to continue to bring you the very best content, please do us a favor and hit that subscribe button.
00:28:53.200 --> 00:29:00.559
It really does make a difference and lets us continue to improve the quality of our guests and build an even better product for you.
00:29:00.880 --> 00:29:02.799
Alright, back to the episode.
00:29:03.200 --> 00:29:08.400
So I I feel uh I think there's a lot of kind of value, right, in sort of that portion of it.
00:29:08.480 --> 00:29:18.400
Uh, but I do I want to sort of geek out a little bit about sort of product scope in this era, because I think the the the the thought about a software factory is just about like how all-encompassing it is, right?
00:29:18.480 --> 00:29:32.559
And I had uh uh quite a few months ago now, I had Merrill from uh uh Graphite uh on the podcast, and this was before the cursor acquisition, and you know, we we talked about whether code review will be you know a part of the of the factory, right?
00:29:32.720 --> 00:29:40.559
And and I feel uh I guess I would I would claim today that the answer is yes, code review is a part of the software factory.
00:29:40.720 --> 00:29:57.359
In fact, if you don't have kind of increasing autonomy, you know, with with sort of growing confidence and competence to be able to uh to merge things more and more you know autonomously, um, then then you're stuck, you know, like that becomes the bottleneck and you're you're just not gonna be able to push through it.
00:29:57.440 --> 00:30:11.519
Um and also uh in that conversation, we talked about how you know it like at the time they built like their first features to sort of say, hey, we when you found like a bug right in the code review of it, then hey, you can actually sort of fix it over here uh in the path.
00:30:11.599 --> 00:30:14.400
It's like, well, how how are you different than the coding agent?
00:30:14.559 --> 00:30:22.079
Uh I don't know, by the way, if the cursor conversation is already happening in the background and that happened, but uh uh you know the um you know how are you fixing there?
00:30:22.160 --> 00:30:26.000
And then can can't I just run your review as part of that coding agent?
00:30:26.079 --> 00:30:27.440
Like, why do I need to wait for the PR?
00:30:27.599 --> 00:30:29.039
Can I can I run it early?
00:30:29.200 --> 00:30:41.279
And so I guess I'm I'm sort of saying all of this because like how how do you think about does that not extend uh if you think about feature flags and this type of like dynamic insights, you can think of it as like a continuation of quality, right?
00:30:41.359 --> 00:30:51.599
You can look at things statically, you can look at sort of dynamic tests on it, and then you can look at like actual kind of real world experience uh of this capability that you're releasing it.
00:30:52.400 --> 00:30:54.559
Like does it make sense eventually?
00:30:54.640 --> 00:31:06.559
Like what's your what's your um uh uh vision or sort of like a prediction around the world in five years' time in terms of like division of kind of core capability providers, right?
00:31:06.640 --> 00:31:08.480
You know, in uh in the space?
00:31:08.960 --> 00:31:11.920
I think there will still be the last mile issue.
00:31:12.160 --> 00:31:18.960
Like you and a when you're talking about the factory metaphor, I I talked about I was thinking about how there's always a bottleneck somewhere.
00:31:19.119 --> 00:31:19.599
Yeah.
00:31:19.839 --> 00:31:23.920
And as soon as you f fix a bottleneck in one place, the bottlenecks move somewhere else.
00:31:25.920 --> 00:31:32.000
Um you know, so the bottleneck, I'd say, for most of my career has been just code.
00:31:33.279 --> 00:31:43.440
You know, uh how many engineers can we hire, how fast can they get up to speed, how many and then there's all sorts of artificial metrics of productivity that were gamed, like lines of code.
00:31:43.680 --> 00:31:48.079
I I don't actually care about lines of code, I care about productive code.
00:31:48.400 --> 00:31:51.920
Uh now code is incredibly cheap.
00:31:52.240 --> 00:31:56.319
Like you can ask Claude to make you code, and it will make far more code than you care to.
00:31:56.559 --> 00:32:01.680
Um and then the bottleneck was code review, and I think there's a lot of very good vendors right now for code review.
00:32:02.079 --> 00:32:15.599
Um, and then the bottleneck right now for me is there's a lot of code that's been code reviewed, and there's still this last mile of is it actually having a productive impact on the end person?
00:32:16.079 --> 00:32:20.240
And I say person, it could be uh a system, it doesn't have to be a person.
00:32:20.880 --> 00:32:25.119
But that gauge of is this doing something that I want?
00:32:25.440 --> 00:32:32.720
And I think is it kind of uh properly improving whatever is those business metrics that I've sort of defined as uh as good?
00:32:33.200 --> 00:32:42.079
Yeah, and I and I think it a change from the world in the past, uh, because I talked earlier about the age of package releases, is you used not to be able to get that telemetry.
00:32:42.480 --> 00:32:54.720
Like if you were shipping, like let's say, uh I'm old enough to remember Windows 95, like it came as this immense 32 disc blob, where it's impossible to say, hey, is this feature better than this feature?
00:32:54.880 --> 00:32:55.599
You just couldn't know it.
00:32:55.680 --> 00:33:03.519
But now with the ability to break down uh changes into smaller and smaller bundles, you can actually have a really good gauge of is this working?
00:33:03.759 --> 00:33:09.359
But you also want to be able to say, hey, all these changes together, is this having a holistic good effect?
00:33:09.519 --> 00:33:09.839
Right.
00:33:10.160 --> 00:33:10.400
Yeah.
00:33:10.559 --> 00:33:15.759
No, I think the management of it is quite complex, and I feel like there's a lot of uh industry learnings.
00:33:15.920 --> 00:33:28.960
I guess when I think about software factories, I feel on one hand, I feel the software factory foundation will be all-encompassing, as in it it will, it will like eventually you want to aspire to a thing that is the full loop, right?
00:33:29.039 --> 00:33:30.400
That that goes all the way.
00:33:30.640 --> 00:33:42.799
And i i I will say that many years ago already, various uh uh kind of uh tech leaders that I spoke to spoke about their RD organization as a software factory, which may or may not have been endearing to their teams.
00:33:42.960 --> 00:33:48.720
But they they are like, you know, it was still about like a repeatable process of software generation that goes end to end.
00:33:48.880 --> 00:33:59.599
Um I do think within that there will be sort of worlds of specialization for sort of difficult parts of it, for security, you know, for uh you know, for various aspects of sort of quality.
00:33:59.759 --> 00:34:04.559
And I think uh I think very much likely that sort of uh feature flags is its own domain.
00:34:04.720 --> 00:34:10.480
But it's just sort of interesting to think about scope product kind of in the uh in that sort of destination.
00:34:11.199 --> 00:34:20.800
Yeah, and something I want to emphasize is that um you don't want to lose the the forest for leaves, if that makes sense.
00:34:20.960 --> 00:34:30.000
Like you can definitely over-optimize for very narrow features and lose track of uh this isn't actually product anymore, and we've over-optimized.
00:34:30.159 --> 00:34:46.239
Uh uh an example I'll give is um you know, I you can realize that you've backed yourself into a niche product, that you have a very passionate bunch of loyalty, but you have not one in the broader market.
00:34:46.400 --> 00:35:07.360
So I worry that worry is perhaps too strong, but I I worry that with the rate of change that companies will back themselves into a quarter if they don't have some sort of level of judgment at product management about like, hey, we've overoptimized for you know 50 people in London who love this one feature, but the rest of the world is ambivalent.
00:35:07.679 --> 00:35:07.840
Yeah.
00:35:08.079 --> 00:35:08.719
Oh, I love that.
00:35:08.880 --> 00:35:11.840
So like feature flags are amazing, but they don't replace your judgment.
00:35:12.000 --> 00:35:15.440
You just they they just sort of enable you to kind
of apply it more wisely.
00:35:15.519 --> 00:35:16.960
Uh you still need to do that.
00:35:17.039 --> 00:35:25.679
And and I I agree you'll kind of be carry the risk of sort of hitting a local maxima, you know, for for sort of optimizing for it and not really having you have to have some vision.
00:35:25.840 --> 00:35:45.760
And I guess that comes back to like accountability, uh, because if you're gonna make a bold decision, even a medium bold decision, but one that is not sort of purely in the data, but rather your sort of analysis and sort of uh intuition on it, um there probably needs to be a human signing off on that type of change.
00:35:46.000 --> 00:35:58.960
Uh maybe if you're like very, very big and very, very confident and have a very, very fast learning cycle, uh, you know, whatever the TikTok algorithm, right, can probably like you know, make more of these wild bits, but uh Yeah.
00:35:59.280 --> 00:36:10.800
Yeah, I was thinking about it because um I was a product manager at Tripit, uh, which was uh at the time in the 2010s, the number one mobile app on the Apple iOS store.
00:36:11.039 --> 00:36:14.239
And it was a self-uh fulfilling prophecy.
00:36:14.320 --> 00:36:18.320
We were the number one app, so people would go to the app store and download us, we were the number one app.
00:36:18.400 --> 00:36:20.800
Yeah, which was kind of a great place to be.
00:36:21.039 --> 00:36:34.559
Um but the founders of the company, I wasn't a founder by the way, uh I was product, um, had made this really good decision that they focused on Apple early, and one of their big competitors focused on Blackberry.
00:36:35.119 --> 00:36:35.679
Yeah.
00:36:35.920 --> 00:36:39.920
And I don't know if that was something that an AI could ever decide.
00:36:40.400 --> 00:36:40.960
Make the choice.
00:36:41.440 --> 00:36:41.679
Yeah.
00:36:41.920 --> 00:36:58.559
Yeah, they they doubled and tripled down on we want to be the dominant app on Apple, and then Apple expanded versus their competitor who had this shrinking market because people at the time though there there was a time when BlackBerry had more market share, it was it it wasn't sustainable.
00:36:58.800 --> 00:36:59.199
Yeah, yeah.
00:37:00.079 --> 00:37:02.800
I think it would actually be difficult to buy a Blackberry right now.
00:37:03.199 --> 00:37:05.519
Uh I believe I I think I agree.
00:37:05.760 --> 00:37:14.719
I um I have this sort of uh blog post brewing in my head, which I'm probably never gonna write, that is that the uh greatest advantage of humanity is irrationality.
00:37:14.960 --> 00:37:26.000
Uh uh it's uh it's sort of this sort of desire uh to or like this willingness to say um no no, this is right, you know, against all the data.
00:37:26.159 --> 00:37:37.599
And this is like how all great companies, how all great you know, pieces of art have been created, how all uh uh so it's not really about whether an LLM or an agent would be able to conceive of this idea.
00:37:37.760 --> 00:37:39.360
They'd be able to conceive of all ideas.
00:37:39.519 --> 00:37:48.880
It's about the sort of like irrational dedication, which typically fails, but then for a subset of people succeeds, uh you know, subset of cases succeeds uh because of that.
00:37:48.960 --> 00:37:51.199
And and sometimes like actually changes reality, right?
00:37:51.280 --> 00:38:00.079
Like I think the work you've done with Launch Darkly, I'd like to think the work we've done at Sneak, you know, from a uh DevSec uh perspective, you know, they they don't just roll into the market.
00:38:00.320 --> 00:38:04.719
It's the sort of persistence that actually helps shape some aspect of the market.
00:38:05.519 --> 00:38:20.159
Yeah, so when I when I so I I started Launch Darkley in 2014 with my co-founder, uh who'd been at Elassi and I'd been at Tripit, and we both had this strong conviction that we did not want to have um an on-prem version.
00:38:21.519 --> 00:38:29.360
And it was because I had been the product manager and program manager for supporter platform matrix at vignette, and I was like, this is hell.
00:38:29.519 --> 00:38:30.639
Yeah, you had the scars.
00:38:31.280 --> 00:38:33.840
You're sorting out databases, you're sorting out app servers.
00:38:33.920 --> 00:38:35.119
I was like, don't want to do that again.
00:38:35.199 --> 00:38:36.000
We're SAS.
00:38:36.639 --> 00:38:38.880
And my co-founder was the same.
00:38:39.199 --> 00:38:41.760
And at the time it was very contrarian.
00:38:42.000 --> 00:38:48.559
Like I remember we were trying to sell to customers, and A, we were trying to sell them on fast releases, which some of them just didn't want at all.
00:38:49.920 --> 00:38:56.559
And then some of them were like, well, uh, you know, we would we would never use a vendor for this that we cannot install.
00:38:57.119 --> 00:39:05.119
Um, so but we held firm to that conviction that though we could have gotten more sales, it would have been a quagmire of pain.
00:39:05.280 --> 00:39:05.760
Yeah.
00:39:06.239 --> 00:39:08.079
Uh to have an installed version.
00:39:08.400 --> 00:39:08.559
Yeah.
00:39:08.719 --> 00:39:08.960
Yeah.
00:39:09.039 --> 00:39:11.599
We uh I wish we had that much conviction at Snake.
00:39:11.760 --> 00:39:14.400
We kind of went back and forth, back and forth on it.
00:39:14.480 --> 00:39:24.239
And it has uh but uh, you know, we we uh uh you know what I'm not even sure what the status is, but I'm pretty sure uh today there's on-prem components, but there isn't an on-prem system.
00:39:24.480 --> 00:39:30.960
Let me kind of shift us a little bit to uh add a degree of freedom, if you will, on it, uh, which is runtime agents.
00:39:31.039 --> 00:39:35.599
Uh you know, like you uh up until now we're talking about software that runs in runtime.
00:39:35.760 --> 00:39:38.320
Yes, it is modifying faster.
00:39:38.719 --> 00:39:48.320
Uh and you know, by using feature flags, you can try out capabilities within there and you know uh
allow it to change and learn even faster.
00:39:48.480 --> 00:39:51.679
Um is there such a thing as feature flags for agents?
00:39:51.840 --> 00:40:00.000
Like when you're what as a as a portion of your product becomes instead of software, you deliver a bunch of that value in the form of agents.
00:40:00.159 --> 00:40:04.559
Like what is the equivalent of feature flags uh in that world?
00:40:04.960 --> 00:40:08.719
So I feel like you're uh feeding me marketing lines.
00:40:08.800 --> 00:40:11.760
So um we do have a product called agent control.
00:40:12.800 --> 00:40:32.400
Um and it is where you can put guardrails around an agent and put in parameters so that if your models drift, uh it can either shut off the agent completely or just dial it down or so that you can have the ability to control an agent in production without a redeploy.
00:40:33.280 --> 00:40:47.039
And and this will be like how how is that different to just sort of the core management of the agent or like how how people run their agent in like with without LodgeDarkly, right?
00:40:47.920 --> 00:40:56.159
Uh so we give you a layer where you can easily switch models or even switch the prompts without having to do a redeploy.
00:40:56.239 --> 00:40:57.519
So you can do it at runtime.
00:40:57.920 --> 00:41:06.079
And it's again like one of these uh you know, folks in London get one sort of uh prompt versus you know people elsewhere getting another and sort of seeing what happens.
00:41:06.639 --> 00:41:09.920
Uh correct, or um monitoring for errors.
00:41:10.079 --> 00:41:25.199
Um so if you have, for example, a chatbot that suddenly is getting a lot of customer complaints, um, you can dial it back or switch Swiss models, or uh if it's suddenly burning a lot of tokens, uh you can switch between models on the fly.
00:41:25.440 --> 00:41:25.760
Yeah.
00:41:26.159 --> 00:41:30.960
Who do you think is the primary user for the sort of the runtime control of the agent?
00:41:31.039 --> 00:41:32.159
And is that is that different?
00:41:32.239 --> 00:41:40.719
Is it still about sort of like managing product functionality that's available to people, or is it about actually developing the sort of the prompts and the choices of models?
00:41:40.880 --> 00:41:50.079
Like almost the more kind of implementation, it's the same goal, but it's a different um uh it's a it's a change of the instrumentation that makes it happen.
00:41:50.639 --> 00:41:53.760
I think people are still figuring out what agents are.
00:41:54.639 --> 00:42:02.880
Um so I think within LaunchDarkly, we have both very ephemeral agents and long-running agents.
00:42:03.119 --> 00:42:10.079
Um I'll use an ephemeral agent as an example of you know, we're working on a project to redevelop part of the code base.
00:42:10.480 --> 00:42:17.199
We spin up uh you know 20, 30 agents to work on it, and they are specific to this specific task.
00:42:17.599 --> 00:42:20.639
Um we have other, I'd say, kind of long-running agents.
00:42:20.719 --> 00:42:27.920
Uh, like, so for example, we have an agent called Vega, which is constantly doing flag cleanup on everything that we produce.
00:42:28.079 --> 00:42:39.039
We, like everybody else, uh didn't always practice what we preached, and we had some stale flags, you know, some dating back, you know, to when we were still a startup at Heavy Bit.
00:42:39.360 --> 00:42:50.639
Um, in this agentic world, we now have agents that just go around, examine every flag and say, is this still in use, is it still needed, and if not, uh safely delete it.
00:42:50.880 --> 00:42:52.639
And and that's just really fun.
00:42:52.880 --> 00:42:57.920
I I um I I think that's a real best practice that AI has made a lot easier.
00:42:58.159 --> 00:43:08.639
So I think these are actually pretty different use cases, and I would say I think right now I think the industry as a whole is figuring out what the right word what what the right use cases are.
00:43:08.880 --> 00:43:15.119
Um to use subconsumer use cases, uh uh it's funny because I come from a travel background because I was a trip it.
00:43:15.440 --> 00:43:20.559
Right now there's a lot of chatter about like uh an agent that will book your next vacation.
00:43:20.880 --> 00:43:22.559
Okay, that's a one-time use case.
00:43:22.719 --> 00:43:39.599
Like if I'm trying to figure out how to go to Atlanta to see my aunt, that's a specifically defined project versus a long-running agent of uh, you know, look at New York Times cooking recipes every week and buy the groceries so I can make the most top three popular ones.
00:43:39.920 --> 00:43:40.960
Yeah, yeah.
00:43:41.440 --> 00:43:42.000
Interesting.
00:43:42.159 --> 00:43:42.400
Okay.
00:43:42.480 --> 00:43:59.679
So even as you think, so I guess in that context, when you do that type of runtime controls, you come back to you need a person, like they need to figure out what is the question they're asking when they're making a modification, uh, because the sort of the mental model of what is it trying to do, what is the context in which it runs, uh, once again.
00:44:00.159 --> 00:44:12.079
Again, what is what is success, not even for the product as a whole, but for this experiment uh that you're doing here, all of those things need to be pretty clear in your in your mind and then subsequently in the instructions.
00:44:12.480 --> 00:44:16.000
Yeah, and to go back to the the cooking agent, because I think it's a good example.
00:44:16.159 --> 00:44:24.239
If the agent went rogue and said instead of the top three recipes, I'm gonna choose the top ten recipe because that's better, it's like, well, that's actually way too much food for me.
00:44:24.400 --> 00:44:24.960
Yeah.
00:44:26.719 --> 00:44:29.280
Um so uh I think we're sort of running out of time here.
00:44:29.679 --> 00:44:31.920
I've got like five more questions on it, but I'll I'll ask one.
00:44:32.159 --> 00:44:35.679
Or maybe I'll ask one of those questions that is five in one, and
we'll sort of find out.
00:44:35.840 --> 00:44:44.639
Um so you you've built, you know, you you you found it non-starkly even before I founded Sneak, you know, like a year before on it, you know, you've been you've just sort of built in the cloud era.
00:44:44.960 --> 00:44:56.239
Uh you you had some time away from the CEO seat, you know, for a couple of years, you came back to uh to kind of I don't want to say reinvent, but like uh adapt the company to the uh the AI uh era.
00:44:56.719 --> 00:45:07.679
Um what would you say is different about running a company in you know a decade ago, you know, and like what is needed to make things successful versus today?
00:45:08.320 --> 00:45:11.119
Oh gosh, I mean I could talk about that for hours.
00:45:11.360 --> 00:45:17.360
Um the company ten years ago, I mean, we were a tiny startup.
00:45:17.519 --> 00:45:26.320
You know, we were you know eight people at HeavyBit, the uh DevTool incubator, and now we're 600 people worldwide.
00:45:26.800 --> 00:45:31.840
Um and we have thousands of happy customers that I'm proud of.
00:45:32.159 --> 00:45:41.760
I'd say the big change right now is that a lot of established software principles are up for grabs, but yet still the same.
00:45:42.159 --> 00:45:51.440
So, like some of the things we've talked about in terms of you know moving the bottleneck within the software development lifecycle is it's very different.
00:45:51.679 --> 00:46:01.440
Um, but I think the core belief in that our job as a company is to deliver value to our customers and to empower our employees hasn't changed at all.
00:46:01.679 --> 00:46:01.840
Yeah.
00:46:02.239 --> 00:46:16.400
Do you think is is there something different you look for in uh in a team in a higher, maybe don't sort of look quite at the sort of the 14 person, but try to sort of imagine a 600-person company a decade ago or like maybe even somewhere along the way versus now?
00:46:16.639 --> 00:46:22.559
Like are there different aptitudes that you look for in a developer on the team uh now versus uh before?
00:46:23.199 --> 00:46:33.599
So uh we we just uh republished our values, and a core one that's been constant is the ability to learn and grow.
00:46:34.239 --> 00:46:42.719
Um so I really encourage the team to always think about how they could be doing things different and better.
00:46:43.519 --> 00:46:46.559
And I think AI has just made that even more fun.
00:46:47.039 --> 00:46:52.239
Uh, you know, so we were having a discussion about, you know, should we encourage people to take smart bets?
00:46:52.480 --> 00:46:55.840
And I said I would like to encourage them to take some dumb bets too.
00:46:56.719 --> 00:47:00.559
You know, if you're only taking smart bets, you're not taking enough bets.
00:47:00.719 --> 00:47:12.719
Um if you're taking stuff where you're like, hey, I'm not really sure if this is gonna work out, like that that's the edge where I want people to be, and not in a not in a silly way of like, hey, I know that if I push this button, it's gonna take down production.
00:47:12.800 --> 00:47:13.519
Please don't do that.
00:47:13.679 --> 00:47:14.000
Yeah.
00:47:14.239 --> 00:47:20.559
Um but like hey, this feature with AI is a lot easier to build.
00:47:21.039 --> 00:47:29.280
And so we can cut off some of the debate about whether or not it's worth building because it's actually faster to just build it than have that debate.
00:47:29.679 --> 00:47:31.840
Um that that's super fun.
00:47:32.559 --> 00:47:33.840
Yeah, yeah, I love that.
00:47:33.920 --> 00:47:34.639
And I agree with that.
00:47:34.719 --> 00:47:47.920
Like uh adaptability is the name of the game, and uh, I guess a part of it is even how both what you build and how you build uh is all but guaranteed to be different in a year's time, uh, which is not the reality we had before.
00:47:48.000 --> 00:47:54.320
You know, like uh we had uh we had things that would change over a decade, not over a quarter.
00:47:54.559 --> 00:47:56.800
Uh but uh you know.
00:47:57.119 --> 00:48:04.800
Yeah, and I think even about how we're talking about AI in the spring, uh now it's October, and we're we're using it differently.
00:48:04.960 --> 00:48:10.880
Um the frontier models have come out with different things, we've changed how we approach it, and we know that this will continue to change.
00:48:10.960 --> 00:48:13.840
And some people find this terrifying.
00:48:14.239 --> 00:48:15.920
I find it really exhilarating.
00:48:16.000 --> 00:48:16.880
I'm like, this is cool.
00:48:17.119 --> 00:48:32.960
Yeah, yeah, you're sort of uh uh outcome and kind of creativity oriented, and it's interesting to think about the people that are more craft-oriented, they feel like someone's taking away their craft, uh, and those who are more like problem-solving or like outcome-oriented, uh, I think get more satisfaction out of it.
00:48:33.599 --> 00:48:45.360
To go back to the factory model, I I think of it as moving from perhaps a home workshop where you have some limited tools to full makerspace where you have access to all of a sudden all these things that can help you build.
00:48:45.679 --> 00:48:45.920
Yeah.
00:48:46.079 --> 00:48:47.760
Yeah, lots of excitement in that.
00:48:48.079 --> 00:48:49.840
Edith, this has been a great conversation.
00:48:49.920 --> 00:48:51.519
Thanks for for the time here.
00:48:51.760 --> 00:48:57.840
Excited to see how you uh continue to sort of build and uh and adapt, and uh, you know, so much more to build here.
00:48:57.920 --> 00:48:59.199
So thanks for tuning in.
00:48:59.519 --> 00:49:00.320
Yeah, thank you.
00:49:00.400 --> 00:49:01.679
It's great to spend some time with you.
00:49:02.239 --> 00:49:02.400
Cool.
00:49:02.559 --> 00:49:05.760
And thanks everyone for listening in, and I hope you join us for the next one.
00:49:06.079 --> 00:49:10.880
The AI Native Dev is brought to you by Tesla, the package manager for skills and context.
00:49:10.960 --> 00:49:14.000
Your hosts are Guy Pajani and me, Simon Maple.
00:49:14.079 --> 00:49:15.760
Our producer is Tom Dowler.
00:49:15.920 --> 00:49:19.199
The AI Native Dev is not just a podcast, it's a community.
00:49:19.280 --> 00:49:22.719
And we host monthly meetups at the Tesla offices in central London.
00:49:22.880 --> 00:49:26.320
Visit Tesla.io forward slash community to learn more.
00:49:26.559 --> 00:49:27.920
I hope to see you there.