WEBVTT
00:00:00.080 --> 00:00:04.480
If an AI agent is going and writing code for you, do you have any idea whether it's secure or not?
00:00:04.639 --> 00:00:09.759
Or are you just relying on the same model that went and created the code to tell you that it's secure?
00:00:23.600 --> 00:00:24.719
Welcome to Future Proof.
00:00:24.800 --> 00:00:27.199
I'm Prakash Chandran, CEO of Xano.
00:00:27.519 --> 00:00:37.200
Today I'm joined by Tim Olshansky, co-founder of Fencer, a security platform that replaces the patchwork of scanners, monitors, and spreadsheets most startups cobble together.
00:00:37.439 --> 00:00:40.960
I know from personal experience, and actually fixes what it finds.
00:00:41.119 --> 00:00:57.439
Tim is a serial founder and engineering leader who's been CTO of a publicly traded company, scaled engineering teams to over 200, and built products for brands like Chipotle and Burger King, as well as the DoD and three letter agencies that will go unspecified.
00:00:57.679 --> 00:01:06.000
He's seen firsthand what happens when compliance becomes a prerequisite for growth and what breaks when teams skip the security foundation to ship faster.
00:01:06.239 --> 00:01:12.879
There's a question Tim's been asking that I think every AI builder needs to hear, and that is who approved this change?
00:01:13.120 --> 00:01:23.439
As agents start writing code, opening PRs, and sometimes merging changes on their own, the chain of human accountability that every compliance framework was built around is quietly breaking.
00:01:23.680 --> 00:01:27.760
This is a conversation about what that means and what to do about it.
00:01:28.000 --> 00:01:30.079
Tim, thank you so much for being here.
00:01:30.159 --> 00:01:31.439
It's really great to have you.
00:01:31.840 --> 00:01:33.439
Hi, Prakash, and hi everybody.
00:01:33.519 --> 00:01:34.640
It's uh it's nice to be here.
00:01:34.719 --> 00:01:36.319
It's been a it's been a while.
00:01:36.719 --> 00:01:37.760
It has been a while.
00:01:37.920 --> 00:01:42.640
And fun fact, if I may mention, you are an investor uh in Xano.
00:01:42.959 --> 00:01:54.000
Just kind of thank you so much for not only your belief in us, but it also just kind of goes to show like the technologists that you are not only contributing to the space, but investing in platforms like ours.
00:01:54.799 --> 00:01:56.239
Yeah, I love what you're building.
00:01:56.400 --> 00:01:56.719
Sorry.
00:01:56.879 --> 00:01:58.719
It was it wasn't a hard choice.
00:01:59.120 --> 00:01:59.840
Awesome.
00:02:00.000 --> 00:02:02.319
So I wanted to start with a little bit of background.
00:02:02.400 --> 00:02:15.439
So, you know, you've co-founded, as I mentioned in the intro, you know, multiple companies, been the CTO of a public company, scaled engineering teams, and then you decided to build something around security, a security platform for startups.
00:02:15.680 --> 00:02:21.759
So I'm curious like what you saw that made you feel like the security problem was worth solving.
00:02:22.159 --> 00:02:28.319
Yeah, it's uh poignant, I think, because everybody is now thinking about building their own software and building their own products.
00:02:28.400 --> 00:02:32.240
And so um, hopefully, security is something that people are thinking about a lot more.
00:02:32.400 --> 00:02:47.759
But over the course of my career, it was one of those things that I became responsible for as the CTO, technical co-founder, head of engineering, uh, software engineer on a team that I often felt was not being done well at the earlier stages.
00:02:47.919 --> 00:02:58.800
And then when you know I was a part of larger organizations, the level of bureaucracy and difficulty it was to be able to use these tools and use them well was just crazy.
00:02:58.960 --> 00:03:04.080
So we we we set out on this mission um to build something effectively to solve our own problem.
00:03:04.240 --> 00:03:09.039
Uh, myself and my co-founder had felt the pain uh and we wanted to make this easier.
00:03:09.360 --> 00:03:17.759
I had, you know, obviously had to go and implement a lot of these things myself uh in the the pre fencer days.
00:03:17.919 --> 00:03:20.319
And so I think we had a great idea for what we wanted to do.
00:03:20.400 --> 00:03:25.680
And then at the same time, uh knowing the pain ourselves, we'd spoken to lots of different companies.
00:03:25.759 --> 00:03:34.479
I actually spoke to Sean at Zano and and and others uh across the industry about some of their challenges in the early days, and same message.
00:03:34.639 --> 00:03:36.159
Like this is this is difficult.
00:03:36.319 --> 00:03:46.000
Uh so we we saw that problem combined with the fact that AI was writing a lot of code, knowing that more code is going to be written in the future, more products are gonna get built.
00:03:46.080 --> 00:03:48.000
This felt like a great space to build in.
00:03:49.120 --> 00:03:50.560
Yeah, that makes a lot of sense.
00:03:50.719 --> 00:04:03.759
In in the conversations that you've been having, I feel like you started or you kind of saw early on that this security gap was something even before AI and AI building has had taken off, like it really, really has today.
00:04:03.919 --> 00:04:08.800
Um, it only feels actually now like that that has been exacerbated.
00:04:08.879 --> 00:04:13.680
You know, like there's so much that people rely on now uh in terms of AI building.
00:04:13.840 --> 00:04:21.839
And I know that one of the in doing research for this, one of the things that you wrote is, you know, AI tools are great for building features, but they're just terrible at security.
00:04:22.000 --> 00:04:23.519
I wonder if you could just speak to that.
00:04:23.839 --> 00:04:24.160
Yeah.
00:04:24.319 --> 00:04:37.680
Um, you know, it's funny, just as like a little bit of history, you know, 10 years ago, if not more, uh, it was pretty rare to see a company less than 50 employees go through the SOC2 certification process.
00:04:38.000 --> 00:04:49.040
Uh and a lot of new platforms got built, you know, Vanta, Drata, others uh that I'm sure folks that are listening, watching have probably heard of, um, have made that easier.
00:04:49.199 --> 00:05:01.439
So everyone is going and getting these compliance certifications that come with security requirements, who are perhaps selling SaaS products uh or just want to have this checkbox against their business.
00:05:01.600 --> 00:05:08.639
Uh they're able to get them at a much earlier stage, but they're not actually doing half the things that they're saying or promising that they're going to be doing.
00:05:09.680 --> 00:05:28.000
And so that's that's like the kind of industry has pushed this uh compliance standard to people, compliance certification to people at earlier and earlier stages, but they aren't themselves equipped with the processes and understanding of what to go do, they just delegate to some tool to go figure that out for them.
00:05:28.079 --> 00:05:36.639
And so that's that's a big part of what we're trying to help with is actually make security accessible and something that you can actually accomplish.
00:05:36.879 --> 00:05:44.079
And so when you say that you have uh you know code review processes and approval processes and all these other things, do you really?
00:05:44.240 --> 00:05:45.839
Do you know what's actually going on?
00:05:46.000 --> 00:05:59.519
Uh are you just checking the box if uh if an AI agent is going and writing code for you uh and it produces a pull request or a merge request or you know, some sort of change set for you to look at.
00:05:59.680 --> 00:06:01.680
Do you have any idea whether it's secure or not?
00:06:01.839 --> 00:06:06.480
Or are you just relying on the same model that went and created the code to tell you that it's secure?
00:06:06.800 --> 00:06:11.279
It's like asking a junior engineer to tell you whether they think the code they wrote is secure.
00:06:11.439 --> 00:06:13.439
And so that's that's kind of the premise, right?
00:06:13.600 --> 00:06:16.399
The models are great, but and they're only getting better.
00:06:16.560 --> 00:06:22.160
But even still, like you still want a second set of eyes on many changes as much as possible.
00:06:22.560 --> 00:06:34.959
Yeah, I think also no matter how good the models get, I think that security, compliance, and how it relates to your business and the rules around your business will always be something that needs to that needs oversight.
00:06:35.600 --> 00:06:44.079
Um, so I would love for you to talk a little bit about Fencer in the context of actually security and compliance and how people should be thinking about it.
00:06:44.240 --> 00:06:53.279
Who are the types of people or what types of applications need to start thinking about security very early on, and then introduce a little bit about what fencer does.
00:06:53.600 --> 00:06:54.000
Yeah.
00:06:54.160 --> 00:07:08.800
You know, it's timing's interesting because if uh if mythos and all these new LLMs and models and everything else get released to the public in the next week and all of a sudden everything's hacked all the time, and then you know, we all need to be dealing with security.
00:07:09.040 --> 00:07:13.600
I don't know what our banks are gonna do, you know, uh doom and gloom, etc.
00:07:13.920 --> 00:07:17.920
Although I think it's a good thing because all of these bugs that have been there forever will finally get fixed.
00:07:18.079 --> 00:07:25.920
But um, you know, really what we set out to do is to make this easier for companies or businesses that don't have a security team, right?
00:07:26.000 --> 00:07:28.319
And and probably don't even have a head of security.
00:07:28.399 --> 00:07:37.759
It's some individual that spends maybe 20% of their time per month maybe uh dealing with security issues.
00:07:38.000 --> 00:07:41.439
And that usually means companies with less than 200 employees, right?
00:07:41.519 --> 00:07:44.480
Um often smaller than that, less than 20 employees.
00:07:44.800 --> 00:07:50.000
And that's the sweet spot, I think, to go and set up a good security foundation.
00:07:50.079 --> 00:08:17.759
Uh, and that's uh you know, a number of different things that I could go on for hours, but it ranges from just having decent uh awareness of where you may have security holes with scanning tools and and and what have you, uh doing penetration testing if you're building software, uh, and if you're not building software for external consumption, then at least being aware of where you may have holes uh for your own internal consumption is pr is pretty useful.
00:08:18.000 --> 00:08:29.759
Uh because while you may not sell software, the business that you're in may be a great target for an opportunistic hacker in Eastern Europe or somewhere else, um, not just a nation state.
00:08:29.920 --> 00:08:34.879
And uh and the things that you do at that stage are pretty straightforward uh and not that expensive.
00:08:35.919 --> 00:08:37.120
Kind of like brushing your teeth, right?
00:08:37.200 --> 00:08:38.080
It's good hygiene.
00:08:38.240 --> 00:08:40.879
You do it a little bit if you do it every day for two minutes.
00:08:41.120 --> 00:08:42.559
You don't have to go to the dentist.
00:08:42.639 --> 00:08:49.840
It's kind of a similar premise, uh, albeit the consequences are probably far worse than a filling, um, if you get it wrong.
00:08:50.559 --> 00:08:51.120
Right.
00:08:52.080 --> 00:09:04.639
And so I'm curious, like in the context of fencer, like I mean, that totally makes sense in terms of like how you need to be thinking about it, especially in the in the world that we're living in now with all these new releases.
00:09:04.799 --> 00:09:12.960
Um, when you started thinking about fencer, when we talk about security and compliance, that's a really broad surface area.
00:09:13.039 --> 00:09:13.200
Yeah.
00:09:13.360 --> 00:09:27.440
So, how do you think about compartmentalizing the different parts of security and how people can and how people can, for example, leverage a tool like Fencer to start becoming secure from day from day one, really?
00:09:28.080 --> 00:09:34.879
So there's there's a lot of different areas that need to be secure uh across a business, right?
00:09:35.039 --> 00:09:41.519
And while we can't personally cover all of them uh as a business, there's lots of other great players in the space.
00:09:41.600 --> 00:09:53.519
The way that we generally think about it is for a company in that kind of category that I described, 200 or fewer, uh, you have software that you write, which means code.
00:09:53.759 --> 00:09:56.240
You have infrastructure that you run on, right?
00:09:56.399 --> 00:10:05.519
Whether your own, I mean, I don't see many companies going and buying racks and putting them in a data center, uh, they're more likely to be building on top of a platform like Xano, right?
00:10:05.600 --> 00:10:09.679
And so they're they're building their tools and applications on on top of others.
00:10:09.919 --> 00:10:16.080
Um they they have networks that they're responsible for, and they may not realize it.
00:10:16.159 --> 00:10:26.960
You know, just because you're building on top of a service like Xano, you know, implicitly there is a network that your systems communicate with that you need to be aware of.
00:10:27.120 --> 00:10:28.320
You have your own user accounts.
00:10:28.399 --> 00:10:38.000
You know, if you're part of a company, you're there's gonna be lots of different identities that are interacting, and not just humans, you've got agents, you've got applications, you've got bots, uh, et cetera.
00:10:38.320 --> 00:10:50.080
Um and and beyond just being aware of those uh perhaps security holes or misconfigurations, uh, you also need the observability or the monitoring of those things, right?
00:10:50.240 --> 00:11:09.279
So ingesting or collecting events from lots of different data sources to help you flag and identify when you know somebody has gone and used some automated tool against your app or against your system to see if any of those uh if those holes exist.
00:11:09.600 --> 00:11:11.679
And and I think these are sort of like baseline things.
00:11:11.759 --> 00:11:16.799
And I haven't even touched on your own laptop as a potential issue or your phone or something like that, right?
00:11:16.879 --> 00:11:22.399
Because that as well is uh is another common source of uh security issue.
00:11:22.879 --> 00:11:30.159
You know, the the most common place where companies get hacked from is just somebody messed up with their uh username and password.
00:11:30.320 --> 00:11:39.440
They reused a password on a some SaaS site that wasn't doing a good job, and now their password has been exposed, or they're using a common password that someone was able to brute force.
00:11:39.519 --> 00:11:46.960
So I think it's nearly a quarter of all hacks start with uh somebody's bad password or bad username and password.
00:11:47.200 --> 00:11:50.559
And so identity is a big piece of it, uh, and we help monitor that.
00:11:50.720 --> 00:12:00.080
Um and and so there's you know those big areas that I mentioned, the five or six that I mentioned, that we sort of have pulled together into one platform and give you much greater control and visibility.
00:12:00.320 --> 00:12:13.039
Uh and then if you're the type of business that needs to prove to others that you're secure, we then automate all of the evidence collection and all of the bits and pieces to go and show people, hey, we actually take security seriously.
00:12:13.120 --> 00:12:16.559
Here are all the things we're doing, here are the things we found, and here are the things that we fixed.
00:12:17.120 --> 00:12:19.039
Not to mention helping you fix it.
00:12:19.360 --> 00:12:29.600
Yeah, it's really it's really interesting because I feel like um a lot of the other tools uh in the land or compliance tools in the landscape that you mentioned are very much like checklist driven.
00:12:29.679 --> 00:12:32.720
Like you need to do all of these things to get this certification.
00:12:32.960 --> 00:12:45.759
Where I think um what's what I've always liked about Fencer, and I think really the way that people need to be approaching building applications today is a very like um behavioral or security first.
00:12:45.919 --> 00:12:56.159
Like you're doing the the tool is actually helping you remain secure and giving you auditability and observability over thing, and actually the compliance certification is an after-effect of that.
00:12:56.240 --> 00:12:56.639
That's right.
00:12:56.960 --> 00:12:59.279
Um so that that that makes a lot of sense to me.
00:12:59.519 --> 00:13:06.639
You know, you said something interesting around like how there's kind of like a compliance supply chain or they're like a system.
00:13:07.039 --> 00:13:12.480
And you know, relevant, obviously, we had the uh light LLM supply chain attack.
00:13:12.559 --> 00:13:17.200
Um, there are other examples of that kind of surface that you're not alone.
00:13:17.279 --> 00:13:20.639
You don't, when you install one thing, you're not just getting that thing.
00:13:20.799 --> 00:13:39.759
I was wondering if you could help explain to the audience um, maybe at a higher level, about supply chain risks, uh compliance chain risks, and what they need to be aware of as they're depending on uh some of these foundational, sometimes open source products and and and thereabout.
00:13:40.320 --> 00:13:40.720
Yeah.
00:13:40.960 --> 00:13:50.320
It's you know, I think um, and I I don't want I don't want to uh make this sound too simple, but just like in the physical world that there's a supply chain of goods, right?
00:13:50.399 --> 00:13:58.000
Like there's no there's no car today where every individual component gets made by Ford, right, or GM or a single company.
00:13:58.320 --> 00:14:06.879
Uh there's many different businesses that go and produce these pieces that get brought together to make a car the same way software is made today.
00:14:07.039 --> 00:14:13.759
It's just the main difference in the digital world is the delivery mechanism is over the internet, so it's basically lightning fast.
00:14:14.240 --> 00:14:24.559
Uh and that there are uh selfless people who will happily give up their own time to create software for everybody's benefit and expect no financial reward.
00:14:24.639 --> 00:14:26.000
And so that's open source software.
00:14:26.159 --> 00:14:41.440
And so a large, huge portion of uh the internet today and a lot of what we have built all of our software on relies on these open source software projects, which are the un effectively the fundamental components of a lot of the digital supply chain uh that we rely on today.
00:14:41.759 --> 00:14:56.080
And they're they're interesting because these tools, uh, these libraries and these pieces of software are like our light LLM is an example of Trivi, another recent one, uh that was actually the one that caused Lite LLM to be hacked.
00:14:56.159 --> 00:15:05.919
And then there was Axios, which is a popular uh JavaScript or TypeScript library uh they're used for building a lot of web applications today and used in a lot of different products.
00:15:06.159 --> 00:15:17.120
Um they're all open source projects, and so if a hacker gains access to them, they inadvertently gain access to lots of commercial software and lots of commercial products.
00:15:17.360 --> 00:15:30.960
And so what these threat actors, uh, these hackers have figured out is that they don't need to try and catch a break into these private organizations because they're actually doing something around security.
00:15:31.120 --> 00:15:49.360
They're now finding that actually the open source, you know, uh effectively volunteer community is a great place to go and uh exploit their limited resources and security gaps to be able to then gain access to many, many more um organizations and take advantage of them.
00:15:49.519 --> 00:15:53.360
So that's only going to get worse and faster over time.
00:15:53.840 --> 00:16:10.720
And I think the kind of relating to your previous point about fixing things for people, you know, you might not know about these problems, and you know about them when somebody kind of exploits them and they make a public statement, which is what the uh organization, you know, the hacking group, uh did.
00:16:10.960 --> 00:16:14.159
They tweeted about it and showed everyone how great they are.
00:16:14.399 --> 00:16:18.320
Um what's important now is that as soon as you find it, you fix it.
00:16:18.879 --> 00:16:21.279
Because not everyone can be exploited instantly.
00:16:21.440 --> 00:16:31.840
And so if you are the lucky many who aren't being regularly exploited, uh then as soon as you know, being able to respond and react and fix those things is really, really important.
00:16:32.159 --> 00:16:33.440
I think that's sort of the future.
00:16:33.600 --> 00:16:40.879
As we get better at finding these things, we will actually lag in our ability to fix these things, and that's always been the biggest problem.
00:16:41.200 --> 00:16:44.240
Uh still is the biggest problem, I think will be the biggest problem for a while.
00:16:44.399 --> 00:16:52.879
Like, how can you quickly go and upgrade your supply chain components uh to be on the safe versions so you yourself aren't at risk?
00:16:54.159 --> 00:16:56.639
Yeah, and that flows through everywhere and proving it, right?
00:16:56.720 --> 00:16:58.639
So to the compliance piece as well.
00:16:58.960 --> 00:16:59.600
Totally.
00:16:59.840 --> 00:17:16.400
And like in I just want to highlight something that you said that I think is really important because as AI tooling and AI building adoption um accelerates and it's been accelerating exponentially, um, so does the dependency surface area.
00:17:16.640 --> 00:17:23.359
And that dependency surface area, in many cases, people don't even realize the dependencies that they're leveraging, right?
00:17:23.440 --> 00:17:25.759
Their AI is making the decisions for them.
00:17:25.920 --> 00:17:28.400
Um there's a lot of dependencies that are installed.
00:17:28.640 --> 00:17:39.359
And so, you know, it's it's not only just like the security first mentality, is just understanding what is it that you have and also the way that it's being pulled in, right?
00:17:39.440 --> 00:17:40.480
Is it like the latest?
00:17:40.559 --> 00:17:42.160
Is it always fetching the latest?
00:17:42.400 --> 00:17:44.160
What is the secure way to proceed?
00:17:44.319 --> 00:17:49.200
And I think that's a question that people, number one, they I don't know that they know how to begin.
00:17:49.279 --> 00:17:55.680
Like real builders today, or if I'm an application development leader uh managing my team, like where do I begin?
00:17:55.839 --> 00:17:57.839
What is the safe way to do this?
00:17:58.000 --> 00:18:02.240
Because to say not to leverage some of these open source projects seemed crazy.
00:18:02.480 --> 00:18:04.000
So, what is the middle ground?
00:18:04.079 --> 00:18:08.880
Like, how what would you recommend to people thinking about this and they want to do things securely?
00:18:09.200 --> 00:18:20.079
Yeah, it's it's you know, getting started is actually a lot easier now than it was probably a month ago, uh, thanks to some of these uh very public disclosures of uh of these hacks.
00:18:20.559 --> 00:18:25.039
Yeah, it's it's actually I think um straightforward.
00:18:25.279 --> 00:18:31.839
You know, we at Fencer have published blogs on the topic, um, just to be a shameless shill uh for a moment.
00:18:32.160 --> 00:18:34.480
But the I think I'll I'll allow it.
00:18:34.720 --> 00:18:35.519
You'll allow it.
00:18:35.680 --> 00:18:39.119
Um, it's use use a package manager, right?
00:18:39.359 --> 00:19:00.400
I think basically everyone does these days, but you know, use a package manager and don't just use a package manager blindly, but use a package manager with a lock file or some other manifest that you can be explicit in specifying which version of your dependencies should be installed at any moment and which version of your dependencies when you run your application will be present and available.
00:19:00.960 --> 00:19:21.200
Uh and ensure that you're using a package manager that will effectively create a uh a hash or you know, basically an integrity value, a value with a lot of integrity that you can trust that the underlying software uh that you're importing has not been modified in any way.
00:19:21.359 --> 00:19:32.480
Um there's in every ecosystem that exists in the Java world, in the JavaScript world, in the uh Python world, Ruby, Go, you name it, there's there's great package managers.
00:19:32.720 --> 00:19:35.279
Um the that's number one.
00:19:35.359 --> 00:19:37.440
So use a good package manager with a lock file.
00:19:37.519 --> 00:19:39.519
Uh use their cooldown feature.
00:19:39.680 --> 00:19:46.720
Nearly all of them at this point have a way for you to say, hey, don't use the latest thing.
00:19:47.039 --> 00:19:47.680
Wait.
00:19:48.079 --> 00:19:55.359
If something is new today when I try to install a new version, uh don't use it unless it's at least seven days old or three days old or whatever.
00:19:55.839 --> 00:19:56.880
Specify a cooldown.
00:19:56.960 --> 00:20:05.599
Uh and I generally recommend to people at least two days, uh, but probably beyond five or six days, six days, it's diminishing returns.
00:20:05.920 --> 00:20:15.759
You know, most uh most open source software, I'm I'd say 95% will have a fix out within two days.
00:20:16.000 --> 00:20:23.119
Um and so you know if if you're risk-averse like we are, we're a security vendor, we've got it set to a week.
00:20:23.279 --> 00:20:33.440
Um and the most common time for hacks to occur is a Friday night uh because people are not gonna check some automated process until Monday.
00:20:33.599 --> 00:20:35.200
And so give yourself some time.
00:20:35.279 --> 00:20:39.279
Um and then I can obviously keep going on, but that's some of the simplest things that you can go do.
00:20:39.519 --> 00:20:47.039
No, I think that's I think that's really that's really helpful just as a foundation for how people should be thinking about it.
00:20:47.200 --> 00:20:52.799
So, first and foremost, just understand some of the dependencies that your application, your AI application is generating.
00:20:52.960 --> 00:20:58.000
And then uh secondly, yeah, use some of those best practices and uh especially that cooldown period.
00:20:58.079 --> 00:20:58.880
That makes a lot of sense.
00:20:59.119 --> 00:21:03.839
This world that we're moving into, where agents are, for example, doing more of the work.
00:21:03.920 --> 00:21:11.279
And one of the things that you have been talking about that teams aren't necessarily confronting is that question that I mentioned up at the top.
00:21:11.440 --> 00:21:12.960
Who approved this change?
00:21:13.440 --> 00:21:19.920
Um, I'd love for you to walk us through what breaks in the compliance chain when agents start writing and merging code.
00:21:20.000 --> 00:21:21.440
Like how do you think about this?
00:21:21.759 --> 00:21:33.279
Yeah, I I can't remember who it was published a blog post recently, you know, big tech company about how something like 30% of their changes are now being reviewed, approved, and merged by AI.
00:21:33.440 --> 00:21:45.200
And so as as a lot of engineering practices have kind of come from the bigger companies to the smaller companies, I imagine that that's coming for everyone if they're not already doing it in certain cases.
00:21:45.440 --> 00:21:49.680
Certainly solo builders are probably already doing that, but they're on their own, so that makes sense.
00:21:49.920 --> 00:21:59.839
Um it's it's a it's an interesting one because general security best practices said that the people who the person who wrote the code should not be the one to approve the code.
00:22:00.960 --> 00:22:01.279
Right.
00:22:01.440 --> 00:22:04.799
So you in the pre-AI world, you'd have a teammate take a look at it.
00:22:04.880 --> 00:22:08.400
And tools like GitHub and GitLab even make that really easy for you.
00:22:08.559 --> 00:22:11.759
You'd have a pull request, you take a look at the change, looks good.
00:22:12.079 --> 00:22:20.559
Click a button, it's approved, and you know, merged, and whatever automation you have in place to uh update your systems and servers uh would play out.
00:22:21.200 --> 00:22:27.119
In AI world, you know, again, I I believe in sort of trust but verify.
00:22:27.599 --> 00:22:37.519
Uh and if you're if you're confident that the tool is doing the right thing, right, you've let your AI systems run wild and approve and merge things for you.
00:22:37.680 --> 00:22:47.119
I'm hoping that you have CI in place, continuous integration, uh, where you're constantly running all different kinds of tests and checks to make sure that the changes are safe.
00:22:47.279 --> 00:22:53.680
Uh and that you're keeping, effectively, you may not realize it, but you're keeping good logs of what happened.
00:22:53.920 --> 00:22:59.039
So you can see, okay, this change was made, Claude made this change, opened a pull request.
00:22:59.200 --> 00:23:03.200
All of your checks, the 50 of them hopefully have all ran.
00:23:03.920 --> 00:23:10.559
Codex went and reviewed the pull request and gave it an A and said that, hey, this is actually a good change.
00:23:10.720 --> 00:23:15.839
There's no fixes needed, let's approve it, and then it automatically gets merged and deployed.
00:23:16.079 --> 00:23:17.519
Every one of those gets kept.
00:23:17.599 --> 00:23:19.839
And these tools already keep this information for you, right?
00:23:19.920 --> 00:23:22.640
Like every pull request that gets merged is logged in GitHub.
00:23:22.799 --> 00:23:24.720
Every CI check is logged in GitHub.
00:23:24.880 --> 00:23:26.720
All of these things are effectively logged for you.
00:23:26.880 --> 00:23:47.359
So the thing that I think is missing that companies uh should be doing if they're not already doing, is just randomly checking those merged PRs on a regular basis to see, you know, did something actually get missed doing their own effectively post hoc uh code review to verify that that their systems are working correctly.
00:23:47.519 --> 00:24:02.720
And I think that's the big thing is that as we move into this you know very fast-moving AI AI first world, uh, and we're sort of more as an overseer orchestrator sitting above and watching what they're doing, we've got to we can trust that uh things are working, we've got to verify regularly.
00:24:02.799 --> 00:24:04.000
And I think that's what's important.
00:24:04.079 --> 00:24:08.319
It's checking, taking a look, uh, and flagging when things have gone wrong.
00:24:08.480 --> 00:24:17.200
And when they've gone wrong, that you go and take the time and energy to go fix it uh at a system at systematic level or systemic level so it doesn't happen again.
00:24:17.440 --> 00:24:18.240
Change the problem.
00:24:18.640 --> 00:24:19.599
Add more CI.
00:24:20.880 --> 00:24:24.079
I think there's two things that two questions this brings up for me.
00:24:24.160 --> 00:24:32.240
And I I guess the first, one thing that I'm realizing about the way you're you're talking about this is it's not just a technology problem, actually.
00:24:32.319 --> 00:24:35.200
It's a it's a it's a people and process problem, right?
00:24:35.359 --> 00:24:48.960
Like you have to change the way that you operate, uh the way you view security, the way you you know randomly check um the history of uh the PRs, that needs to change in terms of the way that you build.
00:24:49.279 --> 00:25:04.880
So, you know, at at Fencer, and like when you work with customers, how do you recommend from a process and procedure standpoint that they change out outside of what you just recommended, these random checks, what else needs to change in terms of the way people build software today with AI?
00:25:05.200 --> 00:25:05.519
Yeah.
00:25:05.759 --> 00:25:12.960
I th I think you know there's a lot of talk about things like harness engineering and all these other sort of very AI-specific mechanisms.
00:25:13.200 --> 00:25:25.039
But you know, I I think in a broad sense, it's define all of the rules or preconditions a change needs to meet in order to be considered valid or secure or both.
00:25:26.240 --> 00:25:29.680
And that might just be initially, and that's hopefully a checklist.
00:25:29.759 --> 00:25:35.359
And then you know, for anyone who's gone through a SOC2 process uh or some sort of compliance certification, they've probably already done that.
00:25:35.839 --> 00:25:40.960
And then they've gone and created this um this list of pre-approved changes.
00:25:41.119 --> 00:25:48.079
So under which conditions can something be effectively released to all of our customers and what does it need to show?
00:25:48.240 --> 00:25:55.119
So go through that exercise and use normal words and language, like as a human, uh, nothing particularly technical.
00:25:55.200 --> 00:25:58.720
It's like, hey, all of these tests that we wrote need to pass.
00:25:58.880 --> 00:26:06.000
Uh I need to log in and check every screen in my app to make sure that nothing broke and so forth and so on.
00:26:06.079 --> 00:26:06.240
Right.
00:26:06.319 --> 00:26:17.759
So whatever you would do as a as a person, define those things and work on adding more automation around those things so that you can get to that point where things are automatically merged.
00:26:17.920 --> 00:26:24.880
The second thing is uh making sure you define what the set of things are that do and don't need human oversight.
00:26:26.480 --> 00:26:35.119
If nothing else, that exercise will help identify for you where you may have gaps or holes, right, in terms of your processes and workflows.
00:26:35.839 --> 00:26:53.119
And then once you've done that, you know, make sure that you have an actual like monthly review process, weekly review process, quarterly review process, whatever makes sense for your business or your team, uh, and look through all the things that fell into the, hey, this is pure fully automated.
00:26:53.759 --> 00:27:01.039
Um we will uh go and check them out and make sure that they're uh appropriate, right?
00:27:01.200 --> 00:27:03.359
That our processes and workflows haven't broken.
00:27:03.599 --> 00:27:07.759
And that's I think exclusively a people and process problem, right?
00:27:07.839 --> 00:27:09.119
There's no technology for that.
00:27:10.000 --> 00:27:23.279
Yeah, and I think implicit in all of that is a solid understanding of your business logic and the things that need to be true uh to derive and surface value for your customers.
00:27:23.359 --> 00:27:29.279
And the reason why I emphasize that point is because uh this kind of gets me to my next question.
00:27:29.599 --> 00:27:36.480
When you are bytecoding or just kind of letting AI take the wheel, a lot of those decisions are made for you.
00:27:36.640 --> 00:27:40.640
And it's making a lot of assumptions around, well, I think that this is how it should work.
00:27:40.799 --> 00:27:53.519
And when you lose oversight around what needs to be true, what needs to be secure, even if the AI is writing technically correct code, I think you are opening your uh yourself up without even knowing it to another attack surface.
00:27:53.839 --> 00:27:54.799
Yeah, 100%.
00:27:55.200 --> 00:28:14.400
The other thing to consider is that a lot of what the LLMs have been trained on is publicly available code and publicly available processes, typically open source processes, which are not in and of themselves bad, but they're not you know called the highest corporate standards of security.
00:28:14.640 --> 00:28:28.240
And so what effectively gets encoded in the model that is then used to write code for you is sort of those processes, not what the best companies, you know, with the most secure systems and workflows are doing.
00:28:28.640 --> 00:28:35.920
And and that's that I think nuance maybe gets lost with folks when they're like, oh yeah, Claude, just go review what I what you changed and tell me if it's secure.
00:28:36.079 --> 00:28:43.200
You know, it's missing a huge data set that it could have been trained on, that it isn't trained on, that actually would find all those things.
00:28:43.920 --> 00:28:45.519
That's such an important point.
00:28:45.920 --> 00:28:49.759
Um yeah, so that's the the point I made around vibe coding.
00:28:49.839 --> 00:29:04.319
I'm I'm kind of curious as to your thoughts around, again, this acceleration of AI software creation and the single player modality of people using vibe coding tools and pushing out applications.
00:29:04.480 --> 00:29:13.680
Um, because you know, a lot of the things that we've been talking about is like, you know, an application and a process that has CI.
00:29:13.759 --> 00:29:15.039
They're thinking about those things.
00:29:15.119 --> 00:29:22.880
But there's a lot of applications being pushed by an individual who doesn't have a lot of experience just leveraging a vibe coding tool to do that.
00:29:23.119 --> 00:29:34.000
What are some of the dangers that they might encounter or they should be thinking about as they're pushing a vibe coding app to production or having users start to subscribe and use the product?
00:29:34.400 --> 00:29:34.799
Yeah.
00:29:34.960 --> 00:29:51.039
Um, you know, we again to be shameless, um we have we have tools that help you find a lot of really common things uh that can help you ensure that not just the code, but the other systems that you're built on top of are actually configured correctly.
00:29:51.279 --> 00:29:59.359
And so what I would say is the code itself may not necessarily be the thing uh that you get wrong.
00:30:00.240 --> 00:30:08.400
And you know, AI makes it really easy to generate a lot of code, and that code may actually be secure a lot of the time, and that gives a false sense of security.
00:30:09.200 --> 00:30:15.519
The um recently, it was actually not that recent now, in it's more like a decade in AI years.
00:30:15.759 --> 00:30:21.119
Uh Maltbook, you you remember Maltbook, uh, that is now gonna be part of Facebook, I guess.
00:30:21.200 --> 00:30:27.440
It's gonna, we're just gonna be talking to to open claw agents, malt bots, uh, all the time.
00:30:27.839 --> 00:30:34.799
Um you know what the the the code itself that the team wrote, and it was two guys behind that, by the way, that built um maltbook.
00:30:35.519 --> 00:30:41.039
The code itself wasn't insecure, it was the way that they were using Superbase was the thing that was insecure.
00:30:41.200 --> 00:30:43.359
And so then, you know, they got hacked.
00:30:43.440 --> 00:30:48.799
And you know, the other thing to consider is use platforms that are secure by default, like Xano.
00:30:49.279 --> 00:30:52.799
That will uh ensure that you don't make that kind of mistake.
00:30:52.960 --> 00:30:55.200
Uh but I think that's one area that gets overlooked.
00:30:55.359 --> 00:30:58.079
And I think the second thing is maybe more philosophical.
00:30:58.240 --> 00:31:05.440
You know, I've I've shifted away from thinking of AI and these model tools uh as being perfect.
00:31:05.680 --> 00:31:09.440
They're kind of uh maybe now an above-average employee.
00:31:09.839 --> 00:31:10.000
Right.
00:31:10.079 --> 00:31:16.400
And so if you were working with other humans, would you assume that everything they say and everything they do is always right and always perfect?
00:31:16.720 --> 00:31:17.039
No.
00:31:18.079 --> 00:31:25.920
But we seem to have ascribed that same level of you know precision and accuracy to a tool that we're using.
00:31:26.079 --> 00:31:32.240
Uh and I think that kind of like change and mindset can go a long way to you know solo vibe coders.
00:31:33.519 --> 00:31:38.000
You're you're commanding a fleet of digital employees, they're not right all the time.
00:31:38.079 --> 00:31:42.319
You should check their work and make sure that they're not telling you trash or nonsense.
00:31:44.319 --> 00:31:46.480
Yeah, you know, um it's so funny.
00:31:46.559 --> 00:31:57.599
I just I sat down with uh Doug Merritt from um from Splunk uh a couple weeks ago, and he said something uh that really hits home and is relevant to what you're saying.
00:31:57.759 --> 00:32:03.759
He said, as agents become more human in their behavior, humans have become more binary.
00:32:04.000 --> 00:32:17.279
So we're we're in this moment where the tools are getting smarter, but our thinking is getting shallower, and we're just like instead of that deep synthesis and thinking that humans are great at, we've just been relegated to clicking accept and okay.
00:32:17.440 --> 00:32:17.599
Yeah.
00:32:17.920 --> 00:32:18.400
Yeah, yeah, yeah.
00:32:18.480 --> 00:32:18.640
Yeah.
00:32:18.880 --> 00:32:20.000
It's just so interesting.
00:32:20.079 --> 00:32:24.000
And like we have to like infuse humanity back into the build process, right?
00:32:24.160 --> 00:32:24.559
That's right.
00:32:24.640 --> 00:32:24.960
That's it.
00:32:25.119 --> 00:32:26.319
It's uh uh it's interesting.
00:32:26.400 --> 00:32:39.359
I know if if the way that you interacted with an LLM was through like a video call, uh, where there was some sort of like, you know, call it uh a human-like avatar that you're interacting with, you probably wouldn't trust it.
00:32:39.519 --> 00:32:41.440
Like you wouldn't take it at face value.
00:32:41.680 --> 00:32:51.839
But then because you do it through text and Claude has this very authoritative style, or open, you know, open AI Codex GPT has this authoritative style of writing, you're just like, oh yeah, clearly it's right.
00:32:52.000 --> 00:32:53.920
So yes, accept, move on.
00:32:55.519 --> 00:33:14.640
You know, this is, I mean, maybe this is more philosophical, but one of the things that I've been thinking about is as we have this again of this acceleration of AI writing software, people vibe coding things, we're that means that us as users and as consumers are gonna have just a lot more solutions available at our fingertips.
00:33:14.880 --> 00:33:29.920
And, you know, it it really feels like, at least for me, I would be looking for some sort of security badge or something that says, like, I've built this in this certain way, because most of the time, the the software is looking for credentials.
00:33:30.079 --> 00:33:36.559
It asks you to put in your personal information, your address, your your password that then can be exposed.
00:33:36.640 --> 00:33:43.039
And if it's hacked and if it's not built in the right way, there is a broader ecosystem that is now at risk.
00:33:43.200 --> 00:33:44.480
How do you think about that?
00:33:44.559 --> 00:33:56.160
Like as this acceleration of software uh, you know, I guess proliferates throughout the world that's generally AI generated and trained, like you said, on an open source repository.
00:33:56.480 --> 00:34:00.960
How do you think secure usage looks like in the future as a consumer?
00:34:01.279 --> 00:34:01.680
Yeah.
00:34:01.839 --> 00:34:04.400
Uh, you know, there's that's a big one.
00:34:04.480 --> 00:34:09.840
I think there's a lot of different areas that uh get impacted by that change.
00:34:10.000 --> 00:34:13.199
I think one of them is the underlying supply chain, as we talked about.
00:34:13.360 --> 00:34:19.760
You know, I think what's possible, maybe probable, is more walled gardens.
00:34:20.000 --> 00:34:35.039
So instead of this kind of very federated public way of using open source software, what people end up actually doing is buying open source software from vendors that have done the work to make it secure in the first place.
00:34:35.280 --> 00:34:51.519
There's already a few companies in the world uh starting to do that, you know, where they publish uh hardened Docker images uh so that you can use a piece of software with the belief that this other company has taken the time and effort to make it secure.
00:34:51.760 --> 00:34:56.800
And so I think that plays out in different layers of the supply chain and how we deliver software to people.
00:34:57.039 --> 00:35:00.239
I think identity becomes much, much more important.
00:35:00.320 --> 00:35:15.679
And so there's a there's a draft right now uh in the IETF on how to actually track when agents have been delegated to by a people and how to effectively expose who the person is that the agent has been delegated to.
00:35:15.760 --> 00:35:25.519
So then you can kind of identify: okay, is this somebody acting on behalf, you know, as an agent, little a agent, the way the word agent is meant to be uh understood.
00:35:25.679 --> 00:35:32.639
Um is this some sort of rogue piece of software going wild and maybe doing things that it shouldn't, right?
00:35:32.800 --> 00:35:35.519
So the hacking and whatever else.
00:35:36.800 --> 00:35:39.440
Trust essentially becomes the core component.
00:35:39.599 --> 00:35:50.320
And I think what we start looking for more and more, in the same way that deep fakes make you not trust videos anymore or the news, you know, you're gonna trust software less and less.
00:35:50.480 --> 00:35:52.639
You will, as you said, look for these credentials.
00:35:52.880 --> 00:35:58.960
I think it would be dangerous for any one vendor to say, hey, we, you know, you can trust us to say that we made this secure.
00:35:59.199 --> 00:36:10.639
I think you have encryption protocols, crypto actually probably brings some utility here because it distributes trust in this very uh very secure uh manner.
00:36:10.960 --> 00:36:18.960
And some of these technological things that maybe have not had as much adoption become a uh much more common now.
00:36:19.599 --> 00:36:40.960
I'm curious how you balance that speed with control uh in a world where like it's yes, of course, security is important for a lot of platforms and services, but especially for a security company and platform like Fencer, how do you work responsibly leveraging AI in your day-to-day and with your teams?
00:36:41.360 --> 00:36:45.519
Yeah, uh actually, this has changed literally in the last 30 days.
00:36:45.679 --> 00:36:51.199
If you if we had been chatting a month ago or longer, you know, what I'm about to say would be quite different.
00:36:51.280 --> 00:36:57.440
But we're now probably close to 100% AI driven in terms of code and what have you.
00:36:57.599 --> 00:37:00.000
Uh a month ago, it may have been 40%.
00:37:00.480 --> 00:37:05.119
So, you know, in the span of a month, we've basically gone to the point where nobody really writes any code by hand.
00:37:06.000 --> 00:37:17.679
Um still things, you know, we we don't have an AI agent go and drive a browser for us and click buttons on a screen for us yet, but I'm sure that'll come in time.
00:37:18.000 --> 00:37:24.400
Um But we you know we use tools like Claude and what have you to go and write code for us.
00:37:24.559 --> 00:37:30.239
Uh I think everyone on the team, even before they open a pull request or something like that, will go review the work themselves.
00:37:30.320 --> 00:37:36.800
So we haven't yet gone to that point that I think other teams have where they've abdicated responsibility for the reviewing the code themselves.
00:37:37.039 --> 00:37:38.559
They're just like, okay, it works.
00:37:38.639 --> 00:37:40.800
Uh I wrote the tests, I wrote a spec.
00:37:40.960 --> 00:37:43.519
Great, I'm gonna go, you know, throw it over the fence.
00:37:43.679 --> 00:37:44.400
We don't do that yet.
00:37:44.480 --> 00:37:47.280
We're still um a bit hand more hands-on there.
00:37:47.599 --> 00:37:54.960
Uh we've we have probably 25,000 unit tests and I don't know, several thousand end-to-end tests.
00:37:55.039 --> 00:37:58.079
So we've we've gone pretty wild on writing tests.
00:37:58.239 --> 00:38:09.199
Uh, and so we have a very extensive test suite, a lot of CI processes and and workflows to kind of encode and enforce our standards as much as we can.
00:38:09.360 --> 00:38:14.000
And we were doing that just to stop religious debates between engineers.
00:38:14.159 --> 00:38:18.960
Uh, it just happens that that's paid off for us with with AI agents as well, right?
00:38:19.119 --> 00:38:25.199
No, no one debates whether we need tabs or spaces, and if Claude goes and adds tabs, like that's just caught and we don't have to worry about it.
00:38:25.679 --> 00:38:31.920
Um, but we we've done we did a lot of that work pre-AI, uh, and that has now paid off for us.
00:38:32.079 --> 00:38:37.760
And so when we get to a code review, um we look at humans look at the architecture.
00:38:38.000 --> 00:38:43.199
We spend less time on the stylistic parts of the code, but we still require all code to go through human review at the moment.
00:38:43.599 --> 00:38:52.079
30 days ago, I would have talked to a Tim that had adopted it 40 or where AI is writing 40% of the code.
00:38:52.159 --> 00:38:53.280
Now it's 100.
00:38:53.440 --> 00:38:58.400
And now you're uh at a place where you're still making sure to review everything.
00:38:58.480 --> 00:39:00.960
You haven't fully abdicated responsibilities.
00:39:01.199 --> 00:39:08.639
I'm curious for other engineering leaders listening to this who might be at 0% and they're like, look, I've been holding back.
00:39:09.119 --> 00:39:23.119
What were the gates or goalposts or milestones that made you feel comfortable in increasing, I guess, AI's uh responsibility in writing more and more of your code?
00:39:23.679 --> 00:39:34.079
And then the final piece to that is when will you feel comfortable abdicating your responsibility because it is writing all of the code for you?
00:39:34.400 --> 00:39:34.800
Yeah.
00:39:35.039 --> 00:39:39.519
Uh I think firstly, the just the quality of the models improved drastically.
00:39:39.599 --> 00:39:47.840
So the actual results, it very quickly became clear to me that I didn't need to, you know, closely review every change.
00:39:48.559 --> 00:39:51.920
Uh and so maybe maybe it's a little bit more than a month ago now.
00:39:52.159 --> 00:39:57.519
Certainly since November, there was already a change, but even you know, in the last month or two, it's gotten better.
00:39:57.679 --> 00:40:12.800
Um, the the actual results were very consistent with our quality standards, met you know, the style with which we write code, the the types of patterns that we follow, and things like that, just did a much better job of identifying those things.
00:40:13.039 --> 00:40:30.079
And you know, pre our extreme adoption, I know that what I was skeptical of was whether an LLM would write code in a way that was consistent with how we wrote it as a team, that would follow the standards or would follow the uh architectural patterns that we had established.
00:40:30.239 --> 00:40:43.039
You know, the way that you interact with these sorts of things, or the way that you build a feature X over this way is you know, a certain components get wound together versus just write any old sloppy result.
00:40:43.519 --> 00:40:51.280
Um and I think now that I've seen that it's really good at it, I I think that's what would have held me back a while ago.
00:40:51.519 --> 00:40:59.440
I think the the larger organizations that have restrained people from using LLMs, I think fear two things.
00:40:59.679 --> 00:41:04.000
One, you know, is anthropic or open AI just training on their on their code, right?
00:41:04.079 --> 00:41:13.119
And so they're giving away their secrets, effectively, their core IP, to some third party, which I, you know, you've got to trust the agreements that you sign with companies at some point.
00:41:13.280 --> 00:41:15.360
Um to put that to one side.
00:41:15.599 --> 00:41:22.559
I think the one that I acknowledge when people are appropriately skeptical is the security piece.
00:41:22.719 --> 00:41:41.280
Specifically, will the AI go and find some sort of secret that you've got in an ENV file or a text file or some sort of file on your computer, in your file system, that gives it access to way more than it should, and you don't necessarily see that.
00:41:41.599 --> 00:41:47.760
Uh and I think the only way to overcome that is to sandbox it better, right?
00:41:47.920 --> 00:41:49.360
Run it inside of a container.
00:41:50.079 --> 00:41:53.679
Um because the productivity gains are just undeniable.
00:41:54.000 --> 00:42:01.440
And I think uh the previous skepticism around hallucination, whatever else, like I don't think is justified at all anymore.
00:42:01.760 --> 00:42:04.880
Uh and when you tell it not to hallucinate, it doesn't hallucinate, right?
00:42:05.039 --> 00:42:24.000
So it's uh um uh I I think that that you must, it is a tidal wave, you have to adopt it, and the things that would have made you skeptical one, two, three months ago are no longer there, and now it's just putting the right constraints in place.
00:42:24.159 --> 00:42:35.280
So spend some time, create a little, you know, isolated Docker box, you know, EC2 instance, whatever you use to allow your team to use it, and just go go for it.
00:42:36.000 --> 00:42:36.880
I love it.
00:42:37.119 --> 00:42:46.559
Um the final question I had just before we close is how you think about hiring um and building the right team in today's world.
00:42:47.440 --> 00:42:51.119
Um is that who are the people that you look for?
00:42:51.280 --> 00:42:53.760
Like what are the qualities, what are the traits?
00:42:54.000 --> 00:43:00.880
Um, what type of AI capability do they need to have before you consider them, for example, working at a place like Fencer?
00:43:06.000 --> 00:43:07.920
I don't know if people heard that deep sigh.
00:43:08.159 --> 00:43:09.519
That was a deep sigh.
00:43:09.679 --> 00:43:20.079
I think this is the single hardest question, at least for me right now, the single hardest question uh in in this new world, more so than these like technical uh things that we've been discussing.
00:43:20.960 --> 00:43:25.360
I really, I really don't know what to look for right now in hiring.
00:43:25.440 --> 00:43:37.280
Um, because I think the things that I used to use as good signals of a person's ability and competence and you know success uh are very different in the current world.
00:43:38.320 --> 00:43:43.519
And I think it's you know, I haven't had many opportunities to test it out.
00:43:43.599 --> 00:43:51.199
It's not like we've gone and hired a hundred people in the last three months to really refine what we would actually look for uh to find somebody that's successful.
00:43:51.440 --> 00:43:58.320
A lot of our hiring happened over a year ago um or about a year ago now, and and things have changed drastically in the last 12 months.
00:43:58.480 --> 00:43:59.440
So I don't know.
00:43:59.599 --> 00:44:03.199
I I think certain fundamentals are still just as important as they were before.
00:44:03.440 --> 00:44:09.199
I think certain personality traits are just as important now as they were before.
00:44:09.519 --> 00:44:13.360
And what motivates people just as important now as it was before.
00:44:13.679 --> 00:44:15.840
Are people purely intrinsically motivated?
00:44:16.000 --> 00:44:22.480
Are they only motivated effectively through you know having high compensation or some sort of explicit reward?
00:44:22.559 --> 00:44:26.880
You know, the the things that uh I would look for is obviously a balance.
00:44:27.039 --> 00:44:34.400
Um still makes sense, but in terms of assessing a person's ability to get work done, it's very different.
00:44:34.639 --> 00:44:36.559
And I don't know, I don't have a good answer.
00:44:36.639 --> 00:44:38.480
It's something I'm still trying to figure out.
00:44:38.639 --> 00:44:46.880
Um and I, you know, right now, the way I work a lot of the time is here is a feature description that I wrote down.
00:44:47.119 --> 00:44:48.559
I give it to Claude.
00:44:48.800 --> 00:44:52.400
Claude creates some code, I look at the code, it makes sense.
00:44:52.639 --> 00:44:54.159
Kind of how I would have written it.
00:44:54.239 --> 00:44:54.960
That's good.
00:44:55.119 --> 00:45:00.239
Like if I tested somebody on their ability to do that, like I don't know that many people would fail.
00:45:00.480 --> 00:45:04.079
So there's obviously much more to that.
00:45:04.239 --> 00:45:13.280
Um, and and more importantly, I think now it's about assessing how well someone understands what an LLM is doing, at least in a technology company.
00:45:13.760 --> 00:45:16.079
You're such a uh seasoned engineering leader.
00:45:16.159 --> 00:45:20.159
You've been at engineering and you've led engineering teams, big and small.
00:45:20.320 --> 00:45:23.599
And even you are in a place that we are where we're figuring this out.
00:45:23.679 --> 00:45:37.599
And I actually think that's going to bring a lot of comfort to people that the uncertainty that they feel around who is going to be the highest leverage individual to bring onto your team right now is changing and transforming.
00:45:37.679 --> 00:45:50.000
And I think like right now it's uh, you know, for us, it's kind of like someone with high agency, intrinsically motivated, and just a voracious appetite to learn.
00:45:50.159 --> 00:46:00.880
Um, if you have that as a foundation, I mean, I think that at least, you know, gives you like that initial prerequisite to like, I think that you're going to be successful here.
00:46:01.119 --> 00:46:24.960
But still, I think what we've what we find, especially in engineering, is like the more just fundamental computer science, computer like software engineering principles you foundationally understand, the higher the leverage you are to, for example, leverage something like conductor, use multiple agents uh at the same time, and be productive in a way that someone that doesn't have that muscle memory can't do.
00:46:25.280 --> 00:46:25.679
That's right.
00:46:25.840 --> 00:46:28.960
Um but it's still, I mean, I'm I'm with you.
00:46:29.039 --> 00:46:37.199
And I and I and I'm so glad that you shared that you didn't know the real answer to the question because I think it's something that we're all asking ourselves and trying to figure out.
00:46:37.360 --> 00:46:37.519
Yeah.
00:46:37.679 --> 00:46:43.840
I mean, it's if anybody is out there saying that they definitely know, you know, I I don't know that I believe them.
00:46:44.400 --> 00:46:49.599
People are uh interested in learning more about you, about Fencer, where should they go?
00:46:49.920 --> 00:46:59.599
Yeah, well I'm active on Twitter uh and LinkedIn, uh Timolsch on Twitter, uh Tim Olshansky on LinkedIn, and uh Fencer Fencer Security.
00:46:59.679 --> 00:47:03.360
Uh you can find us online and uh major social channels.
00:00:00.080 --> 00:00:04.480
If an AI agent is going and writing code for you, do you have any idea whether it's secure or not?
00:00:04.639 --> 00:00:09.759
Or are you just relying on the same model that went and created the code to tell you that it's secure?
00:00:23.600 --> 00:00:24.719
Welcome to Future Proof.
00:00:24.800 --> 00:00:27.199
I'm Prakash Chandran, CEO of Xano.
00:00:27.519 --> 00:00:37.200
Today I'm joined by Tim Olshansky, co-founder of Fencer, a security platform that replaces the patchwork of scanners, monitors, and spreadsheets most startups cobble together.
00:00:37.439 --> 00:00:40.960
I know from personal experience, and actually fixes what it finds.
00:00:41.119 --> 00:00:57.439
Tim is a serial founder and engineering leader who's been CTO of a publicly traded company, scaled engineering teams to over 200, and built products for brands like Chipotle and Burger King, as well as the DoD and three letter agencies that will go unspecified.
00:00:57.679 --> 00:01:06.000
He's seen firsthand what happens when compliance becomes a prerequisite for growth and what breaks when teams skip the security foundation to ship faster.
00:01:06.239 --> 00:01:12.879
There's a question Tim's been asking that I think every AI builder needs to hear, and that is who approved this change?
00:01:13.120 --> 00:01:23.439
As agents start writing code, opening PRs, and sometimes merging changes on their own, the chain of human accountability that every compliance framework was built around is quietly breaking.
00:01:23.680 --> 00:01:27.760
This is a conversation about what that means and what to do about it.
00:01:28.000 --> 00:01:30.079
Tim, thank you so much for being here.
00:01:30.159 --> 00:01:31.439
It's really great to have you.
00:01:31.840 --> 00:01:33.439
Hi, Prakash, and hi everybody.
00:01:33.519 --> 00:01:34.640
It's uh it's nice to be here.
00:01:34.719 --> 00:01:36.319
It's been a it's been a while.
00:01:36.719 --> 00:01:37.760
It has been a while.
00:01:37.920 --> 00:01:42.640
And fun fact, if I may mention, you are an investor uh in Xano.
00:01:42.959 --> 00:01:54.000
Just kind of thank you so much for not only your belief in us, but it also just kind of goes to show like the technologists that you are not only contributing to the space, but investing in platforms like ours.
00:01:54.799 --> 00:01:56.239
Yeah, I love what you're building.
00:01:56.400 --> 00:01:56.719
Sorry.
00:01:56.879 --> 00:01:58.719
It was it wasn't a hard choice.
00:01:59.120 --> 00:01:59.840
Awesome.
00:02:00.000 --> 00:02:02.319
So I wanted to start with a little bit of background.
00:02:02.400 --> 00:02:15.439
So, you know, you've co-founded, as I mentioned in the intro, you know, multiple companies, been the CTO of a public company, scaled engineering teams, and then you decided to build something around security, a security platform for startups.
00:02:15.680 --> 00:02:21.759
So I'm curious like what you saw that made you feel like the security problem was worth solving.
00:02:22.159 --> 00:02:28.319
Yeah, it's uh poignant, I think, because everybody is now thinking about building their own software and building their own products.
00:02:28.400 --> 00:02:32.240
And so um, hopefully, security is something that people are thinking about a lot more.
00:02:32.400 --> 00:02:47.759
But over the course of my career, it was one of those things that I became responsible for as the CTO, technical co-founder, head of engineering, uh, software engineer on a team that I often felt was not being done well at the earlier stages.
00:02:47.919 --> 00:02:58.800
And then when you know I was a part of larger organizations, the level of bureaucracy and difficulty it was to be able to use these tools and use them well was just crazy.
00:02:58.960 --> 00:03:04.080
So we we we set out on this mission um to build something effectively to solve our own problem.
00:03:04.240 --> 00:03:09.039
Uh, myself and my co-founder had felt the pain uh and we wanted to make this easier.
00:03:09.360 --> 00:03:17.759
I had, you know, obviously had to go and implement a lot of these things myself uh in the the pre fencer days.
00:03:17.919 --> 00:03:20.319
And so I think we had a great idea for what we wanted to do.
00:03:20.400 --> 00:03:25.680
And then at the same time, uh knowing the pain ourselves, we'd spoken to lots of different companies.
00:03:25.759 --> 00:03:34.479
I actually spoke to Sean at Zano and and and others uh across the industry about some of their challenges in the early days, and same message.
00:03:34.639 --> 00:03:36.159
Like this is this is difficult.
00:03:36.319 --> 00:03:46.000
Uh so we we saw that problem combined with the fact that AI was writing a lot of code, knowing that more code is going to be written in the future, more products are gonna get built.
00:03:46.080 --> 00:03:48.000
This felt like a great space to build in.
00:03:49.120 --> 00:03:50.560
Yeah, that makes a lot of sense.
00:03:50.719 --> 00:04:03.759
In in the conversations that you've been having, I feel like you started or you kind of saw early on that this security gap was something even before AI and AI building has had taken off, like it really, really has today.
00:04:03.919 --> 00:04:08.800
Um, it only feels actually now like that that has been exacerbated.
00:04:08.879 --> 00:04:13.680
You know, like there's so much that people rely on now uh in terms of AI building.
00:04:13.840 --> 00:04:21.839
And I know that one of the in doing research for this, one of the things that you wrote is, you know, AI tools are great for building features, but they're just terrible at security.
00:04:22.000 --> 00:04:23.519
I wonder if you could just speak to that.
00:04:23.839 --> 00:04:24.160
Yeah.
00:04:24.319 --> 00:04:37.680
Um, you know, it's funny, just as like a little bit of history, you know, 10 years ago, if not more, uh, it was pretty rare to see a company less than 50 employees go through the SOC2 certification process.
00:04:38.000 --> 00:04:49.040
Uh and a lot of new platforms got built, you know, Vanta, Drata, others uh that I'm sure folks that are listening, watching have probably heard of, um, have made that easier.
00:04:49.199 --> 00:05:01.439
So everyone is going and getting these compliance certifications that come with security requirements, who are perhaps selling SaaS products uh or just want to have this checkbox against their business.
00:05:01.600 --> 00:05:08.639
Uh they're able to get them at a much earlier stage, but they're not actually doing half the things that they're saying or promising that they're going to be doing.
00:05:09.680 --> 00:05:28.000
And so that's that's like the kind of industry has pushed this uh compliance standard to people, compliance certification to people at earlier and earlier stages, but they aren't themselves equipped with the processes and understanding of what to go do, they just delegate to some tool to go figure that out for them.
00:05:28.079 --> 00:05:36.639
And so that's that's a big part of what we're trying to help with is actually make security accessible and something that you can actually accomplish.
00:05:36.879 --> 00:05:44.079
And so when you say that you have uh you know code review processes and approval processes and all these other things, do you really?
00:05:44.240 --> 00:05:45.839
Do you know what's actually going on?
00:05:46.000 --> 00:05:59.519
Uh are you just checking the box if uh if an AI agent is going and writing code for you uh and it produces a pull request or a merge request or you know, some sort of change set for you to look at.
00:05:59.680 --> 00:06:01.680
Do you have any idea whether it's secure or not?
00:06:01.839 --> 00:06:06.480
Or are you just relying on the same model that went and created the code to tell you that it's secure?
00:06:06.800 --> 00:06:11.279
It's like asking a junior engineer to tell you whether they think the code they wrote is secure.
00:06:11.439 --> 00:06:13.439
And so that's that's kind of the premise, right?
00:06:13.600 --> 00:06:16.399
The models are great, but and they're only getting better.
00:06:16.560 --> 00:06:22.160
But even still, like you still want a second set of eyes on many changes as much as possible.
00:06:22.560 --> 00:06:34.959
Yeah, I think also no matter how good the models get, I think that security, compliance, and how it relates to your business and the rules around your business will always be something that needs to that needs oversight.
00:06:35.600 --> 00:06:44.079
Um, so I would love for you to talk a little bit about Fencer in the context of actually security and compliance and how people should be thinking about it.
00:06:44.240 --> 00:06:53.279
Who are the types of people or what types of applications need to start thinking about security very early on, and then introduce a little bit about what fencer does.
00:06:53.600 --> 00:06:54.000
Yeah.
00:06:54.160 --> 00:07:08.800
You know, it's timing's interesting because if uh if mythos and all these new LLMs and models and everything else get released to the public in the next week and all of a sudden everything's hacked all the time, and then you know, we all need to be dealing with security.
00:07:09.040 --> 00:07:13.600
I don't know what our banks are gonna do, you know, uh doom and gloom, etc.
00:07:13.920 --> 00:07:17.920
Although I think it's a good thing because all of these bugs that have been there forever will finally get fixed.
00:07:18.079 --> 00:07:25.920
But um, you know, really what we set out to do is to make this easier for companies or businesses that don't have a security team, right?
00:07:26.000 --> 00:07:28.319
And and probably don't even have a head of security.
00:07:28.399 --> 00:07:37.759
It's some individual that spends maybe 20% of their time per month maybe uh dealing with security issues.
00:07:38.000 --> 00:07:41.439
And that usually means companies with less than 200 employees, right?
00:07:41.519 --> 00:07:44.480
Um often smaller than that, less than 20 employees.
00:07:44.800 --> 00:07:50.000
And that's the sweet spot, I think, to go and set up a good security foundation.
00:07:50.079 --> 00:08:17.759
Uh, and that's uh you know, a number of different things that I could go on for hours, but it ranges from just having decent uh awareness of where you may have security holes with scanning tools and and and what have you, uh doing penetration testing if you're building software, uh, and if you're not building software for external consumption, then at least being aware of where you may have holes uh for your own internal consumption is pr is pretty useful.
00:08:18.000 --> 00:08:29.759
Uh because while you may not sell software, the business that you're in may be a great target for an opportunistic hacker in Eastern Europe or somewhere else, um, not just a nation state.
00:08:29.920 --> 00:08:34.879
And uh and the things that you do at that stage are pretty straightforward uh and not that expensive.
00:08:35.919 --> 00:08:37.120
Kind of like brushing your teeth, right?
00:08:37.200 --> 00:08:38.080
It's good hygiene.
00:08:38.240 --> 00:08:40.879
You do it a little bit if you do it every day for two minutes.
00:08:41.120 --> 00:08:42.559
You don't have to go to the dentist.
00:08:42.639 --> 00:08:49.840
It's kind of a similar premise, uh, albeit the consequences are probably far worse than a filling, um, if you get it wrong.
00:08:50.559 --> 00:08:51.120
Right.
00:08:52.080 --> 00:09:04.639
And so I'm curious, like in the context of fencer, like I mean, that totally makes sense in terms of like how you need to be thinking about it, especially in the in the world that we're living in now with all these new releases.
00:09:04.799 --> 00:09:12.960
Um, when you started thinking about fencer, when we talk about security and compliance, that's a really broad surface area.
00:09:13.039 --> 00:09:13.200
Yeah.
00:09:13.360 --> 00:09:27.440
So, how do you think about compartmentalizing the different parts of security and how people can and how people can, for example, leverage a tool like Fencer to start becoming secure from day from day one, really?
00:09:28.080 --> 00:09:34.879
So there's there's a lot of different areas that need to be secure uh across a business, right?
00:09:35.039 --> 00:09:41.519
And while we can't personally cover all of them uh as a business, there's lots of other great players in the space.
00:09:41.600 --> 00:09:53.519
The way that we generally think about it is for a company in that kind of category that I described, 200 or fewer, uh, you have software that you write, which means code.
00:09:53.759 --> 00:09:56.240
You have infrastructure that you run on, right?
00:09:56.399 --> 00:10:05.519
Whether your own, I mean, I don't see many companies going and buying racks and putting them in a data center, uh, they're more likely to be building on top of a platform like Xano, right?
00:10:05.600 --> 00:10:09.679
And so they're they're building their tools and applications on on top of others.
00:10:09.919 --> 00:10:16.080
Um they they have networks that they're responsible for, and they may not realize it.
00:10:16.159 --> 00:10:26.960
You know, just because you're building on top of a service like Xano, you know, implicitly there is a network that your systems communicate with that you need to be aware of.
00:10:27.120 --> 00:10:28.320
You have your own user accounts.
00:10:28.399 --> 00:10:38.000
You know, if you're part of a company, you're there's gonna be lots of different identities that are interacting, and not just humans, you've got agents, you've got applications, you've got bots, uh, et cetera.
00:10:38.320 --> 00:10:50.080
Um and and beyond just being aware of those uh perhaps security holes or misconfigurations, uh, you also need the observability or the monitoring of those things, right?
00:10:50.240 --> 00:11:09.279
So ingesting or collecting events from lots of different data sources to help you flag and identify when you know somebody has gone and used some automated tool against your app or against your system to see if any of those uh if those holes exist.
00:11:09.600 --> 00:11:11.679
And and I think these are sort of like baseline things.
00:11:11.759 --> 00:11:16.799
And I haven't even touched on your own laptop as a potential issue or your phone or something like that, right?
00:11:16.879 --> 00:11:22.399
Because that as well is uh is another common source of uh security issue.
00:11:22.879 --> 00:11:30.159
You know, the the most common place where companies get hacked from is just somebody messed up with their uh username and password.
00:11:30.320 --> 00:11:39.440
They reused a password on a some SaaS site that wasn't doing a good job, and now their password has been exposed, or they're using a common password that someone was able to brute force.
00:11:39.519 --> 00:11:46.960
So I think it's nearly a quarter of all hacks start with uh somebody's bad password or bad username and password.
00:11:47.200 --> 00:11:50.559
And so identity is a big piece of it, uh, and we help monitor that.
00:11:50.720 --> 00:12:00.080
Um and and so there's you know those big areas that I mentioned, the five or six that I mentioned, that we sort of have pulled together into one platform and give you much greater control and visibility.
00:12:00.320 --> 00:12:13.039
Uh and then if you're the type of business that needs to prove to others that you're secure, we then automate all of the evidence collection and all of the bits and pieces to go and show people, hey, we actually take security seriously.
00:12:13.120 --> 00:12:16.559
Here are all the things we're doing, here are the things we found, and here are the things that we fixed.
00:12:17.120 --> 00:12:19.039
Not to mention helping you fix it.
00:12:19.360 --> 00:12:29.600
Yeah, it's really it's really interesting because I feel like um a lot of the other tools uh in the land or compliance tools in the landscape that you mentioned are very much like checklist driven.
00:12:29.679 --> 00:12:32.720
Like you need to do all of these things to get this certification.
00:12:32.960 --> 00:12:45.759
Where I think um what's what I've always liked about Fencer, and I think really the way that people need to be approaching building applications today is a very like um behavioral or security first.
00:12:45.919 --> 00:12:56.159
Like you're doing the the tool is actually helping you remain secure and giving you auditability and observability over thing, and actually the compliance certification is an after-effect of that.
00:12:56.240 --> 00:12:56.639
That's right.
00:12:56.960 --> 00:12:59.279
Um so that that that makes a lot of sense to me.
00:12:59.519 --> 00:13:06.639
You know, you said something interesting around like how there's kind of like a compliance supply chain or they're like a system.
00:13:07.039 --> 00:13:12.480
And you know, relevant, obviously, we had the uh light LLM supply chain attack.
00:13:12.559 --> 00:13:17.200
Um, there are other examples of that kind of surface that you're not alone.
00:13:17.279 --> 00:13:20.639
You don't, when you install one thing, you're not just getting that thing.
00:13:20.799 --> 00:13:39.759
I was wondering if you could help explain to the audience um, maybe at a higher level, about supply chain risks, uh compliance chain risks, and what they need to be aware of as they're depending on uh some of these foundational, sometimes open source products and and and thereabout.
00:13:40.320 --> 00:13:40.720
Yeah.
00:13:40.960 --> 00:13:50.320
It's you know, I think um, and I I don't want I don't want to uh make this sound too simple, but just like in the physical world that there's a supply chain of goods, right?
00:13:50.399 --> 00:13:58.000
Like there's no there's no car today where every individual component gets made by Ford, right, or GM or a single company.
00:13:58.320 --> 00:14:06.879
Uh there's many different businesses that go and produce these pieces that get brought together to make a car the same way software is made today.
00:14:07.039 --> 00:14:13.759
It's just the main difference in the digital world is the delivery mechanism is over the internet, so it's basically lightning fast.
00:14:14.240 --> 00:14:24.559
Uh and that there are uh selfless people who will happily give up their own time to create software for everybody's benefit and expect no financial reward.
00:14:24.639 --> 00:14:26.000
And so that's open source software.
00:14:26.159 --> 00:14:41.440
And so a large, huge portion of uh the internet today and a lot of what we have built all of our software on relies on these open source software projects, which are the un effectively the fundamental components of a lot of the digital supply chain uh that we rely on today.
00:14:41.759 --> 00:14:56.080
And they're they're interesting because these tools, uh, these libraries and these pieces of software are like our light LLM is an example of Trivi, another recent one, uh that was actually the one that caused Lite LLM to be hacked.
00:14:56.159 --> 00:15:05.919
And then there was Axios, which is a popular uh JavaScript or TypeScript library uh they're used for building a lot of web applications today and used in a lot of different products.
00:15:06.159 --> 00:15:17.120
Um they're all open source projects, and so if a hacker gains access to them, they inadvertently gain access to lots of commercial software and lots of commercial products.
00:15:17.360 --> 00:15:30.960
And so what these threat actors, uh, these hackers have figured out is that they don't need to try and catch a break into these private organizations because they're actually doing something around security.
00:15:31.120 --> 00:15:49.360
They're now finding that actually the open source, you know, uh effectively volunteer community is a great place to go and uh exploit their limited resources and security gaps to be able to then gain access to many, many more um organizations and take advantage of them.
00:15:49.519 --> 00:15:53.360
So that's only going to get worse and faster over time.
00:15:53.840 --> 00:16:10.720
And I think the kind of relating to your previous point about fixing things for people, you know, you might not know about these problems, and you know about them when somebody kind of exploits them and they make a public statement, which is what the uh organization, you know, the hacking group, uh did.
00:16:10.960 --> 00:16:14.159
They tweeted about it and showed everyone how great they are.
00:16:14.399 --> 00:16:18.320
Um what's important now is that as soon as you find it, you fix it.
00:16:18.879 --> 00:16:21.279
Because not everyone can be exploited instantly.
00:16:21.440 --> 00:16:31.840
And so if you are the lucky many who aren't being regularly exploited, uh then as soon as you know, being able to respond and react and fix those things is really, really important.
00:16:32.159 --> 00:16:33.440
I think that's sort of the future.
00:16:33.600 --> 00:16:40.879
As we get better at finding these things, we will actually lag in our ability to fix these things, and that's always been the biggest problem.
00:16:41.200 --> 00:16:44.240
Uh still is the biggest problem, I think will be the biggest problem for a while.
00:16:44.399 --> 00:16:52.879
Like, how can you quickly go and upgrade your supply chain components uh to be on the safe versions so you yourself aren't at risk?
00:16:54.159 --> 00:16:56.639
Yeah, and that flows through everywhere and proving it, right?
00:16:56.720 --> 00:16:58.639
So to the compliance piece as well.
00:16:58.960 --> 00:16:59.600
Totally.
00:16:59.840 --> 00:17:16.400
And like in I just want to highlight something that you said that I think is really important because as AI tooling and AI building adoption um accelerates and it's been accelerating exponentially, um, so does the dependency surface area.
00:17:16.640 --> 00:17:23.359
And that dependency surface area, in many cases, people don't even realize the dependencies that they're leveraging, right?
00:17:23.440 --> 00:17:25.759
Their AI is making the decisions for them.
00:17:25.920 --> 00:17:28.400
Um there's a lot of dependencies that are installed.
00:17:28.640 --> 00:17:39.359
And so, you know, it's it's not only just like the security first mentality, is just understanding what is it that you have and also the way that it's being pulled in, right?
00:17:39.440 --> 00:17:40.480
Is it like the latest?
00:17:40.559 --> 00:17:42.160
Is it always fetching the latest?
00:17:42.400 --> 00:17:44.160
What is the secure way to proceed?
00:17:44.319 --> 00:17:49.200
And I think that's a question that people, number one, they I don't know that they know how to begin.
00:17:49.279 --> 00:17:55.680
Like real builders today, or if I'm an application development leader uh managing my team, like where do I begin?
00:17:55.839 --> 00:17:57.839
What is the safe way to do this?
00:17:58.000 --> 00:18:02.240
Because to say not to leverage some of these open source projects seemed crazy.
00:18:02.480 --> 00:18:04.000
So, what is the middle ground?
00:18:04.079 --> 00:18:08.880
Like, how what would you recommend to people thinking about this and they want to do things securely?
00:18:09.200 --> 00:18:20.079
Yeah, it's it's you know, getting started is actually a lot easier now than it was probably a month ago, uh, thanks to some of these uh very public disclosures of uh of these hacks.
00:18:20.559 --> 00:18:25.039
Yeah, it's it's actually I think um straightforward.
00:18:25.279 --> 00:18:31.839
You know, we at Fencer have published blogs on the topic, um, just to be a shameless shill uh for a moment.
00:18:32.160 --> 00:18:34.480
But the I think I'll I'll allow it.
00:18:34.720 --> 00:18:35.519
You'll allow it.
00:18:35.680 --> 00:18:39.119
Um, it's use use a package manager, right?
00:18:39.359 --> 00:19:00.400
I think basically everyone does these days, but you know, use a package manager and don't just use a package manager blindly, but use a package manager with a lock file or some other manifest that you can be explicit in specifying which version of your dependencies should be installed at any moment and which version of your dependencies when you run your application will be present and available.
00:19:00.960 --> 00:19:21.200
Uh and ensure that you're using a package manager that will effectively create a uh a hash or you know, basically an integrity value, a value with a lot of integrity that you can trust that the underlying software uh that you're importing has not been modified in any way.
00:19:21.359 --> 00:19:32.480
Um there's in every ecosystem that exists in the Java world, in the JavaScript world, in the uh Python world, Ruby, Go, you name it, there's there's great package managers.
00:19:32.720 --> 00:19:35.279
Um the that's number one.
00:19:35.359 --> 00:19:37.440
So use a good package manager with a lock file.
00:19:37.519 --> 00:19:39.519
Uh use their cooldown feature.
00:19:39.680 --> 00:19:46.720
Nearly all of them at this point have a way for you to say, hey, don't use the latest thing.
00:19:47.039 --> 00:19:47.680
Wait.
00:19:48.079 --> 00:19:55.359
If something is new today when I try to install a new version, uh don't use it unless it's at least seven days old or three days old or whatever.
00:19:55.839 --> 00:19:56.880
Specify a cooldown.
00:19:56.960 --> 00:20:05.599
Uh and I generally recommend to people at least two days, uh, but probably beyond five or six days, six days, it's diminishing returns.
00:20:05.920 --> 00:20:15.759
You know, most uh most open source software, I'm I'd say 95% will have a fix out within two days.
00:20:16.000 --> 00:20:23.119
Um and so you know if if you're risk-averse like we are, we're a security vendor, we've got it set to a week.
00:20:23.279 --> 00:20:33.440
Um and the most common time for hacks to occur is a Friday night uh because people are not gonna check some automated process until Monday.
00:20:33.599 --> 00:20:35.200
And so give yourself some time.
00:20:35.279 --> 00:20:39.279
Um and then I can obviously keep going on, but that's some of the simplest things that you can go do.
00:20:39.519 --> 00:20:47.039
No, I think that's I think that's really that's really helpful just as a foundation for how people should be thinking about it.
00:20:47.200 --> 00:20:52.799
So, first and foremost, just understand some of the dependencies that your application, your AI application is generating.
00:20:52.960 --> 00:20:58.000
And then uh secondly, yeah, use some of those best practices and uh especially that cooldown period.
00:20:58.079 --> 00:20:58.880
That makes a lot of sense.
00:20:59.119 --> 00:21:03.839
This world that we're moving into, where agents are, for example, doing more of the work.
00:21:03.920 --> 00:21:11.279
And one of the things that you have been talking about that teams aren't necessarily confronting is that question that I mentioned up at the top.
00:21:11.440 --> 00:21:12.960
Who approved this change?
00:21:13.440 --> 00:21:19.920
Um, I'd love for you to walk us through what breaks in the compliance chain when agents start writing and merging code.
00:21:20.000 --> 00:21:21.440
Like how do you think about this?
00:21:21.759 --> 00:21:33.279
Yeah, I I can't remember who it was published a blog post recently, you know, big tech company about how something like 30% of their changes are now being reviewed, approved, and merged by AI.
00:21:33.440 --> 00:21:45.200
And so as as a lot of engineering practices have kind of come from the bigger companies to the smaller companies, I imagine that that's coming for everyone if they're not already doing it in certain cases.
00:21:45.440 --> 00:21:49.680
Certainly solo builders are probably already doing that, but they're on their own, so that makes sense.
00:21:49.920 --> 00:21:59.839
Um it's it's a it's an interesting one because general security best practices said that the people who the person who wrote the code should not be the one to approve the code.
00:22:00.960 --> 00:22:01.279
Right.
00:22:01.440 --> 00:22:04.799
So you in the pre-AI world, you'd have a teammate take a look at it.
00:22:04.880 --> 00:22:08.400
And tools like GitHub and GitLab even make that really easy for you.
00:22:08.559 --> 00:22:11.759
You'd have a pull request, you take a look at the change, looks good.
00:22:12.079 --> 00:22:20.559
Click a button, it's approved, and you know, merged, and whatever automation you have in place to uh update your systems and servers uh would play out.
00:22:21.200 --> 00:22:27.119
In AI world, you know, again, I I believe in sort of trust but verify.
00:22:27.599 --> 00:22:37.519
Uh and if you're if you're confident that the tool is doing the right thing, right, you've let your AI systems run wild and approve and merge things for you.
00:22:37.680 --> 00:22:47.119
I'm hoping that you have CI in place, continuous integration, uh, where you're constantly running all different kinds of tests and checks to make sure that the changes are safe.
00:22:47.279 --> 00:22:53.680
Uh and that you're keeping, effectively, you may not realize it, but you're keeping good logs of what happened.
00:22:53.920 --> 00:22:59.039
So you can see, okay, this change was made, Claude made this change, opened a pull request.
00:22:59.200 --> 00:23:03.200
All of your checks, the 50 of them hopefully have all ran.
00:23:03.920 --> 00:23:10.559
Codex went and reviewed the pull request and gave it an A and said that, hey, this is actually a good change.
00:23:10.720 --> 00:23:15.839
There's no fixes needed, let's approve it, and then it automatically gets merged and deployed.
00:23:16.079 --> 00:23:17.519
Every one of those gets kept.
00:23:17.599 --> 00:23:19.839
And these tools already keep this information for you, right?
00:23:19.920 --> 00:23:22.640
Like every pull request that gets merged is logged in GitHub.
00:23:22.799 --> 00:23:24.720
Every CI check is logged in GitHub.
00:23:24.880 --> 00:23:26.720
All of these things are effectively logged for you.
00:23:26.880 --> 00:23:47.359
So the thing that I think is missing that companies uh should be doing if they're not already doing, is just randomly checking those merged PRs on a regular basis to see, you know, did something actually get missed doing their own effectively post hoc uh code review to verify that that their systems are working correctly.
00:23:47.519 --> 00:24:02.720
And I think that's the big thing is that as we move into this you know very fast-moving AI AI first world, uh, and we're sort of more as an overseer orchestrator sitting above and watching what they're doing, we've got to we can trust that uh things are working, we've got to verify regularly.
00:24:02.799 --> 00:24:04.000
And I think that's what's important.
00:24:04.079 --> 00:24:08.319
It's checking, taking a look, uh, and flagging when things have gone wrong.
00:24:08.480 --> 00:24:17.200
And when they've gone wrong, that you go and take the time and energy to go fix it uh at a system at systematic level or systemic level so it doesn't happen again.
00:24:17.440 --> 00:24:18.240
Change the problem.
00:24:18.640 --> 00:24:19.599
Add more CI.
00:24:20.880 --> 00:24:24.079
I think there's two things that two questions this brings up for me.
00:24:24.160 --> 00:24:32.240
And I I guess the first, one thing that I'm realizing about the way you're you're talking about this is it's not just a technology problem, actually.
00:24:32.319 --> 00:24:35.200
It's a it's a it's a people and process problem, right?
00:24:35.359 --> 00:24:48.960
Like you have to change the way that you operate, uh the way you view security, the way you you know randomly check um the history of uh the PRs, that needs to change in terms of the way that you build.
00:24:49.279 --> 00:25:04.880
So, you know, at at Fencer, and like when you work with customers, how do you recommend from a process and procedure standpoint that they change out outside of what you just recommended, these random checks, what else needs to change in terms of the way people build software today with AI?
00:25:05.200 --> 00:25:05.519
Yeah.
00:25:05.759 --> 00:25:12.960
I th I think you know there's a lot of talk about things like harness engineering and all these other sort of very AI-specific mechanisms.
00:25:13.200 --> 00:25:25.039
But you know, I I think in a broad sense, it's define all of the rules or preconditions a change needs to meet in order to be considered valid or secure or both.
00:25:26.240 --> 00:25:29.680
And that might just be initially, and that's hopefully a checklist.
00:25:29.759 --> 00:25:35.359
And then you know, for anyone who's gone through a SOC2 process uh or some sort of compliance certification, they've probably already done that.
00:25:35.839 --> 00:25:40.960
And then they've gone and created this um this list of pre-approved changes.
00:25:41.119 --> 00:25:48.079
So under which conditions can something be effectively released to all of our customers and what does it need to show?
00:25:48.240 --> 00:25:55.119
So go through that exercise and use normal words and language, like as a human, uh, nothing particularly technical.
00:25:55.200 --> 00:25:58.720
It's like, hey, all of these tests that we wrote need to pass.
00:25:58.880 --> 00:26:06.000
Uh I need to log in and check every screen in my app to make sure that nothing broke and so forth and so on.
00:26:06.079 --> 00:26:06.240
Right.
00:26:06.319 --> 00:26:17.759
So whatever you would do as a as a person, define those things and work on adding more automation around those things so that you can get to that point where things are automatically merged.
00:26:17.920 --> 00:26:24.880
The second thing is uh making sure you define what the set of things are that do and don't need human oversight.
00:26:26.480 --> 00:26:35.119
If nothing else, that exercise will help identify for you where you may have gaps or holes, right, in terms of your processes and workflows.
00:26:35.839 --> 00:26:53.119
And then once you've done that, you know, make sure that you have an actual like monthly review process, weekly review process, quarterly review process, whatever makes sense for your business or your team, uh, and look through all the things that fell into the, hey, this is pure fully automated.
00:26:53.759 --> 00:27:01.039
Um we will uh go and check them out and make sure that they're uh appropriate, right?
00:27:01.200 --> 00:27:03.359
That our processes and workflows haven't broken.
00:27:03.599 --> 00:27:07.759
And that's I think exclusively a people and process problem, right?
00:27:07.839 --> 00:27:09.119
There's no technology for that.
00:27:10.000 --> 00:27:23.279
Yeah, and I think implicit in all of that is a solid understanding of your business logic and the things that need to be true uh to derive and surface value for your customers.
00:27:23.359 --> 00:27:29.279
And the reason why I emphasize that point is because uh this kind of gets me to my next question.
00:27:29.599 --> 00:27:36.480
When you are bytecoding or just kind of letting AI take the wheel, a lot of those decisions are made for you.
00:27:36.640 --> 00:27:40.640
And it's making a lot of assumptions around, well, I think that this is how it should work.
00:27:40.799 --> 00:27:53.519
And when you lose oversight around what needs to be true, what needs to be secure, even if the AI is writing technically correct code, I think you are opening your uh yourself up without even knowing it to another attack surface.
00:27:53.839 --> 00:27:54.799
Yeah, 100%.
00:27:55.200 --> 00:28:14.400
The other thing to consider is that a lot of what the LLMs have been trained on is publicly available code and publicly available processes, typically open source processes, which are not in and of themselves bad, but they're not you know called the highest corporate standards of security.
00:28:14.640 --> 00:28:28.240
And so what effectively gets encoded in the model that is then used to write code for you is sort of those processes, not what the best companies, you know, with the most secure systems and workflows are doing.
00:28:28.640 --> 00:28:35.920
And and that's that I think nuance maybe gets lost with folks when they're like, oh yeah, Claude, just go review what I what you changed and tell me if it's secure.
00:28:36.079 --> 00:28:43.200
You know, it's missing a huge data set that it could have been trained on, that it isn't trained on, that actually would find all those things.
00:28:43.920 --> 00:28:45.519
That's such an important point.
00:28:45.920 --> 00:28:49.759
Um yeah, so that's the the point I made around vibe coding.
00:28:49.839 --> 00:29:04.319
I'm I'm kind of curious as to your thoughts around, again, this acceleration of AI software creation and the single player modality of people using vibe coding tools and pushing out applications.
00:29:04.480 --> 00:29:13.680
Um, because you know, a lot of the things that we've been talking about is like, you know, an application and a process that has CI.
00:29:13.759 --> 00:29:15.039
They're thinking about those things.
00:29:15.119 --> 00:29:22.880
But there's a lot of applications being pushed by an individual who doesn't have a lot of experience just leveraging a vibe coding tool to do that.
00:29:23.119 --> 00:29:34.000
What are some of the dangers that they might encounter or they should be thinking about as they're pushing a vibe coding app to production or having users start to subscribe and use the product?
00:29:34.400 --> 00:29:34.799
Yeah.
00:29:34.960 --> 00:29:51.039
Um, you know, we again to be shameless, um we have we have tools that help you find a lot of really common things uh that can help you ensure that not just the code, but the other systems that you're built on top of are actually configured correctly.
00:29:51.279 --> 00:29:59.359
And so what I would say is the code itself may not necessarily be the thing uh that you get wrong.
00:30:00.240 --> 00:30:08.400
And you know, AI makes it really easy to generate a lot of code, and that code may actually be secure a lot of the time, and that gives a false sense of security.
00:30:09.200 --> 00:30:15.519
The um recently, it was actually not that recent now, in it's more like a decade in AI years.
00:30:15.759 --> 00:30:21.119
Uh Maltbook, you you remember Maltbook, uh, that is now gonna be part of Facebook, I guess.
00:30:21.200 --> 00:30:27.440
It's gonna, we're just gonna be talking to to open claw agents, malt bots, uh, all the time.
00:30:27.839 --> 00:30:34.799
Um you know what the the the code itself that the team wrote, and it was two guys behind that, by the way, that built um maltbook.
00:30:35.519 --> 00:30:41.039
The code itself wasn't insecure, it was the way that they were using Superbase was the thing that was insecure.
00:30:41.200 --> 00:30:43.359
And so then, you know, they got hacked.
00:30:43.440 --> 00:30:48.799
And you know, the other thing to consider is use platforms that are secure by default, like Xano.
00:30:49.279 --> 00:30:52.799
That will uh ensure that you don't make that kind of mistake.
00:30:52.960 --> 00:30:55.200
Uh but I think that's one area that gets overlooked.
00:30:55.359 --> 00:30:58.079
And I think the second thing is maybe more philosophical.
00:30:58.240 --> 00:31:05.440
You know, I've I've shifted away from thinking of AI and these model tools uh as being perfect.
00:31:05.680 --> 00:31:09.440
They're kind of uh maybe now an above-average employee.
00:31:09.839 --> 00:31:10.000
Right.
00:31:10.079 --> 00:31:16.400
And so if you were working with other humans, would you assume that everything they say and everything they do is always right and always perfect?
00:31:16.720 --> 00:31:17.039
No.
00:31:18.079 --> 00:31:25.920
But we seem to have ascribed that same level of you know precision and accuracy to a tool that we're using.
00:31:26.079 --> 00:31:32.240
Uh and I think that kind of like change and mindset can go a long way to you know solo vibe coders.
00:31:33.519 --> 00:31:38.000
You're you're commanding a fleet of digital employees, they're not right all the time.
00:31:38.079 --> 00:31:42.319
You should check their work and make sure that they're not telling you trash or nonsense.
00:31:44.319 --> 00:31:46.480
Yeah, you know, um it's so funny.
00:31:46.559 --> 00:31:57.599
I just I sat down with uh Doug Merritt from um from Splunk uh a couple weeks ago, and he said something uh that really hits home and is relevant to what you're saying.
00:31:57.759 --> 00:32:03.759
He said, as agents become more human in their behavior, humans have become more binary.
00:32:04.000 --> 00:32:17.279
So we're we're in this moment where the tools are getting smarter, but our thinking is getting shallower, and we're just like instead of that deep synthesis and thinking that humans are great at, we've just been relegated to clicking accept and okay.
00:32:17.440 --> 00:32:17.599
Yeah.
00:32:17.920 --> 00:32:18.400
Yeah, yeah, yeah.
00:32:18.480 --> 00:32:18.640
Yeah.
00:32:18.880 --> 00:32:20.000
It's just so interesting.
00:32:20.079 --> 00:32:24.000
And like we have to like infuse humanity back into the build process, right?
00:32:24.160 --> 00:32:24.559
That's right.
00:32:24.640 --> 00:32:24.960
That's it.
00:32:25.119 --> 00:32:26.319
It's uh uh it's interesting.
00:32:26.400 --> 00:32:39.359
I know if if the way that you interacted with an LLM was through like a video call, uh, where there was some sort of like, you know, call it uh a human-like avatar that you're interacting with, you probably wouldn't trust it.
00:32:39.519 --> 00:32:41.440
Like you wouldn't take it at face value.
00:32:41.680 --> 00:32:51.839
But then because you do it through text and Claude has this very authoritative style, or open, you know, open AI Codex GPT has this authoritative style of writing, you're just like, oh yeah, clearly it's right.
00:32:52.000 --> 00:32:53.920
So yes, accept, move on.
00:32:55.519 --> 00:33:14.640
You know, this is, I mean, maybe this is more philosophical, but one of the things that I've been thinking about is as we have this again of this acceleration of AI writing software, people vibe coding things, we're that means that us as users and as consumers are gonna have just a lot more solutions available at our fingertips.
00:33:14.880 --> 00:33:29.920
And, you know, it it really feels like, at least for me, I would be looking for some sort of security badge or something that says, like, I've built this in this certain way, because most of the time, the the software is looking for credentials.
00:33:30.079 --> 00:33:36.559
It asks you to put in your personal information, your address, your your password that then can be exposed.
00:33:36.640 --> 00:33:43.039
And if it's hacked and if it's not built in the right way, there is a broader ecosystem that is now at risk.
00:33:43.200 --> 00:33:44.480
How do you think about that?
00:33:44.559 --> 00:33:56.160
Like as this acceleration of software uh, you know, I guess proliferates throughout the world that's generally AI generated and trained, like you said, on an open source repository.
00:33:56.480 --> 00:34:00.960
How do you think secure usage looks like in the future as a consumer?
00:34:01.279 --> 00:34:01.680
Yeah.
00:34:01.839 --> 00:34:04.400
Uh, you know, there's that's a big one.
00:34:04.480 --> 00:34:09.840
I think there's a lot of different areas that uh get impacted by that change.
00:34:10.000 --> 00:34:13.199
I think one of them is the underlying supply chain, as we talked about.
00:34:13.360 --> 00:34:19.760
You know, I think what's possible, maybe probable, is more walled gardens.
00:34:20.000 --> 00:34:35.039
So instead of this kind of very federated public way of using open source software, what people end up actually doing is buying open source software from vendors that have done the work to make it secure in the first place.
00:34:35.280 --> 00:34:51.519
There's already a few companies in the world uh starting to do that, you know, where they publish uh hardened Docker images uh so that you can use a piece of software with the belief that this other company has taken the time and effort to make it secure.
00:34:51.760 --> 00:34:56.800
And so I think that plays out in different layers of the supply chain and how we deliver software to people.
00:34:57.039 --> 00:35:00.239
I think identity becomes much, much more important.
00:35:00.320 --> 00:35:15.679
And so there's a there's a draft right now uh in the IETF on how to actually track when agents have been delegated to by a people and how to effectively expose who the person is that the agent has been delegated to.
00:35:15.760 --> 00:35:25.519
So then you can kind of identify: okay, is this somebody acting on behalf, you know, as an agent, little a agent, the way the word agent is meant to be uh understood.
00:35:25.679 --> 00:35:32.639
Um is this some sort of rogue piece of software going wild and maybe doing things that it shouldn't, right?
00:35:32.800 --> 00:35:35.519
So the hacking and whatever else.
00:35:36.800 --> 00:35:39.440
Trust essentially becomes the core component.
00:35:39.599 --> 00:35:50.320
And I think what we start looking for more and more, in the same way that deep fakes make you not trust videos anymore or the news, you know, you're gonna trust software less and less.
00:35:50.480 --> 00:35:52.639
You will, as you said, look for these credentials.
00:35:52.880 --> 00:35:58.960
I think it would be dangerous for any one vendor to say, hey, we, you know, you can trust us to say that we made this secure.
00:35:59.199 --> 00:36:10.639
I think you have encryption protocols, crypto actually probably brings some utility here because it distributes trust in this very uh very secure uh manner.
00:36:10.960 --> 00:36:18.960
And some of these technological things that maybe have not had as much adoption become a uh much more common now.
00:36:19.599 --> 00:36:40.960
I'm curious how you balance that speed with control uh in a world where like it's yes, of course, security is important for a lot of platforms and services, but especially for a security company and platform like Fencer, how do you work responsibly leveraging AI in your day-to-day and with your teams?
00:36:41.360 --> 00:36:45.519
Yeah, uh actually, this has changed literally in the last 30 days.
00:36:45.679 --> 00:36:51.199
If you if we had been chatting a month ago or longer, you know, what I'm about to say would be quite different.
00:36:51.280 --> 00:36:57.440
But we're now probably close to 100% AI driven in terms of code and what have you.
00:36:57.599 --> 00:37:00.000
Uh a month ago, it may have been 40%.
00:37:00.480 --> 00:37:05.119
So, you know, in the span of a month, we've basically gone to the point where nobody really writes any code by hand.
00:37:06.000 --> 00:37:17.679
Um still things, you know, we we don't have an AI agent go and drive a browser for us and click buttons on a screen for us yet, but I'm sure that'll come in time.
00:37:18.000 --> 00:37:24.400
Um But we you know we use tools like Claude and what have you to go and write code for us.
00:37:24.559 --> 00:37:30.239
Uh I think everyone on the team, even before they open a pull request or something like that, will go review the work themselves.
00:37:30.320 --> 00:37:36.800
So we haven't yet gone to that point that I think other teams have where they've abdicated responsibility for the reviewing the code themselves.
00:37:37.039 --> 00:37:38.559
They're just like, okay, it works.
00:37:38.639 --> 00:37:40.800
Uh I wrote the tests, I wrote a spec.
00:37:40.960 --> 00:37:43.519
Great, I'm gonna go, you know, throw it over the fence.
00:37:43.679 --> 00:37:44.400
We don't do that yet.
00:37:44.480 --> 00:37:47.280
We're still um a bit hand more hands-on there.
00:37:47.599 --> 00:37:54.960
Uh we've we have probably 25,000 unit tests and I don't know, several thousand end-to-end tests.
00:37:55.039 --> 00:37:58.079
So we've we've gone pretty wild on writing tests.
00:37:58.239 --> 00:38:09.199
Uh, and so we have a very extensive test suite, a lot of CI processes and and workflows to kind of encode and enforce our standards as much as we can.
00:38:09.360 --> 00:38:14.000
And we were doing that just to stop religious debates between engineers.
00:38:14.159 --> 00:38:18.960
Uh, it just happens that that's paid off for us with with AI agents as well, right?
00:38:19.119 --> 00:38:25.199
No, no one debates whether we need tabs or spaces, and if Claude goes and adds tabs, like that's just caught and we don't have to worry about it.
00:38:25.679 --> 00:38:31.920
Um, but we we've done we did a lot of that work pre-AI, uh, and that has now paid off for us.
00:38:32.079 --> 00:38:37.760
And so when we get to a code review, um we look at humans look at the architecture.
00:38:38.000 --> 00:38:43.199
We spend less time on the stylistic parts of the code, but we still require all code to go through human review at the moment.
00:38:43.599 --> 00:38:52.079
30 days ago, I would have talked to a Tim that had adopted it 40 or where AI is writing 40% of the code.
00:38:52.159 --> 00:38:53.280
Now it's 100.
00:38:53.440 --> 00:38:58.400
And now you're uh at a place where you're still making sure to review everything.
00:38:58.480 --> 00:39:00.960
You haven't fully abdicated responsibilities.
00:39:01.199 --> 00:39:08.639
I'm curious for other engineering leaders listening to this who might be at 0% and they're like, look, I've been holding back.
00:39:09.119 --> 00:39:23.119
What were the gates or goalposts or milestones that made you feel comfortable in increasing, I guess, AI's uh responsibility in writing more and more of your code?
00:39:23.679 --> 00:39:34.079
And then the final piece to that is when will you feel comfortable abdicating your responsibility because it is writing all of the code for you?
00:39:34.400 --> 00:39:34.800
Yeah.
00:39:35.039 --> 00:39:39.519
Uh I think firstly, the just the quality of the models improved drastically.
00:39:39.599 --> 00:39:47.840
So the actual results, it very quickly became clear to me that I didn't need to, you know, closely review every change.
00:39:48.559 --> 00:39:51.920
Uh and so maybe maybe it's a little bit more than a month ago now.
00:39:52.159 --> 00:39:57.519
Certainly since November, there was already a change, but even you know, in the last month or two, it's gotten better.
00:39:57.679 --> 00:40:12.800
Um, the the actual results were very consistent with our quality standards, met you know, the style with which we write code, the the types of patterns that we follow, and things like that, just did a much better job of identifying those things.
00:40:13.039 --> 00:40:30.079
And you know, pre our extreme adoption, I know that what I was skeptical of was whether an LLM would write code in a way that was consistent with how we wrote it as a team, that would follow the standards or would follow the uh architectural patterns that we had established.
00:40:30.239 --> 00:40:43.039
You know, the way that you interact with these sorts of things, or the way that you build a feature X over this way is you know, a certain components get wound together versus just write any old sloppy result.
00:40:43.519 --> 00:40:51.280
Um and I think now that I've seen that it's really good at it, I I think that's what would have held me back a while ago.
00:40:51.519 --> 00:40:59.440
I think the the larger organizations that have restrained people from using LLMs, I think fear two things.
00:40:59.679 --> 00:41:04.000
One, you know, is anthropic or open AI just training on their on their code, right?
00:41:04.079 --> 00:41:13.119
And so they're giving away their secrets, effectively, their core IP, to some third party, which I, you know, you've got to trust the agreements that you sign with companies at some point.
00:41:13.280 --> 00:41:15.360
Um to put that to one side.
00:41:15.599 --> 00:41:22.559
I think the one that I acknowledge when people are appropriately skeptical is the security piece.
00:41:22.719 --> 00:41:41.280
Specifically, will the AI go and find some sort of secret that you've got in an ENV file or a text file or some sort of file on your computer, in your file system, that gives it access to way more than it should, and you don't necessarily see that.
00:41:41.599 --> 00:41:47.760
Uh and I think the only way to overcome that is to sandbox it better, right?
00:41:47.920 --> 00:41:49.360
Run it inside of a container.
00:41:50.079 --> 00:41:53.679
Um because the productivity gains are just undeniable.
00:41:54.000 --> 00:42:01.440
And I think uh the previous skepticism around hallucination, whatever else, like I don't think is justified at all anymore.
00:42:01.760 --> 00:42:04.880
Uh and when you tell it not to hallucinate, it doesn't hallucinate, right?
00:42:05.039 --> 00:42:24.000
So it's uh um uh I I think that that you must, it is a tidal wave, you have to adopt it, and the things that would have made you skeptical one, two, three months ago are no longer there, and now it's just putting the right constraints in place.
00:42:24.159 --> 00:42:35.280
So spend some time, create a little, you know, isolated Docker box, you know, EC2 instance, whatever you use to allow your team to use it, and just go go for it.
00:42:36.000 --> 00:42:36.880
I love it.
00:42:37.119 --> 00:42:46.559
Um the final question I had just before we close is how you think about hiring um and building the right team in today's world.
00:42:47.440 --> 00:42:51.119
Um is that who are the people that you look for?
00:42:51.280 --> 00:42:53.760
Like what are the qualities, what are the traits?
00:42:54.000 --> 00:43:00.880
Um, what type of AI capability do they need to have before you consider them, for example, working at a place like Fencer?
00:43:06.000 --> 00:43:07.920
I don't know if people heard that deep sigh.
00:43:08.159 --> 00:43:09.519
That was a deep sigh.
00:43:09.679 --> 00:43:20.079
I think this is the single hardest question, at least for me right now, the single hardest question uh in in this new world, more so than these like technical uh things that we've been discussing.
00:43:20.960 --> 00:43:25.360
I really, I really don't know what to look for right now in hiring.
00:43:25.440 --> 00:43:37.280
Um, because I think the things that I used to use as good signals of a person's ability and competence and you know success uh are very different in the current world.
00:43:38.320 --> 00:43:43.519
And I think it's you know, I haven't had many opportunities to test it out.
00:43:43.599 --> 00:43:51.199
It's not like we've gone and hired a hundred people in the last three months to really refine what we would actually look for uh to find somebody that's successful.
00:43:51.440 --> 00:43:58.320
A lot of our hiring happened over a year ago um or about a year ago now, and and things have changed drastically in the last 12 months.
00:43:58.480 --> 00:43:59.440
So I don't know.
00:43:59.599 --> 00:44:03.199
I I think certain fundamentals are still just as important as they were before.
00:44:03.440 --> 00:44:09.199
I think certain personality traits are just as important now as they were before.
00:44:09.519 --> 00:44:13.360
And what motivates people just as important now as it was before.
00:44:13.679 --> 00:44:15.840
Are people purely intrinsically motivated?
00:44:16.000 --> 00:44:22.480
Are they only motivated effectively through you know having high compensation or some sort of explicit reward?
00:44:22.559 --> 00:44:26.880
You know, the the things that uh I would look for is obviously a balance.
00:44:27.039 --> 00:44:34.400
Um still makes sense, but in terms of assessing a person's ability to get work done, it's very different.
00:44:34.639 --> 00:44:36.559
And I don't know, I don't have a good answer.
00:44:36.639 --> 00:44:38.480
It's something I'm still trying to figure out.
00:44:38.639 --> 00:44:46.880
Um and I, you know, right now, the way I work a lot of the time is here is a feature description that I wrote down.
00:44:47.119 --> 00:44:48.559
I give it to Claude.
00:44:48.800 --> 00:44:52.400
Claude creates some code, I look at the code, it makes sense.
00:44:52.639 --> 00:44:54.159
Kind of how I would have written it.
00:44:54.239 --> 00:44:54.960
That's good.
00:44:55.119 --> 00:45:00.239
Like if I tested somebody on their ability to do that, like I don't know that many people would fail.
00:45:00.480 --> 00:45:04.079
So there's obviously much more to that.
00:45:04.239 --> 00:45:13.280
Um, and and more importantly, I think now it's about assessing how well someone understands what an LLM is doing, at least in a technology company.
00:45:13.760 --> 00:45:16.079
You're such a uh seasoned engineering leader.
00:45:16.159 --> 00:45:20.159
You've been at engineering and you've led engineering teams, big and small.
00:45:20.320 --> 00:45:23.599
And even you are in a place that we are where we're figuring this out.
00:45:23.679 --> 00:45:37.599
And I actually think that's going to bring a lot of comfort to people that the uncertainty that they feel around who is going to be the highest leverage individual to bring onto your team right now is changing and transforming.
00:45:37.679 --> 00:45:50.000
And I think like right now it's uh, you know, for us, it's kind of like someone with high agency, intrinsically motivated, and just a voracious appetite to learn.
00:45:50.159 --> 00:46:00.880
Um, if you have that as a foundation, I mean, I think that at least, you know, gives you like that initial prerequisite to like, I think that you're going to be successful here.
00:46:01.119 --> 00:46:24.960
But still, I think what we've what we find, especially in engineering, is like the more just fundamental computer science, computer like software engineering principles you foundationally understand, the higher the leverage you are to, for example, leverage something like conductor, use multiple agents uh at the same time, and be productive in a way that someone that doesn't have that muscle memory can't do.
00:46:25.280 --> 00:46:25.679
That's right.
00:46:25.840 --> 00:46:28.960
Um but it's still, I mean, I'm I'm with you.
00:46:29.039 --> 00:46:37.199
And I and I and I'm so glad that you shared that you didn't know the real answer to the question because I think it's something that we're all asking ourselves and trying to figure out.
00:46:37.360 --> 00:46:37.519
Yeah.
00:46:37.679 --> 00:46:43.840
I mean, it's if anybody is out there saying that they definitely know, you know, I I don't know that I believe them.
00:46:44.400 --> 00:46:49.599
People are uh interested in learning more about you, about Fencer, where should they go?
00:46:49.920 --> 00:46:59.599
Yeah, well I'm active on Twitter uh and LinkedIn, uh Timolsch on Twitter, uh Tim Olshansky on LinkedIn, and uh Fencer Fencer Security.
00:46:59.679 --> 00:47:03.360
Uh you can find us online and uh major social channels.
