Michael Georgiou (00:01.61)
Welcome back to Tales from the Pros. I'm Michael Georgiou joined by my co-host Eric Lawrence. Today, we're tackling one of the most underestimated bottlenecks in product development. The time and complexity behind fixing bugs once your app or your product is live. To help us unpack this at a systems level, we've brought on someone who's worked across product engineering, leadership, and now AI infrastructure. Gal Vered is the co-founder of Checksum AI.
Actually, he's the co-founder and CEO of Checksum AI, an AI-driven platform redefining how software testing works. Prior to founding Checksum, Gao served as a CTO and was a product manager at Google. Operating inside one of the most sophisticated engineering environments in the world, he also holds a dual degree MBA and MS in design innovation from Northwestern's Kellogg School of Management with a focus on data analytics.
Gal brings a perspective that blends the technical execution, product strategy, and business impact, which is exactly what this conversation demands. Gal, great to have you with us, man. Appreciate it. Thank you for being here.
Gal Vered (01:12.672)
Yes, thank you for inviting me and I'm excited for the conversation.
Michael Georgiou (01:16.736)
Yeah, sounds good. Yeah, and know, we, with this episode, you know, we always love to just understand your story. I think it's very powerful to give the audience just a kind of a understanding of your background and kind of how you just got to where you are right now, you know, cause obviously you're doing a lot right and you have an amazing product and yeah, just love to hear more about your story.
Eric Lawrence (01:34.678)
So, thank you. you
Gal Vered (01:41.816)
Sure, so I moved to the US six years ago. You mentioned my master's degree here. And then I joined Google as a product leader. At Google, I worked on products that connect businesses with consumers via voice. And the main thing I saw there is how game-changing the Transformers architecture is.
It's not, I didn't know this is what's going to happen to be honest, but it was very clear to me that new use cases are being enabled with AI and ML technology. So we've built technology that the hold for me feature where Google just holds in line for you when you call a business, the call screening where a business calls you, Google is...
Eric Lawrence (02:28.805)
So,
Gal Vered (02:34.412)
your pixel is able to screen the call for you and basically reduce some spam dramatically, verified business calls that verified basically every business that's called you. So we enable a lot of use cases that I think five years ago, cause this was two or three years ago, five years ago were not even being reported possible. So I think it just gave me a glimpse of like, those AI models, the transformer technology really is enabled in your use cases.
And that's where I decided to go for a check-sum and essentially start thinking about how can we make a people ship software faster with AI? And I think the crux of it is quality. Cause if you know, if you write a piece of software and today people can write piece of software very, very fast with AI, the button that becomes, how do I verify that? How do I know it works? I can open cursor right now.
prompted something and I'll get a piece of software in 10 minutes. I'll get exactly what I asked for in theory, but in reality, there's a very high chance it won't work. And even if it works on my machine, there's a very high chance that it won't work when I deploy.
Michael Georgiou (03:46.316)
Kind of like those other tools like lovable and, and base 44, similar, you know, like some vibe coding tools as well. They try to build it, but it's not really exactly what it's not the quality that you always want. Yeah.
Gal Vered (03:57.736)
Exactly. And then you just end up in this back and forth. You can still ship with AI, we ship with AI all the time, but you'll end up in a back and forth you and the AI agent. Hey, this is not what I want. This doesn't work. Fix this thing, fix this thing. So Chexam builds a continuous quality platform that tests every line of code. When you develop locally, before you integrate it on the PR level, before you deploy to production and after you deploy to production. So.
you can just move much faster and know that the code that the AI agent produce works. And we also integrate directly to cursor and cloud code and the rest to give the feedback directly to AI. So it's about increasing the quality, but it's also about increasing the speed significantly because you will basically helping engineers go out of the loop and review ready to go results versus just an intermediary output from cloud code that needs a lot of work before you can actually ship it.
Michael Georgiou (04:33.164)
Okay.
Michael Georgiou (04:51.884)
And just to piggyback off of that, no, that's very interesting. you know, just did your experience at Google, you know, obviously before you started Checksum, were they doing things very differently there? I know AI has skyrocketed in the last like one to two years. I mean, we all, we all know that, right? Especially on the technical side and the creative side as well. But you just at Google before when you were, you know, when you were working there, were things done very differently or did you...
I'm sure you learned so much probably at your time there, right? Because it's Google. They're so good at the product development.
Gal Vered (05:30.7)
I guess very differently, differently from where, because I think if you compare it to checksum, was done very differently on a few levels. A Google being Google and B checksum being a startup and Google being just a big company. And I think what I saw in Google is, and I worked at startups before, so I was very scrappy. How do we get an MVP in two weeks? How do we get something to work so we can sell it? Google, the products we worked on was two or three years in the making.
And they were built based on products that are in technology that was seven, 10, 20 years in the maker, right? Transformers, the famous article I think was published in 2017. So I think Google has a culture of understanding a very big and deep user problem, have very strong conviction that solving it will be a game-changing capability for our customers.
And then be willing to take the time and invest the resources in solving it, even if the path is going to take one or two years. And we see same patterns with open AI, for example, open AI existed for seven years before they had something out there that was actually used by a lot of people. It's not the advice I would give to startups, by the way. And it's not how we operate the checksum, but it is emotion that was new to me and opened my eyes about having big ideas and the patients together. And I think with checksum, we.
Michael Georgiou (06:39.382)
Good point.
Gal Vered (06:54.911)
We balance, we provide something useful to customers today, but we also have a very big vision that we're willing to make a bet and work through it over the course of the years.
Eric Lawrence (07:03.033)
Interesting. Yeah. I guess gal for you because you know, when you departed Google, that was 2021. If you, if you just think about like what AI has been since that point, it's pretty incredible, but I am interested to know.
As far as when you began Checksum, what was the inspiration behind creating the company? Was it just a problem that you observed through your own experience? What kind of led you to the point of saying, hey, this is something that's needed in the market?
Gal Vered (07:35.948)
Yeah, so there's two pieces for it, the day to day piece. I saw testing from both sides. At Google, everything is tested multiple times. I saw the power of it. When you have a code and you can get feedback immediately and you can just ship it and if everything goes wrong, it's being reverted and you have a system that is very unlikely to produce bad.
Relative, obviously there are still bugs with Google products, but relatively very unlikely. And it's very powerful as a developer that you have this safety net. On the other hand, I also saw the amount of time we invest in it. From the moment a feature was ready, it often would have taken one or two weeks of development to get the testing coverage that we were after and the testing practices. So I saw the power of a testing suite and quality in general. And I also saw the costs that you need to spend.
And also worked at startups where we didn't have a robust testing suite and quality. I saw how, you you push something to production and you know, there are going to be bugs and you'll know, some of your customers will report bugs and you're going to fix them. And you're just hoping the ratio is just enough that you can get by and not piss them off too much. And so, yeah, so I think it was very natural to me. like, Hey, if we can do that with AI, if we can give you Google level quality with
Eric Lawrence (08:35.173)
Yeah.
Michael Georgiou (08:47.882)
Yeah, exactly.
Gal Vered (08:57.134)
startup speed, then we can really transform software engineering. And as we saw that AI agents become a thing, or even when you just use ChiaGPT to generate code, it became even clearer to me that I became the feedback look for ChiaGPT or Cloud Code. Like Cloud Code produces code, I run it, I test it, I tell it all of the phases that goes on. So it just became even more clear to me that if you can automate the process, now you're not only allowing developers to ship high quality code.
But now you're allowing AI agents to be autonomous. And I think that if we are after fully autonomous AI engineers, i.e. a system that you just give it a spec and it pushes all the way to production, right? Think about it. You give cursor a task and then it's in production. The only way to get there is with a very strong verification and quality pipeline. That's the only way. No one will push to product. No one will even almost push to production without a very clear report.
And that's the layer checksum is built in. So we want cursor and anthropic focus on improving your agents that write code. That's not where we want to play. But we think that if we're able to verify the code that it's written together, we're basically going to solve software engineering and create an autonomous AI.
Michael Georgiou (10:11.616)
That's awesome. No, that's really cool, man, because, you know, I mean, like I told you before, know, before we kind of, you know, during the introduction, before we started this episode, you know, we're Imagine Ovation, our company's development shop. And, you know, we're definitely very well aware of bugs, you know, obviously you build something for a startup or a, you know, midsize or a larger company. There's going to be bugs, of course, but the thing is, is that you don't want to spend
You're a time just fixing bugs. You want it to be automatic. You want it to be autonomous, right? Something in the background where it can check and verify, validate the issues, give you test cases back and not just have to do manual testing and use human time, right? Where they can be focusing on building the product with whatever programming languages are using, which is going to be a lot of AI as well, like you mentioned. So I love what you guys are doing because it seems like the value that you're providing to the end user.
is faster, better quality. You're saving time and money with kind of engineering power, engineering time. And I think that's ultimately incredibly valuable. And at the end of the day, you want the product to be better and to treat the customer better.
Gal Vered (11:27.894)
Yeah, I agree. And you said something that we tell our customers in our sales motion all the time. The key print two principles in the base of every product we released a checksum is autonomous and continuous and autonomous in the sense that you don't need to pump the model and ask it, Hey, can you write tests for this? Can you do that? Our agents are autonomous. You give them a task. They walk through the end and they just push the code.
into your repository with test is a bit easier to put the code into the repository. Cause if there's a bug with the test, it's not, you know, it's not gonna impact your user experience. So agents are autonomous and they're continuous. Meaning that they're in every step of the software developer life cycle. I think those two principles are extremely important for quality. Cause quality is often, you know, the weakest link in the chain. You want to know that everything is working versus development where you can, developers can be more hands-on with a
Michael Georgiou (12:04.62)
Mm-hmm.
Gal Vered (12:21.954)
with AI agents and guide them through the process.
Eric Lawrence (12:26.039)
Yeah. One thing I noticed with a lot of, you know, owners and CTOs that are dealing with software on their side.
usually like if they're having problems, it's like if there's smoke, there's fire. If they're having bugs and they're trying to fix it, it usually kind of creates more issues. And I know this because there are a lot of times that people give us a call that says, Hey guys, we've worked with a company for a year and we need help getting this across the finish line. It's not where it is. And it can honestly be a lot to understand the code.
know how to fix it. Sometimes they even need a rebuild if the application is such a mess. I'm wondering, do you get that ever? Do business leaders come to you and say kind of like they have a sick kid, they're like, Hey, I have this, it's a mess. Can you plug in what you have to fix what, you know, to fix what we have basically.
Gal Vered (13:23.884)
Yeah, and this, it connects to my past experiences. So one of the biggest motivation we had for checksum is that every time you came to a place that can't ship because of bugs, basically when you fill up your engineers, I'm assuming it's our engineers 90 % of what they do, they file file, right? You push something, there's a bug, you fix this bug, another bug is created, you fix that bug, the third bug is created.
Michael Georgiou (13:44.224)
Yep. Sometimes there's like a regression instead of a progression, you know? Yeah.
Gal Vered (13:49.022)
Exactly. Exactly. And you need in order to escape this loop, you need to, think a testing suite is the best way to escape this loop. You need to have clear feedback about what's working and what's not. And then the developer, even if you put AI for a second, just for like pure engineering perspective, you write a good testing suite, you establish what's working and what's not in the form of code. Then you start to fix through the bugs. And as you fix the bugs, have clear.
Michael Georgiou (14:00.332)
Thank
Gal Vered (14:18.574)
feedback, immediate feedback, because the immediacy is so important in software engineering, immediate feedback of what you broke as you were fixing it and then you can iterate. So that's how putting AI aside, that's how I did it in the past with engineering teams. And I think that with AI it even becomes more relevant. Because again, you can just write code faster. So I 100 % agree, we see it a lot, a lot of customers come to us and they're like, let's just...
make it a bit more sane. Let's have a test entry that at least tells us nobody's gonna call us in 2 a.m. and scream that the app is in progress. And once we establish that, we go into the more, the deeper level of like, okay, how do we make our engineering team super efficient? How do we allow you to ship code in hours instead of days or instead of weeks? So yeah, it really depends on when the customer or where the development team is in their journey to quality.
Eric Lawrence (15:14.319)
makes sense. Yeah, I am interested to know like, as far as a difference, is there a different approach that you see between when somebody comes to you with an engineering team and something that's been coded out versus somebody who might come to you that has a product that's been completely vibe coded?
Gal Vered (15:33.023)
It's a great question. think first of all, our bread and butter, let's say it like this, our customers with real product that is shipped to customers and more, not larger engineering teams. We have customers with five engineers, 10 engineers, but it's usually a team that comes to us and they want to solve quality on a systematic level. I think with vibe coding, there's a...
Michael Georgiou (15:58.731)
Hmm.
Gal Vered (16:03.126)
And by the way, I see the light in my room just there. I'll turn it. You know what? Let me turn it on and then respond to you. I don't know if you can see, but it drives me crazy. Give me a second.
Michael Georgiou (16:09.151)
Yeah, that's fine. Yeah.
Eric Lawrence (16:09.551)
Yeah, that's It gives you kind of an angelic look.
Michael Georgiou (16:12.725)
Yeah, no worries. Yeah, that's true.
Gal Vered (16:15.65)
Yeah.
Michael Georgiou (16:21.343)
Yeah, we can have Rahul edit this. It's fine. No worries.
Gal Vered (16:25.696)
Okay, so do you want to ask me this question again? Do you want to just cut it in a way I know you probably can?
Eric Lawrence (16:28.761)
Yeah, I can ask it again. As far as when somebody comes to you and they bring a product to you, what we see is it could either be custom built. Maybe it's an application or a piece of software that's been custom coded by engineers versus there are lot of people out there now that are using tools to vibe code to have AI completely write their products, how you handle both of those approaches.
if there's any differences that you see and how you go about testing those.
Gal Vered (16:59.628)
Yeah, that's a great question. think first of all, typically our customers, I wouldn't say only on the enterprise level, we actually see a lot of demand for quality across. have engineering teams of five and 10 people. And we also work with Fortune 500 companies that, you know, we basically provide quality to the entire organization. And it does usually start when you have a few engineers on the team. And with that being said, I think we...
vibe coded applications, it's usually very high. The goal is to get very high signal to noise ratio. So it's not about covering thousands and tens of thousands of different scenarios. Cause usually those apps are simpler and they don't have, you know, hundreds of millions of dollars of revenue every year running on them. So it's more, how do I make a change and just know things working so I can move a bit faster and I don't need to, you know.
worry and do this manual testing every time I make a small change. With larger accounts, or even when you go to five, 10 developers, there's a lot of surface to test. If you think about the Fortune 500 company that is in financial services and we do all of their UI and API testing, that's a huge surface. And then the activity there is mainly about connecting the different systems together. Also, you don't want to test
click on a button, button works. Click on another button, the button works. One of our premises at Checksum is that our tests are mimicking real usage. So we need to connect across all of those systems to make sure that there's a big unit, everything works and not, you know, some one endpoint works, but then in reality, the bugs are mainly in the integrations and the complexity of production, not in the one unit analysis.
Eric Lawrence (18:51.703)
That's interesting. Yeah, no, that's that's helpful to kind of like break down. I think what's interesting is a lot of this can go over the heads of
people that aren't very technical. So one of the things that we definitely wanted to discuss with you that gal is when you sit down with business owners and you just kind of discuss the impact that having these bugs over time and the compound effect that it makes can happen. what, what do you tell these people? What, what sort of impact can these things have on a business over time?
Gal Vered (19:29.676)
That's a really good question. And I think it changes. This is the place where it changes the most every month. So it starts with, because of the hype on Twitter and on LinkedIn, which I think a lot of it, it's not fake, but a lot of it is on experimental projects, not real enterprise. Everyone is feeling behind. Everyone is reading how
Michael Georgiou (19:43.785)
Yeah.
Gal Vered (19:58.058)
Anthropic rebuilt the C compiler, I think it was in like one day and they're like, but I, this feature takes me this very simple feature takes me two weeks and Tropic just built.
Michael Georgiou (20:08.011)
You're so right. just wanted to say I cut a thousand percent agree with you. had to say it because I see this stuff on LinkedIn everywhere. Anyways, go ahead. I'm just like, and I'm not technical like you, but I'm like, what? And there's no way. It's almost too good to be true. Some of the stories that are shown and they give this narrative that it's like, it's just this huge enterprise, amazing piece of software could just be built in a month. it's like, it's not, maybe one day, I'm sure it will. AI is going.
Eric Lawrence (20:10.24)
Yeah
Michael Georgiou (20:36.072)
rapid, right? But I always believe there's always going to be a human component that's attached to the greatness of a product. But anyways, go ahead. I interrupted.
Gal Vered (20:45.612)
No, I 100 % agree. I think everyone is feeling behind. I think the hype on Twitter is real in a sense that those AI models are really amazing at what they can do. But also, it's shipping real enterprise software still takes time, still takes time building, thinking through this.
And, know, again, the best example is Anthropic has like 100 software engineering positions open, not 100 positions filled, 100 positions open right now. So, you know, you still need engineers and there's still a lot of humans in the process, but everyone's feeling behind and there's two, like everyone's feeling behind because they're not feeling like they move fast enough. And at the same time, they use AI agents and they see the errors it produce, even if it's great again, and we use AI.
Michael Georgiou (21:16.492)
Crazy, yeah.
Gal Vered (21:37.878)
All the time, I don't write code anymore by hand. Everything I tell cloud code to write. Like I'm very hands-on with writing code, but the code is still generated with the eye. So there's a huge impact. Don't get me wrong, but at the same time they use it and they see the errors they provide and they see how much guidance they need to provide. So there's a lot of like fear about, we're not moving fast enough, but on the other end, I'm worried that our engineers will just use those tools without any control. Cause I see that it produces.
20, 30, 50 % slope code, however you wanna call it, but it produces errors. So I think CTOs, VPRs, engineering leaders come to check some with those two things in mind. It's like, how do we move faster? How do we, but how do I also make sure that my engineers aren't becoming, aren't pushing code that maybe kind of works today, but in a week or in a month, in one year, we're gonna pay for it because it's gonna be spaghetti code that's unmaintainable. And so that's kind of like the concerns they have.
And then in terms of impact, we think about it in two ways. One is kind of just make it very easy to say yes is the cost savings. So we have one big company, had 40 QA automation engineers before they hired Chexam. Now all of the QA automation engineers were converted into software engineers. So they have zero QA automation engineers. They all do software engineering. Chexam is responsible for quality. So that makes it very easy to say yes because
Eric Lawrence (23:01.413)
Hey.
Michael Georgiou (23:04.041)
That's awesome.
Gal Vered (23:04.931)
because of the cost savings. But the benefit is not the cost savings, it's just the ability to ship much faster and delight your customers.
Michael Georgiou (23:14.217)
And it probably, you know, also I can imagine that, you know, with just the improvements on the entire, just QA testing side, right? That your product delivers to the customer. It has, I think, an important mental component, right? Where it'll release a lot of just overwhelm and anxiety and stress that we all feel, right? When we're trying to build something and...
you know, and, cause we, we care and we want it to be successful. We have all these expectations, right. but at the same time, you know, I think it can really, just make a positive impact on, know, on the end users, on your customers, but also on their customers, whoever they're, you know, whatever they're trying to build, whatever, know, whoever their customers are basically is that it can just provide a lot of top value, mentally, you know,
to the engineering teams, to the founders, to the customers, internally, externally. I think there's a very huge component there. It's just the mental side of it.
Gal Vered (24:23.916)
I agree, I can give you an example of, so we on boarded a customer, I think it was two weeks ago, but I might get the time wrong. And like after 24 hours of using Checksum, they got back to us and say, this platform is amazing, it allows us to ship faster, we really love it. I got on a call with them, because it's such a strong feedback in short time and also proactive feedback is always something I want to understand, what was the magic moment, right? How do we...
encapsulated into a product and making it even more obvious. And I asked them like, how many bugs did we found for you? I assumed like we probably found like a few major bugs and they're like, no, you didn't find any bugs. So I'm like, so what's the, where's the reaction coming from? Like, well, we just created one massive PR and I can see all of the tests that Jackson generated and I can see they all pass. So I just feel like much more confident to.
Michael Georgiou (25:02.954)
if
Gal Vered (25:18.424)
push it and a PR that I would probably let it sit for like a few hours, maybe like a few days, review it. Checksum just allowed me to just merge it. And I think that's something we, today we're not surprised from it, but that's something that, you know, generally surprised us in the beginning where we thought our value is finding bugs, but our value is in finding bugs, but also confidence. Like knowing that there is no bugs in the PR, especially in the AI agent world is as important.
is finding the bugs that are there because again it allows you to move forward.
Eric Lawrence (25:49.852)
Yeah, I can see the value 100%. I we know from experience that usually it's the last 10 % of development that can feel like 90 % of the work because there's just so much that
Michael Georgiou (25:52.671)
Yeah. Yeah.
Gal Vered (26:00.076)
Exactly.
Eric Lawrence (26:03.249)
needs to get polished, that needs to be tested, that needs to just make sure at the end of the day, does everybody have confidence to push the update and to get things live?
Gal Vered (26:14.818)
Yeah, I agree. And that's the difference by the way, Ben, between a testing and reviewing, which is interesting because PR review tools are again, super helpful. We use them internally as well, but PR review tools, they catch bugs and you fix them. But if they don't catch bugs, it's still a weak signal that there are no bugs.
Michael Georgiou (26:32.255)
And who is it, who are you talking about? Who was it?
Gal Vered (26:35.81)
I think there are a lot of PR review testing tools out there, if an AI reviews your PR, or you can usually just ask Claude and Kerser to review the code, I think it gives you lot of confidence if a bug was found to fix it. If there is no bugs, I don't think it gives you confidence that there are no bugs, if you get what I'm saying, because it's more about finding a few bugs that exist less about confidence, and what we see is again, in order to really move fast, you need both.
Michael Georgiou (26:38.22)
PR review, okay, okay, yep, got it.
Michael Georgiou (26:45.803)
Mm.
Michael Georgiou (26:54.858)
Yes.
Gal Vered (27:03.65)
find the bugs, but also known that when there are no bugs, you can trust it and deploy.
Michael Georgiou (27:09.545)
Yeah, no, that's cool. Yeah, no, that's great. Yeah, I definitely think it's a conversation we can talk with my partners and definitely try to test out, check some AI as well. I think it can provide a lot of value to a software agency like us as well. Did you have something you wanted to say,
Eric Lawrence (27:34.306)
Yeah, I only had really one other question gal for you about
Just the process of getting these launched in specifically around your time as a CTO. I'm interested not about like the numbers of being able to launch something and make sure that the bugs are down and that the QA process goes smoothly. I'm wondering what does release anxiety do to an engineering team culturally? Because I'm sure you've seen that firsthand.
Gal Vered (28:07.126)
Yeah. So are you asking from the perspective of AI or just in general?
Eric Lawrence (28:11.855)
just in general from your experience.
Gal Vered (28:14.69)
Yeah. So I've been in organizations again, that you're worried about clicking the publish button. I've been in the, in the previous company that I was the city of when I joined. They used, know, the typical saying is never the flow and fighting, right? Because you don't want to have an outage on the weekend for them. It was the opposite. It was always when we, when I joined, it was very clear that there will be bugs and issues. And again, that was like the first priority. So.
they actually deployed on Friday, because they knew there are gonna be bugs and they knew on weekend, like the usage is low. And it was also a startup. So they plan Friday, you have a few customers that without your bugs, you have two days to use your software a bit, and then you can fix the bugs when customers get to the office on Monday, the software will already be fixed. So it was very, very hard. again, testing was, there was a...
A lot of changes we made in order to solve it, both in the code architecture, et cetera. And I think testing was a big part of it.
Michael Georgiou (29:20.638)
No, it's cool. know, and just like with speaking a little bit just with AI again, you know, it's just because there's so many things changing, Gals, you know, with not just obviously, you know, Checksum has incorporates a lot of AI. I mean, I don't know the ins and outs of the platform, but obviously, you know, you guys have AI within, you know, powerful AI within your platform. And I'm assuming it's from what you said before, it is agentic.
Is that right? Is that correct? Yeah. So, you know, do you think it would be different if tools like yours did not exist, right? Where if you have really, really good engineers, QA engineers and actual developers who are implementing the code and fixing things, and then they're catching issues and then they're fixing it. Do you think, you know, have you seen like
Gal Vered (29:50.318)
Yes.
Michael Georgiou (30:19.188)
a massive, massive shift in just terms of the kind of the overall, obviously speed is a big component, like you mentioned, but even just the quality of it, is that the whole purpose of it? Is the quality of Checksum and even some other tools, it's just that powerful that it catches things, it reports it back to the engineers, they know exactly what it's reported, it kind of gives an audit. Is that correct? Is that kind of how?
Gal Vered (30:46.23)
Yeah, it's correct. And I think, I don't know if you, I'd love to get your opinion, but with us, feel like when you use AI agents to write the code, you don't have the same level of control of the details. something about writing is thinking even beyond code, right? When you write something, you think through it, you think through all of the different angles. And once AI writes the code, like you give the high level, high level objective and the
and what you're trying to achieve and maybe even some architecture. But once AI writes the code, you lose the level of thinking in the tiny details in the edge cases and you kind of can't for the AI agent to do it. That's why even if you review the code very diligently, you're never gonna reach the same level of quality you had had you written data thing yourself. On top of it, think people attention span is getting really short in general these days.
Michael Georgiou (31:42.763)
Very true. Yep.
Gal Vered (31:43.534)
And with AI, it's probably getting even worse. So I even worse. So, I imagine most of engineers, when they get AI code, you scheme it, you read it, but you don't read every line of code, think through this one different language. And so, yeah, I think it's, again, we want to enable people to ship with AI. We want to enable people to scheme AI code and trust it. So that's, I'm not against this trend. I think this is the trend we need to go through in order to realize AI potential. We just think.
There's a missing part of this equation that checks and provides.
Michael Georgiou (32:14.869)
Yeah, no, I think that's a great answer. That was kind of where I was getting at is that, you know, there's just a, think, at least from, not just, how do I say this? Not just what we're seeing with needs from customers that we work with or prospects or what I'm seeing trending through, not just social, but just in the world in general, in tech and business, is that it's like,
There's this like high reliance on AI and I'm guilty of it too with a lot of things. It's hard, right? Cause sometimes you feel stupid even using it so much and you're like, you can't, you don't, you want to think about things that your brain, cause you're, we're smart. The human brain is incredibly, I mean, yeah, it's a whole different discussion, but it's incredibly intelligent, the brain, right? Especially with practice and, and, in training and experience, and development obviously, but basically, you know,
I've noticed like there's a massive kind of reliance on AI. And I love what you said in terms of when people, developers, or even project managers, when they review a product, right? They think about edge cases and unique cases, right? That AI, at least right now, is not...
It's not gauging, it's not capturing, right? And I'm sure it'll get there. mean, God knows where AI is going to be in a couple of years or five or 10 years. But at least, you know, at the moment, I think there's going to be a lot of open doors and new opportunities for people, along with technology and the improvements of tech. And I think it's just going to, as long as we use it with the right intentions and we try to do things the right way to help people and to help our customers, to help our families and others in the world.
I think there can be so much value with kind of, you know, just the balance of using tech and using, you know, AI specifically, and then obviously, you know, allowing human beings to provide the value that they have where AI may not be able to fully capture and provide value to, if that kind of makes sense. It's like a balancing effect.
Gal Vered (34:31.446)
I agree. How do you think about development with AI on your side and the human in the loop? Are you encouraging people to use as much AI as possible? you trying to create some culture that still keeps the human in the loop and kind of making engineers accountable? How do you think about shipping with AI moving fast, but at the same time, do it in a way that delights your customers, not just...
So all of the AI problems at your customers.
Michael Georgiou (35:01.835)
Yeah, no, I mean, that's a great question. I see it kind of internally and externally, like internally, at least for our developers. mean, have over Eric, what, almost probably over 40 engineers now. And we use AI as well. And we use all the cursor and have Anthropic and I think even Copilot, I believe, writes code too. Yep.
Eric Lawrence (35:17.316)
Yep.
Michael Georgiou (35:31.463)
obviously open AI. so, you know, and we've integrated a lot of kind of agentic platforms within our processes and all that. And I can't speak for my business partners and that you are a lot more technical than I am, but I can tell you that it is embedded in the culture because if it, if it does save time, and it improves quality kind of like your product does exactly, then yeah, of course it makes sense to, onboard it and to
incorporate it within the entire kind of culture of the organization, not just with engineering and delivery and support and QA and even design, but it's also with marketing and sales. try to incorporate it, but at the same time, we know where it's at right now because we use it so much. we try to also make sure that we have the personal touch and we have the human element. That's very important.
So it's, like I said, it's kind of a balance, at least internally, you know, and it is improving the quality of our code. is doing things like that that are insanely valuable. Externally, I'll let Eric answer that kind of on the sales side, what he's seeing with AI. You know, I think it's moving also very quickly, just like with the demand of it.
Eric Lawrence (36:51.756)
Yeah, I would agree with that. I know it seems like for two thirds of every new project that we engage with, at least think of it from web applications to, mobile apps, to custom software. People want some sort of AI to be a part of it. And I think from the most basic sense, you can simply integrate with Claude or open AI or something there. But one of the things that we've been doing for a lot of different projects is
When you first launch an app, you want to have a model out there that runs with it. But if you have your own special amount of data that you have on your side, on the background, you have your own model that you're training and getting up to speed to the point where it's good enough that it can eventually be swapped out for whatever AI model you have in there so that it's going to be unique to your own product. That's at least what we're seeing on the product side.
Gal Vered (37:48.022)
I agree. And I think one of the challenges, especially in coding and I'm sure in other places is a short term versus long term. Like I can write a feature or let's say I use the AI to produce code to write a feature. And this feature works right now. Like it's fully working. I tested it, but the code is not architected in the best way. And then there's always the question of, I move fast? Do I ship it and probably pay later when.
my code is spaghetti and it's very hard to make changes. Do I take the time to write it well and again, prompt AI to improve it and then enjoy the benefits later. And the problem with AI is that it's compounding because if you have bad patterns in your code base, the new code that will be generated is very similar to the existing code because the agent reads the files and picks up on patterns.
Michael Georgiou (38:35.392)
Yeah.
Gal Vered (38:44.098)
So it's just compounding. It's not an easy problem to solve. And that's why we see a lot of Vibe-coded applications. They spin up an MVP very fast and then they hit a roadblock where you just can't introduce new features because nothing is working. Like the code is so messy that it's impossible to introduce a new feature, especially with AI. Maybe if you read the code and you think throughout to do it, you can maybe get past it. So.
Michael Georgiou (39:09.235)
It increases the technical debt even more than before, sometimes. Yeah.
Gal Vered (39:12.244)
Exactly. more than before, because it's compounds. Because again, if you have bad pattern in your repository, the AI is going to repeat it again and again and again. And if you have good patterns in your repository, then the code that the AI will produce will be better than it. It will still not be always perfect, but the patterns that we've incorporated there will be the correct pattern.
Michael Georgiou (39:21.855)
Hmm.
Eric Lawrence (39:36.628)
Great. Well, Gal, I only have one other question for you. I want to think future. In five years, what will feel embarrassing about how we handle bugs today?
Michael Georgiou (39:49.739)
You
Gal Vered (39:51.438)
That's a good question, I think.
Michael Georgiou (39:58.315)
.
Gal Vered (39:58.922)
In five years, I think what would be embarrassing or feel outdated is the fact that the bugs were shipped in production regularly and the fact that testing was just being done by testing, by manually opening a browser and clicking around. The thing we're building at Checksum and our vision is to basically build a simulation to how code works in production. So the idea of you needing to test your code.
manually or not even manually automated against scenarios, we want to eliminate it completely because we want to allow you to deploy your software into a digital clean of your production environment, see exactly all of the problems and fix them. Maybe the best analogy is like autonomous cars. When Google has a new model to release for their autonomous cars, they don't go out there, release it and see if the car, even like on a fake street and see if the car.
you know, hit fake pedestrians, right? Definitely not real pedestrians. They have millions of simulations that they first run their model through that simulates real behavior and they can see the model decision and they can measure everything. And then they can take it into, I'm assuming, a street in a very controlled environment. And then they deploy it out there for real cars in San Francisco to drive autonomously. I think today it doesn't exist in software. Everything is just tests that are imagined by people. And what we want to do is now let us
Michael Georgiou (40:59.626)
Yeah.
Eric Lawrence (41:15.716)
Brilliant.
Michael Georgiou (41:23.788)
Mm-hmm.
Gal Vered (41:26.83)
Create a simulation of your real production environment, run it through millions of scenarios, get the feedback within minutes, know exactly what's working and what's not working to a very high accuracy.
Michael Georgiou (41:37.791)
Yeah, you don't just want to launch a product to the public when it's clearly not ready, right? They're probably, especially big companies like Google, Apple, Microsoft that are testing these things to the moon, they are probably past the validation phase for a lot of their products. So once they develop the product and they're...
almost ready to launch and almost could mean like a year or two down the road, right? For them, it's like, oh, we're almost there. And it's like a year or two or three, at least from what I've read and what I understand, it's a long-term thing, right? Like you mentioned with OpenAI, they were developing it seven or eight or nine years, even before we even started hearing about it. And we had no freaking clue. No one knew what was actually in the works. But no, like I think what you said is very valuable, you You gotta make sure that...
The quality of the testing is very strong because you have to look at all the different edge cases and that's probably what a lot of these companies are doing. mean, we even do it, you guys do it, right? I check some internally, right? You want to make sure that before you launch a release or a new product, you want to make sure that it's heavily, heavily tested, right? With the user base or even a closed group before you just put it out there and then it could just fail. And once it gets a bad reputation, it's hard to come back from it.
Gal Vered (43:03.905)
I hug.
Michael Georgiou (43:04.809)
You know, but no, this has been great. Yeah, Gal, I think that's all we have for you today, man. Really appreciate you. Thank you for taking the time to just chat with us about this. you know, just so everyone knows, you know, Eric and I are not on Gal's level in regards to QA. This is why he's here. You know, we want someone like him here and people like him who know exactly what's going on in kind of the world of
Eric Lawrence (43:26.658)
Yeah.
Michael Georgiou (43:34.155)
QA and development and just technology and what's going on at the moment and where things are headed. And we just want to serve others and help people as much as possible in terms of just their learning and inspiration and what they can gain from this. So thank you very much, Gal. Really appreciate you. And can you share with everyone where everyone can find you, your website, social handles?
Gal Vered (43:58.742)
Yeah. First of all, thank you for having me. It was a great conversation. Check some you can find on checksum.ai. So very easy to remember. I'm mainly active on LinkedIn. You can just search Gal Vered. I sometimes post on Twitter, but in reality, not as much, but feel free to check me out there as well.
Michael Georgiou (44:19.884)
Awesome. Sounds good. Thank you so much, Gal. Thank you everyone for listening. And I'm your host, Michael Georgiou from Tales from the Pros, along with my cohost, Eric Lawrence, and our amazing guest, Gal Vered Thank you so much, guys. Really appreciate it. Thank you, Gal. All right, guys. All right. Thanks.
Gal Vered (44:34.434)
Thanks, bye.
Welcome back to Tales from the Pros. I'm Michael Georgiou joined by my co-host Eric Lawrence. Today, we're tackling one of the most underestimated bottlenecks in product development. The time and complexity behind fixing bugs once your app or your product is live. To help us unpack this at a systems level, we've brought on someone who's worked across product engineering, leadership, and now AI infrastructure. Gal Vered is the co-founder of Checksum AI.
Actually, he's the co-founder and CEO of Checksum AI, an AI-driven platform redefining how software testing works. Prior to founding Checksum, Gao served as a CTO and was a product manager at Google. Operating inside one of the most sophisticated engineering environments in the world, he also holds a dual degree MBA and MS in design innovation from Northwestern's Kellogg School of Management with a focus on data analytics.
Gal brings a perspective that blends the technical execution, product strategy, and business impact, which is exactly what this conversation demands. Gal, great to have you with us, man. Appreciate it. Thank you for being here.
Gal Vered (01:12.672)
Yes, thank you for inviting me and I'm excited for the conversation.
Michael Georgiou (01:16.736)
Yeah, sounds good. Yeah, and know, we, with this episode, you know, we always love to just understand your story. I think it's very powerful to give the audience just a kind of a understanding of your background and kind of how you just got to where you are right now, you know, cause obviously you're doing a lot right and you have an amazing product and yeah, just love to hear more about your story.
Eric Lawrence (01:34.678)
So, thank you. you
Gal Vered (01:41.816)
Sure, so I moved to the US six years ago. You mentioned my master's degree here. And then I joined Google as a product leader. At Google, I worked on products that connect businesses with consumers via voice. And the main thing I saw there is how game-changing the Transformers architecture is.
It's not, I didn't know this is what's going to happen to be honest, but it was very clear to me that new use cases are being enabled with AI and ML technology. So we've built technology that the hold for me feature where Google just holds in line for you when you call a business, the call screening where a business calls you, Google is...
Eric Lawrence (02:28.805)
So,
Gal Vered (02:34.412)
your pixel is able to screen the call for you and basically reduce some spam dramatically, verified business calls that verified basically every business that's called you. So we enable a lot of use cases that I think five years ago, cause this was two or three years ago, five years ago were not even being reported possible. So I think it just gave me a glimpse of like, those AI models, the transformer technology really is enabled in your use cases.
And that's where I decided to go for a check-sum and essentially start thinking about how can we make a people ship software faster with AI? And I think the crux of it is quality. Cause if you know, if you write a piece of software and today people can write piece of software very, very fast with AI, the button that becomes, how do I verify that? How do I know it works? I can open cursor right now.
prompted something and I'll get a piece of software in 10 minutes. I'll get exactly what I asked for in theory, but in reality, there's a very high chance it won't work. And even if it works on my machine, there's a very high chance that it won't work when I deploy.
Michael Georgiou (03:46.316)
Kind of like those other tools like lovable and, and base 44, similar, you know, like some vibe coding tools as well. They try to build it, but it's not really exactly what it's not the quality that you always want. Yeah.
Gal Vered (03:57.736)
Exactly. And then you just end up in this back and forth. You can still ship with AI, we ship with AI all the time, but you'll end up in a back and forth you and the AI agent. Hey, this is not what I want. This doesn't work. Fix this thing, fix this thing. So Chexam builds a continuous quality platform that tests every line of code. When you develop locally, before you integrate it on the PR level, before you deploy to production and after you deploy to production. So.
you can just move much faster and know that the code that the AI agent produce works. And we also integrate directly to cursor and cloud code and the rest to give the feedback directly to AI. So it's about increasing the quality, but it's also about increasing the speed significantly because you will basically helping engineers go out of the loop and review ready to go results versus just an intermediary output from cloud code that needs a lot of work before you can actually ship it.
Michael Georgiou (04:33.164)
Okay.
Michael Georgiou (04:51.884)
And just to piggyback off of that, no, that's very interesting. you know, just did your experience at Google, you know, obviously before you started Checksum, were they doing things very differently there? I know AI has skyrocketed in the last like one to two years. I mean, we all, we all know that, right? Especially on the technical side and the creative side as well. But you just at Google before when you were, you know, when you were working there, were things done very differently or did you...
I'm sure you learned so much probably at your time there, right? Because it's Google. They're so good at the product development.
Gal Vered (05:30.7)
I guess very differently, differently from where, because I think if you compare it to checksum, was done very differently on a few levels. A Google being Google and B checksum being a startup and Google being just a big company. And I think what I saw in Google is, and I worked at startups before, so I was very scrappy. How do we get an MVP in two weeks? How do we get something to work so we can sell it? Google, the products we worked on was two or three years in the making.
And they were built based on products that are in technology that was seven, 10, 20 years in the maker, right? Transformers, the famous article I think was published in 2017. So I think Google has a culture of understanding a very big and deep user problem, have very strong conviction that solving it will be a game-changing capability for our customers.
And then be willing to take the time and invest the resources in solving it, even if the path is going to take one or two years. And we see same patterns with open AI, for example, open AI existed for seven years before they had something out there that was actually used by a lot of people. It's not the advice I would give to startups, by the way. And it's not how we operate the checksum, but it is emotion that was new to me and opened my eyes about having big ideas and the patients together. And I think with checksum, we.
Michael Georgiou (06:39.382)
Good point.
Gal Vered (06:54.911)
We balance, we provide something useful to customers today, but we also have a very big vision that we're willing to make a bet and work through it over the course of the years.
Eric Lawrence (07:03.033)
Interesting. Yeah. I guess gal for you because you know, when you departed Google, that was 2021. If you, if you just think about like what AI has been since that point, it's pretty incredible, but I am interested to know.
As far as when you began Checksum, what was the inspiration behind creating the company? Was it just a problem that you observed through your own experience? What kind of led you to the point of saying, hey, this is something that's needed in the market?
Gal Vered (07:35.948)
Yeah, so there's two pieces for it, the day to day piece. I saw testing from both sides. At Google, everything is tested multiple times. I saw the power of it. When you have a code and you can get feedback immediately and you can just ship it and if everything goes wrong, it's being reverted and you have a system that is very unlikely to produce bad.
Relative, obviously there are still bugs with Google products, but relatively very unlikely. And it's very powerful as a developer that you have this safety net. On the other hand, I also saw the amount of time we invest in it. From the moment a feature was ready, it often would have taken one or two weeks of development to get the testing coverage that we were after and the testing practices. So I saw the power of a testing suite and quality in general. And I also saw the costs that you need to spend.
And also worked at startups where we didn't have a robust testing suite and quality. I saw how, you you push something to production and you know, there are going to be bugs and you'll know, some of your customers will report bugs and you're going to fix them. And you're just hoping the ratio is just enough that you can get by and not piss them off too much. And so, yeah, so I think it was very natural to me. like, Hey, if we can do that with AI, if we can give you Google level quality with
Eric Lawrence (08:35.173)
Yeah.
Michael Georgiou (08:47.882)
Yeah, exactly.
Gal Vered (08:57.134)
startup speed, then we can really transform software engineering. And as we saw that AI agents become a thing, or even when you just use ChiaGPT to generate code, it became even clearer to me that I became the feedback look for ChiaGPT or Cloud Code. Like Cloud Code produces code, I run it, I test it, I tell it all of the phases that goes on. So it just became even more clear to me that if you can automate the process, now you're not only allowing developers to ship high quality code.
But now you're allowing AI agents to be autonomous. And I think that if we are after fully autonomous AI engineers, i.e. a system that you just give it a spec and it pushes all the way to production, right? Think about it. You give cursor a task and then it's in production. The only way to get there is with a very strong verification and quality pipeline. That's the only way. No one will push to product. No one will even almost push to production without a very clear report.
And that's the layer checksum is built in. So we want cursor and anthropic focus on improving your agents that write code. That's not where we want to play. But we think that if we're able to verify the code that it's written together, we're basically going to solve software engineering and create an autonomous AI.
Michael Georgiou (10:11.616)
That's awesome. No, that's really cool, man, because, you know, I mean, like I told you before, know, before we kind of, you know, during the introduction, before we started this episode, you know, we're Imagine Ovation, our company's development shop. And, you know, we're definitely very well aware of bugs, you know, obviously you build something for a startup or a, you know, midsize or a larger company. There's going to be bugs, of course, but the thing is, is that you don't want to spend
You're a time just fixing bugs. You want it to be automatic. You want it to be autonomous, right? Something in the background where it can check and verify, validate the issues, give you test cases back and not just have to do manual testing and use human time, right? Where they can be focusing on building the product with whatever programming languages are using, which is going to be a lot of AI as well, like you mentioned. So I love what you guys are doing because it seems like the value that you're providing to the end user.
is faster, better quality. You're saving time and money with kind of engineering power, engineering time. And I think that's ultimately incredibly valuable. And at the end of the day, you want the product to be better and to treat the customer better.
Gal Vered (11:27.894)
Yeah, I agree. And you said something that we tell our customers in our sales motion all the time. The key print two principles in the base of every product we released a checksum is autonomous and continuous and autonomous in the sense that you don't need to pump the model and ask it, Hey, can you write tests for this? Can you do that? Our agents are autonomous. You give them a task. They walk through the end and they just push the code.
into your repository with test is a bit easier to put the code into the repository. Cause if there's a bug with the test, it's not, you know, it's not gonna impact your user experience. So agents are autonomous and they're continuous. Meaning that they're in every step of the software developer life cycle. I think those two principles are extremely important for quality. Cause quality is often, you know, the weakest link in the chain. You want to know that everything is working versus development where you can, developers can be more hands-on with a
Michael Georgiou (12:04.62)
Mm-hmm.
Gal Vered (12:21.954)
with AI agents and guide them through the process.
Eric Lawrence (12:26.039)
Yeah. One thing I noticed with a lot of, you know, owners and CTOs that are dealing with software on their side.
usually like if they're having problems, it's like if there's smoke, there's fire. If they're having bugs and they're trying to fix it, it usually kind of creates more issues. And I know this because there are a lot of times that people give us a call that says, Hey guys, we've worked with a company for a year and we need help getting this across the finish line. It's not where it is. And it can honestly be a lot to understand the code.
know how to fix it. Sometimes they even need a rebuild if the application is such a mess. I'm wondering, do you get that ever? Do business leaders come to you and say kind of like they have a sick kid, they're like, Hey, I have this, it's a mess. Can you plug in what you have to fix what, you know, to fix what we have basically.
Gal Vered (13:23.884)
Yeah, and this, it connects to my past experiences. So one of the biggest motivation we had for checksum is that every time you came to a place that can't ship because of bugs, basically when you fill up your engineers, I'm assuming it's our engineers 90 % of what they do, they file file, right? You push something, there's a bug, you fix this bug, another bug is created, you fix that bug, the third bug is created.
Michael Georgiou (13:44.224)
Yep. Sometimes there's like a regression instead of a progression, you know? Yeah.
Gal Vered (13:49.022)
Exactly. Exactly. And you need in order to escape this loop, you need to, think a testing suite is the best way to escape this loop. You need to have clear feedback about what's working and what's not. And then the developer, even if you put AI for a second, just for like pure engineering perspective, you write a good testing suite, you establish what's working and what's not in the form of code. Then you start to fix through the bugs. And as you fix the bugs, have clear.
Michael Georgiou (14:00.332)
Thank
Gal Vered (14:18.574)
feedback, immediate feedback, because the immediacy is so important in software engineering, immediate feedback of what you broke as you were fixing it and then you can iterate. So that's how putting AI aside, that's how I did it in the past with engineering teams. And I think that with AI it even becomes more relevant. Because again, you can just write code faster. So I 100 % agree, we see it a lot, a lot of customers come to us and they're like, let's just...
make it a bit more sane. Let's have a test entry that at least tells us nobody's gonna call us in 2 a.m. and scream that the app is in progress. And once we establish that, we go into the more, the deeper level of like, okay, how do we make our engineering team super efficient? How do we allow you to ship code in hours instead of days or instead of weeks? So yeah, it really depends on when the customer or where the development team is in their journey to quality.
Eric Lawrence (15:14.319)
makes sense. Yeah, I am interested to know like, as far as a difference, is there a different approach that you see between when somebody comes to you with an engineering team and something that's been coded out versus somebody who might come to you that has a product that's been completely vibe coded?
Gal Vered (15:33.023)
It's a great question. think first of all, our bread and butter, let's say it like this, our customers with real product that is shipped to customers and more, not larger engineering teams. We have customers with five engineers, 10 engineers, but it's usually a team that comes to us and they want to solve quality on a systematic level. I think with vibe coding, there's a...
Michael Georgiou (15:58.731)
Hmm.
Gal Vered (16:03.126)
And by the way, I see the light in my room just there. I'll turn it. You know what? Let me turn it on and then respond to you. I don't know if you can see, but it drives me crazy. Give me a second.
Michael Georgiou (16:09.151)
Yeah, that's fine. Yeah.
Eric Lawrence (16:09.551)
Yeah, that's It gives you kind of an angelic look.
Michael Georgiou (16:12.725)
Yeah, no worries. Yeah, that's true.
Gal Vered (16:15.65)
Yeah.
Michael Georgiou (16:21.343)
Yeah, we can have Rahul edit this. It's fine. No worries.
Gal Vered (16:25.696)
Okay, so do you want to ask me this question again? Do you want to just cut it in a way I know you probably can?
Eric Lawrence (16:28.761)
Yeah, I can ask it again. As far as when somebody comes to you and they bring a product to you, what we see is it could either be custom built. Maybe it's an application or a piece of software that's been custom coded by engineers versus there are lot of people out there now that are using tools to vibe code to have AI completely write their products, how you handle both of those approaches.
if there's any differences that you see and how you go about testing those.
Gal Vered (16:59.628)
Yeah, that's a great question. think first of all, typically our customers, I wouldn't say only on the enterprise level, we actually see a lot of demand for quality across. have engineering teams of five and 10 people. And we also work with Fortune 500 companies that, you know, we basically provide quality to the entire organization. And it does usually start when you have a few engineers on the team. And with that being said, I think we...
vibe coded applications, it's usually very high. The goal is to get very high signal to noise ratio. So it's not about covering thousands and tens of thousands of different scenarios. Cause usually those apps are simpler and they don't have, you know, hundreds of millions of dollars of revenue every year running on them. So it's more, how do I make a change and just know things working so I can move a bit faster and I don't need to, you know.
worry and do this manual testing every time I make a small change. With larger accounts, or even when you go to five, 10 developers, there's a lot of surface to test. If you think about the Fortune 500 company that is in financial services and we do all of their UI and API testing, that's a huge surface. And then the activity there is mainly about connecting the different systems together. Also, you don't want to test
click on a button, button works. Click on another button, the button works. One of our premises at Checksum is that our tests are mimicking real usage. So we need to connect across all of those systems to make sure that there's a big unit, everything works and not, you know, some one endpoint works, but then in reality, the bugs are mainly in the integrations and the complexity of production, not in the one unit analysis.
Eric Lawrence (18:51.703)
That's interesting. Yeah, no, that's that's helpful to kind of like break down. I think what's interesting is a lot of this can go over the heads of
people that aren't very technical. So one of the things that we definitely wanted to discuss with you that gal is when you sit down with business owners and you just kind of discuss the impact that having these bugs over time and the compound effect that it makes can happen. what, what do you tell these people? What, what sort of impact can these things have on a business over time?
Gal Vered (19:29.676)
That's a really good question. And I think it changes. This is the place where it changes the most every month. So it starts with, because of the hype on Twitter and on LinkedIn, which I think a lot of it, it's not fake, but a lot of it is on experimental projects, not real enterprise. Everyone is feeling behind. Everyone is reading how
Michael Georgiou (19:43.785)
Yeah.
Gal Vered (19:58.058)
Anthropic rebuilt the C compiler, I think it was in like one day and they're like, but I, this feature takes me this very simple feature takes me two weeks and Tropic just built.
Michael Georgiou (20:08.011)
You're so right. just wanted to say I cut a thousand percent agree with you. had to say it because I see this stuff on LinkedIn everywhere. Anyways, go ahead. I'm just like, and I'm not technical like you, but I'm like, what? And there's no way. It's almost too good to be true. Some of the stories that are shown and they give this narrative that it's like, it's just this huge enterprise, amazing piece of software could just be built in a month. it's like, it's not, maybe one day, I'm sure it will. AI is going.
Eric Lawrence (20:10.24)
Yeah
Michael Georgiou (20:36.072)
rapid, right? But I always believe there's always going to be a human component that's attached to the greatness of a product. But anyways, go ahead. I interrupted.
Gal Vered (20:45.612)
No, I 100 % agree. I think everyone is feeling behind. I think the hype on Twitter is real in a sense that those AI models are really amazing at what they can do. But also, it's shipping real enterprise software still takes time, still takes time building, thinking through this.
And, know, again, the best example is Anthropic has like 100 software engineering positions open, not 100 positions filled, 100 positions open right now. So, you know, you still need engineers and there's still a lot of humans in the process, but everyone's feeling behind and there's two, like everyone's feeling behind because they're not feeling like they move fast enough. And at the same time, they use AI agents and they see the errors it produce, even if it's great again, and we use AI.
Michael Georgiou (21:16.492)
Crazy, yeah.
Gal Vered (21:37.878)
All the time, I don't write code anymore by hand. Everything I tell cloud code to write. Like I'm very hands-on with writing code, but the code is still generated with the eye. So there's a huge impact. Don't get me wrong, but at the same time they use it and they see the errors they provide and they see how much guidance they need to provide. So there's a lot of like fear about, we're not moving fast enough, but on the other end, I'm worried that our engineers will just use those tools without any control. Cause I see that it produces.
20, 30, 50 % slope code, however you wanna call it, but it produces errors. So I think CTOs, VPRs, engineering leaders come to check some with those two things in mind. It's like, how do we move faster? How do we, but how do I also make sure that my engineers aren't becoming, aren't pushing code that maybe kind of works today, but in a week or in a month, in one year, we're gonna pay for it because it's gonna be spaghetti code that's unmaintainable. And so that's kind of like the concerns they have.
And then in terms of impact, we think about it in two ways. One is kind of just make it very easy to say yes is the cost savings. So we have one big company, had 40 QA automation engineers before they hired Chexam. Now all of the QA automation engineers were converted into software engineers. So they have zero QA automation engineers. They all do software engineering. Chexam is responsible for quality. So that makes it very easy to say yes because
Eric Lawrence (23:01.413)
Hey.
Michael Georgiou (23:04.041)
That's awesome.
Gal Vered (23:04.931)
because of the cost savings. But the benefit is not the cost savings, it's just the ability to ship much faster and delight your customers.
Michael Georgiou (23:14.217)
And it probably, you know, also I can imagine that, you know, with just the improvements on the entire, just QA testing side, right? That your product delivers to the customer. It has, I think, an important mental component, right? Where it'll release a lot of just overwhelm and anxiety and stress that we all feel, right? When we're trying to build something and...
you know, and, cause we, we care and we want it to be successful. We have all these expectations, right. but at the same time, you know, I think it can really, just make a positive impact on, know, on the end users, on your customers, but also on their customers, whoever they're, you know, whatever they're trying to build, whatever, know, whoever their customers are basically is that it can just provide a lot of top value, mentally, you know,
to the engineering teams, to the founders, to the customers, internally, externally. I think there's a very huge component there. It's just the mental side of it.
Gal Vered (24:23.916)
I agree, I can give you an example of, so we on boarded a customer, I think it was two weeks ago, but I might get the time wrong. And like after 24 hours of using Checksum, they got back to us and say, this platform is amazing, it allows us to ship faster, we really love it. I got on a call with them, because it's such a strong feedback in short time and also proactive feedback is always something I want to understand, what was the magic moment, right? How do we...
encapsulated into a product and making it even more obvious. And I asked them like, how many bugs did we found for you? I assumed like we probably found like a few major bugs and they're like, no, you didn't find any bugs. So I'm like, so what's the, where's the reaction coming from? Like, well, we just created one massive PR and I can see all of the tests that Jackson generated and I can see they all pass. So I just feel like much more confident to.
Michael Georgiou (25:02.954)
if
Gal Vered (25:18.424)
push it and a PR that I would probably let it sit for like a few hours, maybe like a few days, review it. Checksum just allowed me to just merge it. And I think that's something we, today we're not surprised from it, but that's something that, you know, generally surprised us in the beginning where we thought our value is finding bugs, but our value is in finding bugs, but also confidence. Like knowing that there is no bugs in the PR, especially in the AI agent world is as important.
is finding the bugs that are there because again it allows you to move forward.
Eric Lawrence (25:49.852)
Yeah, I can see the value 100%. I we know from experience that usually it's the last 10 % of development that can feel like 90 % of the work because there's just so much that
Michael Georgiou (25:52.671)
Yeah. Yeah.
Gal Vered (26:00.076)
Exactly.
Eric Lawrence (26:03.249)
needs to get polished, that needs to be tested, that needs to just make sure at the end of the day, does everybody have confidence to push the update and to get things live?
Gal Vered (26:14.818)
Yeah, I agree. And that's the difference by the way, Ben, between a testing and reviewing, which is interesting because PR review tools are again, super helpful. We use them internally as well, but PR review tools, they catch bugs and you fix them. But if they don't catch bugs, it's still a weak signal that there are no bugs.
Michael Georgiou (26:32.255)
And who is it, who are you talking about? Who was it?
Gal Vered (26:35.81)
I think there are a lot of PR review testing tools out there, if an AI reviews your PR, or you can usually just ask Claude and Kerser to review the code, I think it gives you lot of confidence if a bug was found to fix it. If there is no bugs, I don't think it gives you confidence that there are no bugs, if you get what I'm saying, because it's more about finding a few bugs that exist less about confidence, and what we see is again, in order to really move fast, you need both.
Michael Georgiou (26:38.22)
PR review, okay, okay, yep, got it.
Michael Georgiou (26:45.803)
Mm.
Michael Georgiou (26:54.858)
Yes.
Gal Vered (27:03.65)
find the bugs, but also known that when there are no bugs, you can trust it and deploy.
Michael Georgiou (27:09.545)
Yeah, no, that's cool. Yeah, no, that's great. Yeah, I definitely think it's a conversation we can talk with my partners and definitely try to test out, check some AI as well. I think it can provide a lot of value to a software agency like us as well. Did you have something you wanted to say,
Eric Lawrence (27:34.306)
Yeah, I only had really one other question gal for you about
Just the process of getting these launched in specifically around your time as a CTO. I'm interested not about like the numbers of being able to launch something and make sure that the bugs are down and that the QA process goes smoothly. I'm wondering what does release anxiety do to an engineering team culturally? Because I'm sure you've seen that firsthand.
Gal Vered (28:07.126)
Yeah. So are you asking from the perspective of AI or just in general?
Eric Lawrence (28:11.855)
just in general from your experience.
Gal Vered (28:14.69)
Yeah. So I've been in organizations again, that you're worried about clicking the publish button. I've been in the, in the previous company that I was the city of when I joined. They used, know, the typical saying is never the flow and fighting, right? Because you don't want to have an outage on the weekend for them. It was the opposite. It was always when we, when I joined, it was very clear that there will be bugs and issues. And again, that was like the first priority. So.
they actually deployed on Friday, because they knew there are gonna be bugs and they knew on weekend, like the usage is low. And it was also a startup. So they plan Friday, you have a few customers that without your bugs, you have two days to use your software a bit, and then you can fix the bugs when customers get to the office on Monday, the software will already be fixed. So it was very, very hard. again, testing was, there was a...
A lot of changes we made in order to solve it, both in the code architecture, et cetera. And I think testing was a big part of it.
Michael Georgiou (29:20.638)
No, it's cool. know, and just like with speaking a little bit just with AI again, you know, it's just because there's so many things changing, Gals, you know, with not just obviously, you know, Checksum has incorporates a lot of AI. I mean, I don't know the ins and outs of the platform, but obviously, you know, you guys have AI within, you know, powerful AI within your platform. And I'm assuming it's from what you said before, it is agentic.
Is that right? Is that correct? Yeah. So, you know, do you think it would be different if tools like yours did not exist, right? Where if you have really, really good engineers, QA engineers and actual developers who are implementing the code and fixing things, and then they're catching issues and then they're fixing it. Do you think, you know, have you seen like
Gal Vered (29:50.318)
Yes.
Michael Georgiou (30:19.188)
a massive, massive shift in just terms of the kind of the overall, obviously speed is a big component, like you mentioned, but even just the quality of it, is that the whole purpose of it? Is the quality of Checksum and even some other tools, it's just that powerful that it catches things, it reports it back to the engineers, they know exactly what it's reported, it kind of gives an audit. Is that correct? Is that kind of how?
Gal Vered (30:46.23)
Yeah, it's correct. And I think, I don't know if you, I'd love to get your opinion, but with us, feel like when you use AI agents to write the code, you don't have the same level of control of the details. something about writing is thinking even beyond code, right? When you write something, you think through it, you think through all of the different angles. And once AI writes the code, like you give the high level, high level objective and the
and what you're trying to achieve and maybe even some architecture. But once AI writes the code, you lose the level of thinking in the tiny details in the edge cases and you kind of can't for the AI agent to do it. That's why even if you review the code very diligently, you're never gonna reach the same level of quality you had had you written data thing yourself. On top of it, think people attention span is getting really short in general these days.
Michael Georgiou (31:42.763)
Very true. Yep.
Gal Vered (31:43.534)
And with AI, it's probably getting even worse. So I even worse. So, I imagine most of engineers, when they get AI code, you scheme it, you read it, but you don't read every line of code, think through this one different language. And so, yeah, I think it's, again, we want to enable people to ship with AI. We want to enable people to scheme AI code and trust it. So that's, I'm not against this trend. I think this is the trend we need to go through in order to realize AI potential. We just think.
There's a missing part of this equation that checks and provides.
Michael Georgiou (32:14.869)
Yeah, no, I think that's a great answer. That was kind of where I was getting at is that, you know, there's just a, think, at least from, not just, how do I say this? Not just what we're seeing with needs from customers that we work with or prospects or what I'm seeing trending through, not just social, but just in the world in general, in tech and business, is that it's like,
There's this like high reliance on AI and I'm guilty of it too with a lot of things. It's hard, right? Cause sometimes you feel stupid even using it so much and you're like, you can't, you don't, you want to think about things that your brain, cause you're, we're smart. The human brain is incredibly, I mean, yeah, it's a whole different discussion, but it's incredibly intelligent, the brain, right? Especially with practice and, and, in training and experience, and development obviously, but basically, you know,
I've noticed like there's a massive kind of reliance on AI. And I love what you said in terms of when people, developers, or even project managers, when they review a product, right? They think about edge cases and unique cases, right? That AI, at least right now, is not...
It's not gauging, it's not capturing, right? And I'm sure it'll get there. mean, God knows where AI is going to be in a couple of years or five or 10 years. But at least, you know, at the moment, I think there's going to be a lot of open doors and new opportunities for people, along with technology and the improvements of tech. And I think it's just going to, as long as we use it with the right intentions and we try to do things the right way to help people and to help our customers, to help our families and others in the world.
I think there can be so much value with kind of, you know, just the balance of using tech and using, you know, AI specifically, and then obviously, you know, allowing human beings to provide the value that they have where AI may not be able to fully capture and provide value to, if that kind of makes sense. It's like a balancing effect.
Gal Vered (34:31.446)
I agree. How do you think about development with AI on your side and the human in the loop? Are you encouraging people to use as much AI as possible? you trying to create some culture that still keeps the human in the loop and kind of making engineers accountable? How do you think about shipping with AI moving fast, but at the same time, do it in a way that delights your customers, not just...
So all of the AI problems at your customers.
Michael Georgiou (35:01.835)
Yeah, no, I mean, that's a great question. I see it kind of internally and externally, like internally, at least for our developers. mean, have over Eric, what, almost probably over 40 engineers now. And we use AI as well. And we use all the cursor and have Anthropic and I think even Copilot, I believe, writes code too. Yep.
Eric Lawrence (35:17.316)
Yep.
Michael Georgiou (35:31.463)
obviously open AI. so, you know, and we've integrated a lot of kind of agentic platforms within our processes and all that. And I can't speak for my business partners and that you are a lot more technical than I am, but I can tell you that it is embedded in the culture because if it, if it does save time, and it improves quality kind of like your product does exactly, then yeah, of course it makes sense to, onboard it and to
incorporate it within the entire kind of culture of the organization, not just with engineering and delivery and support and QA and even design, but it's also with marketing and sales. try to incorporate it, but at the same time, we know where it's at right now because we use it so much. we try to also make sure that we have the personal touch and we have the human element. That's very important.
So it's, like I said, it's kind of a balance, at least internally, you know, and it is improving the quality of our code. is doing things like that that are insanely valuable. Externally, I'll let Eric answer that kind of on the sales side, what he's seeing with AI. You know, I think it's moving also very quickly, just like with the demand of it.
Eric Lawrence (36:51.756)
Yeah, I would agree with that. I know it seems like for two thirds of every new project that we engage with, at least think of it from web applications to, mobile apps, to custom software. People want some sort of AI to be a part of it. And I think from the most basic sense, you can simply integrate with Claude or open AI or something there. But one of the things that we've been doing for a lot of different projects is
When you first launch an app, you want to have a model out there that runs with it. But if you have your own special amount of data that you have on your side, on the background, you have your own model that you're training and getting up to speed to the point where it's good enough that it can eventually be swapped out for whatever AI model you have in there so that it's going to be unique to your own product. That's at least what we're seeing on the product side.
Gal Vered (37:48.022)
I agree. And I think one of the challenges, especially in coding and I'm sure in other places is a short term versus long term. Like I can write a feature or let's say I use the AI to produce code to write a feature. And this feature works right now. Like it's fully working. I tested it, but the code is not architected in the best way. And then there's always the question of, I move fast? Do I ship it and probably pay later when.
my code is spaghetti and it's very hard to make changes. Do I take the time to write it well and again, prompt AI to improve it and then enjoy the benefits later. And the problem with AI is that it's compounding because if you have bad patterns in your code base, the new code that will be generated is very similar to the existing code because the agent reads the files and picks up on patterns.
Michael Georgiou (38:35.392)
Yeah.
Gal Vered (38:44.098)
So it's just compounding. It's not an easy problem to solve. And that's why we see a lot of Vibe-coded applications. They spin up an MVP very fast and then they hit a roadblock where you just can't introduce new features because nothing is working. Like the code is so messy that it's impossible to introduce a new feature, especially with AI. Maybe if you read the code and you think throughout to do it, you can maybe get past it. So.
Michael Georgiou (39:09.235)
It increases the technical debt even more than before, sometimes. Yeah.
Gal Vered (39:12.244)
Exactly. more than before, because it's compounds. Because again, if you have bad pattern in your repository, the AI is going to repeat it again and again and again. And if you have good patterns in your repository, then the code that the AI will produce will be better than it. It will still not be always perfect, but the patterns that we've incorporated there will be the correct pattern.
Michael Georgiou (39:21.855)
Hmm.
Eric Lawrence (39:36.628)
Great. Well, Gal, I only have one other question for you. I want to think future. In five years, what will feel embarrassing about how we handle bugs today?
Michael Georgiou (39:49.739)
You
Gal Vered (39:51.438)
That's a good question, I think.
Michael Georgiou (39:58.315)
.
Gal Vered (39:58.922)
In five years, I think what would be embarrassing or feel outdated is the fact that the bugs were shipped in production regularly and the fact that testing was just being done by testing, by manually opening a browser and clicking around. The thing we're building at Checksum and our vision is to basically build a simulation to how code works in production. So the idea of you needing to test your code.
manually or not even manually automated against scenarios, we want to eliminate it completely because we want to allow you to deploy your software into a digital clean of your production environment, see exactly all of the problems and fix them. Maybe the best analogy is like autonomous cars. When Google has a new model to release for their autonomous cars, they don't go out there, release it and see if the car, even like on a fake street and see if the car.
you know, hit fake pedestrians, right? Definitely not real pedestrians. They have millions of simulations that they first run their model through that simulates real behavior and they can see the model decision and they can measure everything. And then they can take it into, I'm assuming, a street in a very controlled environment. And then they deploy it out there for real cars in San Francisco to drive autonomously. I think today it doesn't exist in software. Everything is just tests that are imagined by people. And what we want to do is now let us
Michael Georgiou (40:59.626)
Yeah.
Eric Lawrence (41:15.716)
Brilliant.
Michael Georgiou (41:23.788)
Mm-hmm.
Gal Vered (41:26.83)
Create a simulation of your real production environment, run it through millions of scenarios, get the feedback within minutes, know exactly what's working and what's not working to a very high accuracy.
Michael Georgiou (41:37.791)
Yeah, you don't just want to launch a product to the public when it's clearly not ready, right? They're probably, especially big companies like Google, Apple, Microsoft that are testing these things to the moon, they are probably past the validation phase for a lot of their products. So once they develop the product and they're...
almost ready to launch and almost could mean like a year or two down the road, right? For them, it's like, oh, we're almost there. And it's like a year or two or three, at least from what I've read and what I understand, it's a long-term thing, right? Like you mentioned with OpenAI, they were developing it seven or eight or nine years, even before we even started hearing about it. And we had no freaking clue. No one knew what was actually in the works. But no, like I think what you said is very valuable, you You gotta make sure that...
The quality of the testing is very strong because you have to look at all the different edge cases and that's probably what a lot of these companies are doing. mean, we even do it, you guys do it, right? I check some internally, right? You want to make sure that before you launch a release or a new product, you want to make sure that it's heavily, heavily tested, right? With the user base or even a closed group before you just put it out there and then it could just fail. And once it gets a bad reputation, it's hard to come back from it.
Gal Vered (43:03.905)
I hug.
Michael Georgiou (43:04.809)
You know, but no, this has been great. Yeah, Gal, I think that's all we have for you today, man. Really appreciate you. Thank you for taking the time to just chat with us about this. you know, just so everyone knows, you know, Eric and I are not on Gal's level in regards to QA. This is why he's here. You know, we want someone like him here and people like him who know exactly what's going on in kind of the world of
Eric Lawrence (43:26.658)
Yeah.
Michael Georgiou (43:34.155)
QA and development and just technology and what's going on at the moment and where things are headed. And we just want to serve others and help people as much as possible in terms of just their learning and inspiration and what they can gain from this. So thank you very much, Gal. Really appreciate you. And can you share with everyone where everyone can find you, your website, social handles?
Gal Vered (43:58.742)
Yeah. First of all, thank you for having me. It was a great conversation. Check some you can find on checksum.ai. So very easy to remember. I'm mainly active on LinkedIn. You can just search Gal Vered. I sometimes post on Twitter, but in reality, not as much, but feel free to check me out there as well.
Michael Georgiou (44:19.884)
Awesome. Sounds good. Thank you so much, Gal. Thank you everyone for listening. And I'm your host, Michael Georgiou from Tales from the Pros, along with my cohost, Eric Lawrence, and our amazing guest, Gal Vered Thank you so much, guys. Really appreciate it. Thank you, Gal. All right, guys. All right. Thanks.
Gal Vered (44:34.434)
Thanks, bye.
