Full Transcript for Stop Using AI for Everything

WEBVTT 00:00:19.679 --> 00:00:22.079 Hello and welcome to Full Tech Ahead. 00:00:22.160 --> 00:00:29.839 I'm your host, Amanda Razzani, and with me today, I'm excited to have Federico Romalo, founder and CEO of Density Labs. 00:00:29.920 --> 00:00:31.280 How are you doing today? 00:00:32.159 --> 00:00:33.119 Hello, Amanda. 00:00:33.200 --> 00:00:34.880 It's a pleasure to be here today. 00:00:35.280 --> 00:00:36.880 Happy to have you on the show. 00:00:37.039 --> 00:00:42.399 Can you share a little bit about your background and what do you do over there at Density Labs? 00:00:43.359 --> 00:00:44.079 Yeah, sure. 00:00:44.240 --> 00:00:48.399 So at Density Labs, what we're doing is AI engineering. 00:00:48.880 --> 00:00:57.600 We partner with companies that are trying to figure out how to leverage AI within the organization. 00:00:57.920 --> 00:01:03.759 We take all the hype away and we focused on business impact outcome. 00:01:04.799 --> 00:01:08.799 We track the return of investment of those efforts. 00:01:08.959 --> 00:01:13.040 The other thing we we do is we track the total cost of ownership. 00:01:13.200 --> 00:01:23.599 We've seen a lot of projects where the token consumption goes to the roof and then the cost for running these projects goes super high. 00:01:23.680 --> 00:01:25.519 So we we we work on that. 00:01:25.920 --> 00:01:26.640 Okay, great. 00:01:26.719 --> 00:01:38.480 So we're gonna talk today about your experience in working with these companies and what are some of the challenges and the ROI for software development companies when they're utilizing AI tools. 00:01:38.640 --> 00:01:42.640 I know everybody's trying to harness AI in the best way possible. 00:01:42.879 --> 00:01:50.000 So let's start with the fact, though, that roughly 95% of enterprise AI pilots never reach production. 00:01:50.319 --> 00:01:51.840 Why is that, do you think? 00:01:52.159 --> 00:01:55.439 It's a very interesting situation that is happening with that. 00:01:55.680 --> 00:02:06.079 The main reason is that they test the demo with the best case scenario, and then they don't consider all the use cases. 00:02:06.239 --> 00:02:07.680 That's the first issue. 00:02:07.840 --> 00:02:16.159 The other issue that is happening is that even though AI is a technology tool, it is actually an organizational change. 00:02:16.400 --> 00:02:25.360 And uh a lot of companies don't see it that way, and that's the the uh underlying reason or the root cause of why uh this type of projects fail. 00:02:25.520 --> 00:02:40.240 And the the last issue that I've seen is that because you have so many different possibilities of outcomes or use cases, what we start doing is setting up a model that evaluates another model. 00:02:40.479 --> 00:02:49.280 Uh, that way we reduce those uh that that complexity towards expected outcomes, expected behaviors. 00:02:49.439 --> 00:02:52.479 And then we set guardrails to to avoid that, right? 00:02:52.719 --> 00:03:12.319 But basically that that that issue that that that is that is the the core issue is that it becomes unwindling, unwindly complex when you have so many use cases and you don't have a way to track that the outcome is what you actually expect. 00:03:13.039 --> 00:03:18.159 Can you share a little bit more about you said create a model to review the the model? 00:03:18.319 --> 00:03:19.919 Can you share more about that? 00:03:20.639 --> 00:03:21.599 Yes, absolutely. 00:03:21.840 --> 00:03:32.159 So the first thing we need to understand is that the when we have agents executing and doing actions for us, right? 00:03:32.479 --> 00:03:42.159 What we are actually doing is we're providing tools to a model, to a large language model that is making those decisions and then executing for us, right? 00:03:42.319 --> 00:03:47.919 The issue is that these are non-deterministic outcomes, is what we're going to get, right? 00:03:48.319 --> 00:03:53.360 So, how can we test that the outcome is what we expect? 00:03:53.520 --> 00:04:04.159 What we do is we build uh it's called Eval LLM, and basically uh what we have is another model, and by model what I mean is uh thinking it of as an engine, right? 00:04:04.319 --> 00:04:28.319 Uh, it's the core of every um AI automation tool that we use that can that we can have a conversation or we can it can read images, there are different models, but basically what we do is we set up one model to analyze and make a decision to execute, and then another model to evaluate what the first model is going to do, and then make the decision. 00:04:28.480 --> 00:04:31.439 Okay, does that make sense or it doesn't make sense, right? 00:04:31.600 --> 00:04:40.000 The reason, technical reason we need to use two models is because what we want is to have uh let's call it second opinions, right? 00:04:40.240 --> 00:04:50.399 If we if we have the same models evaluating themselves, then we have the higher probability of them to be to agree to a wrong decision. 00:04:51.600 --> 00:05:02.000 Okay, so when putting AI agents into an engineering org, what are some of the things that break or some of the challenges that you see? 00:05:03.680 --> 00:05:14.879 Even though I am advocating for uh the use of AI, I believe that using at l as little AI as possible is the best approach. 00:05:15.120 --> 00:05:20.959 So when uh what are the best scenarios to leverage AI within an organization? 00:05:21.199 --> 00:05:34.480 So when we have uh business processes, right, um, and we want to automate them, we can use uh an AI agent to just automate the whole workflow, right? 00:05:35.439 --> 00:05:41.519 Or we can build software, we can build code that is actually going to automate that workflow, right? 00:05:41.759 --> 00:05:52.959 So what I am advocating is to build code to solve the automation uh for all the deterministic workflows. 00:05:53.199 --> 00:05:56.639 By deterministic, I mean uh think of math. 00:05:56.800 --> 00:05:59.680 Math is deterministic, one plus one equals two, right?