Is your AI hallucinating? From RAG to vibe coding, and everything in between の全文書き起こし

time in tech, but agentic AI is a huge buzzword right now. 00:02:36.159 --> 00:02:44.400 And like Russell said, it's just becoming this very uh processing heavy thing that people want to use agentic AI. 00:02:44.479 --> 00:02:46.479 Even RAG, sometimes it's unnecessary. 00:02:46.639 --> 00:02:48.159 You can go for the lower level. 00:02:48.560 --> 00:02:49.120 Just a prompt. 00:02:49.360 --> 00:02:50.159 Yeah, just just a prompt. 00:02:50.319 --> 00:02:51.039 Prompt engineering. 00:02:52.479 --> 00:02:55.759 Yeah, you don't always need to be using an LLM even. 00:02:55.919 --> 00:02:57.599 Uh, but it's fine. 00:02:57.680 --> 00:02:59.759 It's the the hype will pass. 00:03:00.240 --> 00:03:01.199 Will calm down, maybe. 00:03:02.800 --> 00:03:13.039 For yourselves as software engineers, um, how is the role of software engineering changing as GNAI and smarter IDEs become more commonplace in sort of the everyday toolbox? 00:03:13.599 --> 00:03:15.919 You're finding a lot of vibe coding. 00:03:16.080 --> 00:03:25.439 I know we talk about vibe coding as all a lot, but you can see that people are struggling to understand their own code anymore because it's just being generated for them. 00:03:25.840 --> 00:03:34.639 With IDEs, sometimes through uh LLMs like Gemini or Chat GPT, people will feed it a problem and have it solve for that problem. 00:03:34.800 --> 00:03:48.400 And then the code that is generated is so specific to how you describe that problem that once you need to tweak it or make it more general, firstly the person who needs to tweak it doesn't know what's happening with the code, so they might mess it up there. 00:03:48.479 --> 00:03:56.639 But otherwise, it might immediately fall over as soon as one of the parameters you entered shifts slightly due to business requirements. 00:03:56.960 --> 00:04:02.960 How does that impact continuity of systems going forward, especially for like enterprise organizations? 00:04:03.759 --> 00:04:15.919 Well, what it's going to mean, and I think we all agree, is that uh the developers that are supposed to be maintaining systems and stuff, they're going to really, really struggle to it's going to become slower, right? 00:04:16.079 --> 00:04:22.480 Right now it's the fastest paradigm to shove some prompts into an LLM and then hope for a good answer back. 00:04:22.639 --> 00:04:31.040 And then you can apply that code, but eventually it's going to become slower because now you have to understand code that you didn't understand to begin with, and then you're going to have to add more code to it. 00:04:31.199 --> 00:04:36.240 And probably if you vibe coded the first solution, you're going to vibe code on and on and on. 00:04:36.319 --> 00:04:42.000 And it's going to become this mess of uh junk, basically. 00:04:42.319 --> 00:04:44.399 I think I would actually counter you on that. 00:04:44.480 --> 00:04:46.399 So I think I like that. 00:04:46.560 --> 00:04:48.639 You know, I love to counter you a lot. 00:04:49.279

--> 00:04:55.680 Um although vibe coding can be quite dangerous, it's also a tool for software developers. 00:04:55.759 --> 00:04:57.759 It's allowed us to learn a lot faster. 00:04:57.839 --> 00:05:01.920 It's allowed us to push into different areas of tech that we haven't used before. 00:05:02.079 --> 00:05:11.519 So if done correctly, it will actually speed up your development and allow you to be a lot more autonomous, which is where I think the software industry is kind of going now. 00:05:11.680 --> 00:05:19.040 We're moving away from very siloed roles where people are either a software developer or a back-end developer. 00:05:19.120 --> 00:05:21.040 And now we're having to become a lot more autonomous. 00:05:21.199 --> 00:05:26.879 You have to maintain the entire system, you have to build the entire system, you have to kind of handle business requirements. 00:05:27.040 --> 00:05:32.800 You're expected to be more of an engineer or an architect rather than just one role, which is which is exciting. 00:05:33.040 --> 00:05:34.639 No, I 100% agree. 00:05:34.800 --> 00:05:41.199 Um, I think it's always easy to slip into the dangers of AI and how they're they're going to ruin your life. 00:05:41.279 --> 00:05:46.240 But even I, it's it's interesting we're having this conversation now because we've had this conversation before. 00:05:46.639 --> 00:05:53.839 And I'd say two months ago, I was asked at some point if I use uh generative AI when I code. 00:05:54.079 --> 00:05:57.600 And I at that time I didn't use generative AI. 00:05:57.839 --> 00:06:03.040 But I found that I needed to uh join a new tech stack, basically. 00:06:03.120 --> 00:06:07.120 I've need to needed to learn new languages and a new framework. 00:06:07.360 --> 00:06:16.800 And I did find that I went and I asked, I explained the problem, and I was like, what type of solution does this framework lend itself to with this problem? 00:06:17.040 --> 00:06:24.000 And I will say I've like, it's much faster than when I used to have to go watch 10,000 tutorials, stack overflow questions. 00:06:24.240 --> 00:06:34.319 And I feel like I understand it much better because there's someone there explaining my explaining a solution to a problem as it's being generated, if that makes sense. 00:06:34.639 --> 00:06:41.199 You're also able to ask the question immediately and that removes the fear of going to your techie or going and searching. 00:06:41.360 --> 00:06:45.360 You just ask a generative AI agent, like this is this is my issue, how do you solve it? 00:06:45.600 --> 00:06:53.040 Yeah, and it and we did mention as well at the time that we would have to, you would have to be more adaptable, basically. 00:06:53.120 --> 00:07:02.319 You need to be more focused on how to solution problems, the most efficient way to solve a problem, not what language you're going to be solving in it, what framework you're going to be solving. 00:07:02.639 --> 00:07:12.639 But I suppose the most efficient, but also

making sure that it is supportable and reliable and will stand, you withstand whatever pressures that system's going to be putting on it. 00:07:12.879 --> 00:07:23.199 And just jumping back to the to the risks that you mentioned earlier, do you think that there are different risks at for you as an engineer at different stages of your career when using these tools? 00:07:23.360 --> 00:07:24.639 Rosal, do you want to take this one? 00:07:24.959 --> 00:07:25.519 Definitely. 00:07:25.680 --> 00:07:29.600 So I think, like we said, it's a tool, you need to know how to use it well. 00:07:29.759 --> 00:07:44.560 If you're not someone who's able to realize what your role is in generative AI and what the role of generative AI is, it will become a problem because then you are just going to rely on the code being generated without understanding it. 00:07:44.720 --> 00:07:47.680 And I see that a lot with graduates now. 00:07:47.839 --> 00:07:50.160 I'd ask them a question, I'd be like, why did you do this? 00:07:50.319 --> 00:07:51.600 I know it's completely wrong. 00:07:51.759 --> 00:07:54.639 They're like, I don't know, I used AI to help me. 00:07:54.800 --> 00:07:58.399 That is not how we expect people to use AI. 00:07:58.560 --> 00:08:03.120 You need to use it, but you need to help kind of understand where and why you're doing it. 00:08:03.199 --> 00:08:05.360 And if you don't, use it to understand. 00:08:05.519 --> 00:08:07.759 Ask why you're doing this, why you're not doing this. 00:08:07.920 --> 00:08:11.680 You on your own, do your own research and decide what is the best approach. 00:08:12.000 --> 00:08:17.040 Um so you kind of need to know enough to vet it, or if not, make sure that you're asking the right questions. 00:08:17.600 --> 00:08:22.319 I don't think you should be using it just to do all your research, your designing, your implementation. 00:08:22.480 --> 00:08:24.879 Do your research on your own and use this as a check. 00:08:25.040 --> 00:08:26.319 Or as like a buddy. 00:08:26.560 --> 00:08:27.439 It should be your buddy. 00:08:28.079 --> 00:08:29.360 Rubber duck level two. 00:08:29.920 --> 00:08:31.519 You know, I I 100% agree. 00:08:31.680 --> 00:08:32.320 No, yeah. 00:08:32.639 --> 00:08:37.519 Hannah, jumping onto you mentioned data once or twice already. 00:08:38.080 --> 00:08:42.480 Why do so many organizations think more data is better? 00:08:42.639 --> 00:08:47.600 And what's a better approach that they should be taking or could be taking? 00:08:47.919 --> 00:08:58.159 You know what, there is kind of this narrative, and this is from the conferences that we've attended as well, that if you have data sitting somewhere, why is it sitting there and how can you be using it? 00:08:58.320 --> 00:09:00.960 And I think that's a good thought process to have. 00:09:01.279 --> 00:09:08.320 But what you also need to be thinking about is should we be using our data to train a model? 00:09:08.559 --> 00:09:11.279 Is that really what we need to

be using our data for? 00:09:11.519 --> 00:09:14.000 And how do we need to structure our data? 00:09:14.080 --> 00:09:17.120 Because that's becoming so important in this space. 00:09:17.360 --> 00:09:29.840 One of the biggest things when when we develop a POC or when we develop a fully fledged chatbot is looking at your data first and seeing how it needs to be structured for your specific model to understand it. 00:09:30.000 --> 00:09:32.240 Do you need to retrain your model on your data? 00:09:32.480 --> 00:09:35.360 Or does your date the structure of your data need to change? 00:09:35.600 --> 00:09:49.519 And I think there's this big misconception that any type of data can be like whoop and then thrown into an LLM, and then that LLM will be able to reason and understand that data, but the it's it that's just not how it works. 00:09:50.159 --> 00:09:55.679 And I think the big message that I just want to say here is data analysts are important. 00:09:55.840 --> 00:10:05.919 You need people to look at your data and understand what's happening there before you decide that you can feed it into a bot, before you can decide the application for the data. 00:10:06.080 --> 00:10:11.679 It's not really good enough to know what data you're collecting. 00:10:11.759 --> 00:10:16.559 Because a lot of the time it's unstructured data, a lot of the time it's bad data. 00:10:16.720 --> 00:10:22.480 It's not correct, it's nonsense, gobble gook, especially in the case of like audio transcripts or anything like that. 00:10:22.720 --> 00:10:24.480 How good is your transcription software? 00:10:24.559 --> 00:10:26.559 Do you need especially in like the African context? 00:10:26.879 --> 00:10:27.120 Yes. 00:10:28.080 --> 00:10:31.440 How much processing are you going to have to do on that data beforehand? 00:10:31.679 --> 00:10:35.840 And if you're going to ask me next, should you be processing your data? 00:10:36.000 --> 00:10:38.399 Almost 100% yes, depending on the data. 00:10:38.559 --> 00:10:39.519 Sometimes it's really nice. 00:10:39.600 --> 00:10:43.759 It's like a SQL table where you're just collecting names and fields, which is lovely. 00:10:44.159 --> 00:10:44.879 But almost never. 00:10:45.120 --> 00:10:49.200 I've never had a never had someone come to the come to me with that problem. 00:10:49.519 --> 00:10:54.480 So the the if the question is, is garbage in, garbage out, it's that's still true, right? 00:10:54.799 --> 00:10:59.120 LLMs haven't solved the problem of uh bad data. 00:11:00.159 --> 00:11:03.600 How does RAG change this conversation around data? 00:11:03.840 --> 00:11:06.159 Or does it change this conversation around data? 00:11:06.399 --> 00:11:07.679 Definitely not, not at all. 00:11:07.840 --> 00:11:10.720 You still need really good data for RAG. 00:11:11.039 --> 00:11:12.720 Garbage in, garbage out. 00:11:12.879 --> 00:11:16.480 Um, I think with RAG, people, it is very useful. 00:11:16.559 --> 00:11:17.519 It's a great context. 00:11:17.600

--> 00:11:23.360 So RAG is where we yeah, maybe we should give them some introduction. 00:11:23.600 --> 00:11:35.440 So RAG is retrieval augmented generation, and it's a very popular business use case that people are now using where you augment LLMs with extra documentation that you now feed it. 00:11:35.679 --> 00:11:46.480 So the perks of it is that your LLM is now grounded, you're not going to have hallucinations as much, and you're able to personalize the answers that you're getting from your LLMs with very specific data. 00:11:47.039 --> 00:11:56.960 Um, regarding the data of that, if you the process in which you do retrieval augmented generation really requires the data to be good. 00:11:57.039 --> 00:12:12.080 Because if it is not good and it can't find the relevant answer from your data, if you have not prompted it correctly and put the right steps in place, it will either hallucinate with no source or find a source but still hallucinate, or say I don't know. 00:12:12.159 --> 00:12:14.480 So it's still gonna be an irrelevant answer. 00:12:14.720 --> 00:12:23.039 Um, yeah, the ways in which you have to tune your data and structure it is very dependent on the use case, but still very much an important rule. 00:12:23.360 --> 00:12:24.399 Interesting, that's really cool. 00:12:24.960 --> 00:12:31.840 I feel like, sorry, just to chip in there, I feel like what's happened for me is RAG has almost highlighted this garbage in garbage in. 00:12:32.240 --> 00:12:32.720 100%. 00:12:33.200 --> 00:12:44.879 Yeah, because it becomes ever so much more frustrating when a user comes to you after you've made this POC with RAG and they say, this answer is wrong, like your model is wrong, right? 00:12:45.279 --> 00:12:56.639 And then you look at the answer, you look at what it used to answer the question, and it's it's done everything correctly, but you can see that the data that was provided had an inaccurate answer or an out-of-date answer. 00:12:56.799 --> 00:13:09.360 And that's a big issue, especially out of keeping RAG up to date has been a very interesting challenge in terms of the methods that you use to describe this data to your model, right? 00:13:09.840 --> 00:13:13.360 You have to re-describe it every single time it gets updated. 00:13:13.600 --> 00:13:15.279 So that's that's very interesting. 00:13:15.600 --> 00:13:16.399 Very cool. 00:13:16.639 --> 00:13:22.399 Um, switching to maybe I suppose RAG isn't so unknown anymore, but agentic AI. 00:13:22.480 --> 00:13:27.360 I think for a large part of the population, agentic AI is is this unknown concept. 00:13:27.519 --> 00:13:28.559 What is it? 00:13:29.120 --> 00:13:30.639 I think Rosal should take away. 00:13:30.720 --> 00:13:32.320 Oh, I actually just did something cool. 00:13:32.559 --> 00:13:36.399 Um, so agentic AI, how it differs from generative AI. 00:13:36.559 --> 00:13:39.840 Generative AI is using these large language models to generate content. 00:13:40.080 --> 00:13:48.240 Agentic AI

is making use of well-crafted or designed models that have better reasoning, better classification. 00:13:48.320 --> 00:13:54.240 So they're just like a little bit smarter to solve very specific problems and specific tasks. 00:13:54.399 --> 00:14:01.840 So the key difference here is generative AI will generate in a very creative way, whereas agentic AI is meant to be more autonomous at solving problems. 00:14:01.919 --> 00:14:04.000 So we're kind of giving it a little bit more intelligence. 00:14:04.240 --> 00:14:05.279 So it's thinking for itself. 00:14:05.519 --> 00:14:08.799 Yes, the idea is that it would try to think for itself. 00:14:08.960 --> 00:14:20.000 Obviously, it does not fully think for itself, and a lot of the time the agentic systems that you come up with are not very smart, but that's the the main difference. 00:14:20.320 --> 00:14:23.279 So basically, it's step one into AI's world domination. 00:14:23.600 --> 00:14:25.279 It's step one into AGI. 00:14:25.840 --> 00:14:27.120 Yeah, yeah, that's very true. 00:14:27.200 --> 00:14:27.679 That's very true. 00:14:28.000 --> 00:14:31.200 I will say it's almost like what you're doing right now. 00:14:31.279 --> 00:14:38.080 You know, people, someone told you that you need to do this, you know that you needed to ask Russell about agentic AI, right? 00:14:38.159 --> 00:14:50.320 So you've got one main per main interface, and then this interface knows that I need to talk to Russell about agentic AI, and to Hala about data, and I need to talk to Matthew about sound quality, you know. 00:14:50.799 --> 00:15:00.159 And that's I feel like a good high-level Yeah, you're given the high-level instructions and the way in which the agent will take the path of doing it is up to the agent. 00:15:00.399 --> 00:15:03.279 So the agents are very good at one specific thing. 00:15:03.440 --> 00:15:10.080 Um, but if you asked uh Hala to go do gymnastics for you, that would not go super well. 00:15:10.480 --> 00:15:14.879 So maybe this main agent would be pretty good at doing gymnastics. 00:15:14.960 --> 00:15:17.679 Like, all right, but like mess up every now and then. 00:15:17.759 --> 00:15:21.120 But then gymnastics agent over here is just the best. 00:15:21.200 --> 00:15:24.000 It's the Simone Biles of gymnastics, you know. 00:15:24.240 --> 00:15:34.320 So basically, for for the non-techies watching, it's creating it's it's almost like you're building subject matter experts who can think autonomously. 00:15:34.559 --> 00:15:34.879 Exactly. 00:15:34.960 --> 00:15:35.840 Yeah, that is 100%. 00:15:36.559 --> 00:15:37.440 Cool, and thank you. 00:15:37.679 --> 00:15:42.559 So, what excites you both uh about agentic AI, if anything. 00:15:43.360 --> 00:15:51.840 I'm excited, and I think I said this last time as well, at the prospect of not using AI for stuff, and hear me out. 00:15:52.000 --> 00:15:59.759 I want the agents, I want the agent, the main agent as we discussed, to know to maybe not consult another

agent, right? 00:15:59.840 --> 00:16:07.440 It could be something that translates into math, for example, because LLMs are notoriously bad at math. 00:16:08.720 --> 00:16:10.159 It's just because it's don't worry about that. 00:16:10.240 --> 00:16:10.960 We'll talk about that later. 00:16:13.200 --> 00:16:13.440 Right? 00:16:13.600 --> 00:16:16.320 But you don't need a model to solve math problems, right? 00:16:16.399 --> 00:16:21.440 You just need an agent that's capable of doing these math problems, like tough integration problems. 00:16:21.600 --> 00:16:27.120 But as we all know, it's really awful to type a math problem into a machine. 00:16:27.360 --> 00:16:39.679 So if you can tell an LLM, look, I need the integral from here to here of this very complex function, and that can translate it into math language, and then your math agent can take care of that for you. 00:16:39.840 --> 00:16:46.879 That's a very exciting part of it because it's cutting out a lot of processing, unnecessary processing that's happening. 00:16:47.039 --> 00:16:48.960 And that's what I'm a little bit excited for. 00:16:49.039 --> 00:16:52.399 And I mean, obviously, just the general benefits of agentic AI. 00:16:52.639 --> 00:16:52.960 Yeah. 00:16:53.200 --> 00:17:16.720 I think, yeah, I think agentic AI will definitely start to shift the way industries, not even just software industries, are kind of working where the routine tasks will now be taken over by these agents, and that gives humans a lot more time and flexibility to work on things that are a lot more high-level and autonomy in in the software industry as well, which is quite exciting. 00:17:16.960 --> 00:17:18.319 It's very cool. 00:17:18.720 --> 00:17:24.880 Earlier you mentioned, just on this point, earlier you mentioned that AI isn't always the right tool for the job. 00:17:25.039 --> 00:17:33.440 But do you think that agentic AI might be moving towards AI being, well, the specific agentic AI agent being the right tool for the job? 00:17:33.599 --> 00:17:42.000 So, in a way, we're kind of moving towards an era where there will be more AI for these problems that right now AI isn't the answer for. 00:17:42.319 --> 00:17:44.480 I think that was a very complicated question. 00:17:44.559 --> 00:17:46.640 I'm sorry, go to any AI. 00:17:47.599 --> 00:18:01.359 I think agentic AI, generative AI, machine learning, all deep learning, all of these areas within the AI realm is all very important, and every single one has a specific use case. 00:18:01.599 --> 00:18:05.279 There's lots of cases where you do not need agentic AI or generative AI. 00:18:05.359 --> 00:18:09.359 It is a lot more expensive, it's a lot more environmentally unfriendly. 00:18:09.440 --> 00:18:14.400 So there's lots of cases where machine learning and deep learning are perfectly good at solving these problems. 00:18:14.559 --> 00:18:16.079 That's not going to change. 00:18:16.319 --> 00:18:20.559 In time, there will definitely be a lot more cases

where agentic AI is useful. 00:18:20.799 --> 00:18:28.400 But once again, you need to be very critical at noticing where it is needed because agentic AI is a lot more expensive than generative AI. 00:18:28.720 --> 00:18:32.000 I suppose going back to that metaphor, you don't always need a subject matter expert. 00:18:32.079 --> 00:18:33.519 Sometimes you just need someone to do the task. 00:18:33.759 --> 00:18:34.079 Exactly. 00:18:34.559 --> 00:18:36.160 Or you just need to do a Google search, you know. 00:18:36.400 --> 00:18:37.759 Like let's forget about it, you know. 00:18:38.079 --> 00:18:47.359 A lot of the time, like even with the POCs I worked on, people have wanted these multi-stepped agentic systems when you just need a well-pumped LLM. 00:18:47.440 --> 00:18:48.319 There's no need for it. 00:18:48.559 --> 00:18:49.119 That's very true. 00:18:49.279 --> 00:18:49.680 Yeah. 00:18:50.000 --> 00:18:59.599 I also uh I think that's actually going to be the problem, is it is going to be capable, like I said, uh currently LLMs are capable of generating code for you. 00:18:59.759 --> 00:19:06.079 And agentic uh systems are going to be capable of solving a lot more of the problems that couldn't be solved. 00:19:06.240 --> 00:19:10.160 And I think that's going to be more of a problem than it is going to be a solution. 00:19:10.319 --> 00:19:13.119 Not because I'm I'm like cynical about AI. 00:19:13.279 --> 00:19:16.000 I'm a little bit cynical about AI, but not super cynical. 00:19:16.240 --> 00:19:16.400 Yeah. 00:19:16.640 --> 00:19:27.039 Everyone should be a little bit cynical about AI, but it's because you are going to get this very complex system that's going to be sold to you, right? 00:19:27.119 --> 00:19:31.519 I'm predicting packages of agent AIs that are going to be sold to you. 00:19:31.599 --> 00:19:33.599 And you are going to buy it because it will work. 00:19:33.839 --> 00:19:39.119 But what you will have actually just needed is like a database management software or something to that effect. 00:19:39.440 --> 00:19:43.200 And people will not know those things exist because everyone has moved towards agenda AI. 00:19:43.519 --> 00:19:53.519 I suppose that's why it's so important to not just buy off the shelf, but also make sure that you're talking to the experts who can guide you in what you actually need for this problem, specifically within your area. 00:19:53.839 --> 00:20:03.759 Yeah, and I think in the history of humanity, we've kind of proven as long as people are investing money into a problem, it will become a solution. 00:20:03.920 --> 00:20:14.160 If may so as long as we're investing money into improving agentic AI and we're thinking about can we want it to solve this problem, it will end up solving that problem. 00:20:14.559 --> 00:20:15.119 Yeah, absolutely. 00:20:15.359 --> 00:20:16.559 But will it be the right choice? 00:20:17.119 --> 00:20:17.759 Who knows? 00:20:18.000 --> 00:20:27.599 Speaking about humans, how how do you think the role of the humans within all of these tasks will continue to evolve from where it is now? 00:20:28.559 --> 00:20:34.640 I think you will always need to be in the driver's seat, which a lot of people seem to get too comfortable and complacent with. 00:20:35.039 --> 00:20:48.400 Going back to vibe coding, vibe coding is great until you don't understand what the code is being written, and then that's not so useful because although it works, as soon as you need to maintain it, if you have buttons, you need to solve it, you're not gonna know how to do that. 00:20:48.640 --> 00:21:07.599 So as AI evolves, you will have a lot more systems that are in place that are solving problems using AI, but you will always need to be the driver of that, in control of that, knowing where and what is happening and kind of guiding the AI to the right solution because it's it's probability at the end of the day. 00:21:07.680 --> 00:21:09.920 It's not always going to give you the best answer. 00:21:10.160 --> 00:21:14.079 You still need to do research to figure out which is the best solution going forward. 00:21:14.160 --> 00:21:16.720 It will give you a solution, it's not always the best solution. 00:21:17.279 --> 00:21:18.160 That's very true. 00:21:18.559 --> 00:21:27.200 Before we close out, um, can each of you give us a takeaway, a final thought, or just a closing idea that you want to share with the audience? 00:21:28.160 --> 00:21:30.000 That's a good question. 00:21:30.400 --> 00:21:44.400 Um I think you always have to be maybe be slightly afraid of AI, but more importantly, be excited for it and always know that although it's around, you will always be the driver. 00:21:44.640 --> 00:21:48.000 So use it to your benefit, not to your fear. 00:21:50.000 --> 00:22:00.160 I think regarding AI, I'd say be adaptable, be smart, and don't be dumb about your data, please. 00:22:01.119 --> 00:22:02.799 I was waiting for data to come. 00:22:04.079 --> 00:22:08.640 Thank you so much to both of you for you know sitting down and having this conversation. 00:22:08.720 --> 00:22:11.599 I really enjoyed it, and I'm sure everyone else will as well. 00:22:11.759 --> 00:22:15.119 Um, and thank you to all of you for joining us for this episode of Take Unboxed. 00:22:15.200 --> 00:22:16.480 We'll see you at the next one.