このエピソードについて
00:00:00:00 - 00:00:15:04 Unknown The views expressed in this podcast are those of the individual speakers, and do not necessarily reflect the views or policies of their organizations or iCal or its affiliates.
00:00:15:06 - 00:00:29:21 Unknown As we all race to deploy intelligence, how do we quantify the uncertainty of a large language model? Do we know what risk we're being exposed to? And how do we protect against that?
00:00:29:23 - 00:00:53:07 Unknown Hi, everyone. We're here today with, Doctor Michael von Gablenz from Munich. Re. You're the head of Insure AI. I, we're going to be talking today about the. I mean, what you're doing is quite fascinating, actually. You are, as I understand it, looking at the uncertainty around AI models and then figuring out how to insure against that particular risk, is that right?
00:00:53:12 - 00:01:13:00 Unknown Yeah. That's right. Leland, thank you so much for having me today. This is a great pleasure to be here. Very good. Okay. So can we just start a little bit by talking about what is insure AI, please, if you could just describe it. So Intuit is a team within Munich Re. In Munich re is a big insurance and insurance organization.
00:01:13:02 - 00:01:43:16 Unknown And what my team does is that it structures insurance solutions for different forms of AI risks. In particular, the risk that an AI model or generative AI model hallucinates or produces an incorrect output. From our perspective, that's one of the most pronounced risks when it comes to adoption of an AI model, simply because an organization wants to adopt an AI system or change AI system in order to support and decision making, or to automate certain processes or tasks.
00:01:43:18 - 00:02:07:19 Unknown But if you have, the model produces more mistakes or hallucinates more often than expected, then also the processes, the tasks, the executions will be wrong more often, which can lead to financial losses or liabilities down the road. And we believe that the insuring for those risks is very important. Also to drive an adoption of trustworthy AI systems throughout organizations.
00:02:07:21 - 00:02:28:13 Unknown Amazing. I meet chief AI offices almost every day, right? And CIOs, I don't think this is a market that many people realize exists. So talk a little bit about the background on how insurer I came came to be. It is something fairly recent that how long have you been looking at this area? Yeah. So being sure the first AI system in 2018.
00:02:28:13 - 00:02:53:14 Unknown Wow. This was an AI system which was already taking autonomous decisions in autonomous actions. So in the case, that we, that we did, it was basically an AI system designed to detect fraudulent credit card transactions, and users were merchants, who were relying on this AI system to make decisions within milliseconds to say, yeah, either the credit card transaction is fine.
00:02:53:17 - 00:03:20:11 Unknown Now, accept it, or it seems to be a fraud and block it. And from a user perspective. So from a merchants perspective, this poses a risk, because if the AI system accepts more fraudulent credit card transactions than expected, then the merchant as the user has more fraud costs and especially unbudgeted, fraud costs. So this poses pose the challenge for adopting the AI system.
00:03:20:13 - 00:03:53:21 Unknown And the AI provider approached, yeah. Approached uniquely to structure. Yeah. A solution insurance solution to cover, those AI error risks on behalf of the users, the merchants. Very good. And I mean, are there many players in this market at the moment? I mean, it creates huge, but other startups that are coming into this, into the space or any of the other major reinsurers or insurance companies that you also see, we're not seeing major insurance or reinsurance companies, entering the space yet, but we are seeing, several insurer techs are coming into the space.
00:03:53:21 - 00:04:15:13 Unknown So some from the market, some are from. Yeah. From San Francisco Bay area. So it seems that. Yeah, also, the venture community is picking up, on. Yeah. On this topic. And I think that's not surprising because in our mind, insurance is a fundamental. Yeah, instrument has driven many, you know, adoption of, of new technologies.
00:04:15:15 - 00:04:41:17 Unknown So when we're looking back in history, even something like you have steam boilers, the adoption of steam boilers, so they are also insurance was really fundamental. Yeah. Basically. Yeah. Analyzing the safety of those steam boilers and then, providing insurance against major catastrophe in case an interesting boiler blows up. So in my mind, this also has driven then the Industrial revolution forward had a major impact.
00:04:41:19 - 00:05:05:15 Unknown Yeah. It's interesting that you use that as like an analogy or comparable. That's exactly what I was thinking. I mean, this market could be absolutely huge, right? We look at, you know, $2 trillion of investments in AI training over the last two years, the fastest industry in humankind. I'm not going to put you on the spot and ask you to size the addressable market, but where do you see this?
00:05:05:15 - 00:05:26:22 Unknown Munich. We see this going, you know, are there any particular segments that you're focusing on or any particular industries? Can you can you give a sense to where you see the market developing place? You know, from all perspective, every company, globally is looking into adopting AI. And for every I use case where in case the AI.
00:05:27:00 - 00:05:58:18 Unknown Yeah, produces some form of a mistake, and costs occur. Those might be opportunity costs and those might be also actual costs for, for company to for example, we do some work. Well those might also then be liabilities in case an AI model is used, in some kind of customer facing, application areas in all those, those companies, and for all those kind of use cases, there exists a risk exposure there in where the access to risk exposure, the insurance might provide a solution in order to.
00:05:58:19 - 00:06:22:13 Unknown Yeah, take on take off this risk and therefore also drive adoption of this technology. Right. So all these things I hear like IP infringement hallucinations obviously is one of the big one, accuracy of these models, every single one of those is potentially an opportunity for you to provide some level of certainty and assurance of insurance around this, right?
00:06:22:15 - 00:07:08:08 Unknown Yeah. Correct. Huge opportunity. And, I mean, the MIT, yeah, has created a repository around AI risks. I think they have they are now standing at around 1700 different AI risks, which might seem quite overwhelming, but from our perspective, we bucket those risks into two classes. So one are risks which are really driven by an AI model being just at the end of the day, a statistical system with any statistical model, this uncertainty, uncertainty in the output and many of, the risks that you mentioned, the, inaccuracy, the hallucination risk, but also, things like, the copyright infringement risk, they can be traced back to incorrect output by a model.
00:07:08:14 - 00:07:33:10 Unknown And, yeah, this is now, mind a new class of risk of AI, then also new, you know, insurance solution, this new uprising, this new, underwriting considerations with also new considerations around systemic or aggregation risk. Yeah, that that this is needed for this bucket. And the question which I think is going to be on everyone's mind that's definitely in mind is how do you go about quantifying this risk?
00:07:33:10 - 00:07:54:05 Unknown Right. Do you have a team of highly skilled PhDs who are sitting there figuring it out, or how is it done that that's the basis it's pretty much to build up, build up on the role of what we call a research scientist. So those are colleagues with a PhD in mathematics, statistics, computer science. We hired directly from at the from the PhD program.
00:07:54:05 - 00:08:19:15 Unknown So from the universities they were doing research around you have quantification of model uncertainty of related, related research. We hired a those colleagues to put them really on the business side, in insurance in my team in order to help us develop novel quantification methodologies to price, this kind of risk and also to see what is possible to insure and what is not possible to insure.
00:08:19:17 - 00:08:39:12 Unknown So I think this this talent that we need, I think this is really the basis. Are you finding that the domain expertise, the industry, matters a lot? Does it does it factor into your ability to quantify? It plays a role. It plays a role in the sense of, for example, figuring out whether, certain testing regimes are adequate.
00:08:39:14 - 00:09:04:16 Unknown So for the input data that, for example, the model is seeing Turing testing versus Turing production. Is this let's say common or is this pattern reasonable? So in this respect, your domain knowledge can be helpful. That's a unique recruit given that yeah, we entering many different risks from agricultural to yeah. Risks around, you know, certain liabilities and areas.
00:09:05:02 - 00:09:27:23 Unknown We have a lot of different experts in the group, which we can then rely on to bring in this kind of domain than my knowledge. But now perspective, more important is the statistical specific knowledge. So given any kind of black box model, how can we quantify. Yeah. The the uncertainty in, in the output of those models. That's really the key expertise that we have to build up on our business side.
00:09:28:01 - 00:09:48:20 Unknown Does this mean that your insurer AI team has a fairly close maybe working relationship or just observation of all of the frontier model providers? Because I imagine you have to stay fairly close to those. Is that the case? Do you work collaborative with them, or is it more one of you observing them on behalf of your clients? So be observing them.
00:09:48:20 - 00:10:17:18 Unknown And have you also observing how, yeah, our incident model providers, how they are utilizing for example, those foundational models, if they're utilizing them. So many of our clients are also building their own models. But some are also relying then on a foundational model. And for our, for us, it's very important to see how they are also monitoring changes to the foundational model, because we know if the foundation model changes, this will have some downstream impacts.
00:10:17:20 - 00:10:41:08 Unknown So then also the overall models for the fine tuned model, will then also behave differently if the foundation model changes, which can then give rise to additional higher risks or change in risk. From our perspective, this needs to be monitored and controlled. Yeah, this is a really important point because we often work with clients where we've pre integrated the model and we go through their compliance.
00:10:41:15 - 00:10:57:17 Unknown I mean it's almost like one and done. But of course the models are not static. They're constantly changing. There's new versions coming out. They're leapfrogging each other. And I think you raise an interesting point that someone has to worry and quantify. The risk is that every time and these models sometimes are changing on a quarterly basis. Yeah, yeah.
00:10:57:19 - 00:11:29:05 Unknown Do you have clients where you have this ongoing relationship where you're almost having to adjust the uncertainty and the risk depending on the models and how they develop or even the tooling in the applications they're building around the models that is that the case for you? Yes, we require an ongoing monitoring as well as a retraining updating. So from our perspective, in case a new model is trained and should, go into a production, if it performs better than the old model, then of course it's highly preferable that this new model is put in draw into production.
00:11:29:09 - 00:12:00:03 Unknown So then we automatically insure it. However, we always require that there's a testing and that there's a proof that, yeah, the new model is truly performing better for the specific application compared to the, to the old model. And only then it is insured. Let's talk a little bit about the types of risk. Okay. I probably spend most of my time working with clients and partners who think about operational risk, but I'm starting to see to continue.
00:12:00:03 - 00:12:24:23 Unknown Michael, that's just that's not enough. Right. So where do we go beyond operational risk? Yeah. So there's also the different risk component there. And this might be a consequence of a realization of an operational risk. So for example if we have a company which is adopting a generative AI model internally for, for example, extracting information from documents that customers send in this.
00:12:25:05 - 00:12:44:22 Unknown Yeah. The model is now making more mistakes. And also this means that the output cannot be used. And someone so human then needs to redo the work. So that's an operational impact. But at the same time it's in financial impact because this means simply that you're more humans are needed to do the work. More. Yeah.
00:12:45:03 - 00:13:14:04 Unknown Costs occurred because of that. And that's essentially unplanned. And this creates a financial risk for a company. Interesting. So this sort of duality of financial risk and operational risk, it's not evil. It's both again, I think it's something quite innovative that you and your team have developed. What's the reaction you get from clients and tech vendors and partners when you explain these two dimensions and how they now have to sort of go hand in hand?
00:13:14:04 - 00:13:39:01 Unknown Yeah. How do they react? Yeah. So I think it's also many times an educational moment. So as we have said, it's still quite new to see the financial dimension of adopting AI. So many fewer think AI in the future of AI. So similar to adoption of other forms of technology. Yes, but in my mind it's quite distinct. So at the end of the day, will the adoption of a statistical system with all the consequences what this means?
00:13:39:03 - 00:14:05:22 Unknown So we know that, a strong, powerful, statistical system can really improve. Decision making can also lead to companies making smarter decisions. But at the same time, statistical systems are always prone to uncertainty. They're probabilistic systems. So even the best, model, the best AI model with access to all the information universe, that's a mathematical fact that even such a model will make mistakes now and then.
00:14:06:00 - 00:14:26:09 Unknown So from our perspective, it's a matter of assessing the probabilities, then seeing, how can we, yeah. Then deal with, yeah. With this uncertainty is the probability of making an error except for the respective use case. Then do we want to transfer, this residual risk, for example, why insurance is or is it not acceptable?
00:14:26:09 - 00:14:46:20 Unknown And do we need to spend more money, more funds, more effort to somehow reduce the risk by operational means? So I think this duality between operational and risk management, as we call it in financial risk management, I think that's a novel idea for AI and AI governance, but I think it's a very important one. Yeah, truly drive AI adoption forward.
00:14:46:22 - 00:15:07:21 Unknown And yeah, you mean a career. You're providing a service to insurance companies, but also directly to end clients and and to technology companies as well as AI. Right. So technology companies are most of our clients. Right. And yet this year we have started to also provide our AI insurance solutions to other insurance companies. Next, in the role of a reinsurer.
00:15:07:21 - 00:15:36:08 Unknown Yes. Means that insuring those insurance companies for yeah the losses, the risks which arise from our AI insurance products, that's also a preferred way because we would like to bring in more insurance into the space. I risk in the AI adoption is growing, and this means we need more capital providers, more insurance companies coming into the space becoming, you know, comfortable with taking, on, er, risks and insuring those risks, I think.
00:15:36:09 - 00:16:04:02 Unknown Yeah. Many careers, a reinsurer can play a major role in that. Yeah, absolutely. And are you seeing you know, typically if you look at what we've just been talking about, it falls under, the responsibility of the CFO, the CFO. And if a CIO or CIO chief AI officer, are you now seeing these 3 or 4 roles coming together in the same room and having these types of discussions?
00:16:04:03 - 00:16:23:01 Unknown Now, I would add another role, the role of the chief risk management officers. Yes. Then who also has an important role. So traditionally from a, you know, corporate insurance buying behavior, this is done by the risk manager. And you know, when it comes then to I also in my mind the risk manager has an important role at the table.
00:16:23:03 - 00:16:42:20 Unknown So then to say how can we complement. Yeah, the stack governance, process that the company has put in place. But also thinking about the financial wisdom. Yeah. Dimension. How can we utilize insurance to remove uncertainty. And then also this, you know, let's say a trust issue which might come up with the uncertainty of an AI model.
00:16:43:02 - 00:17:05:12 Unknown How can we remove that and therefore make our business units, our service units within the organization, more willing to, experiment with AI or rely on AI and, and drive adoption at scale? So I think removing this trust by, having both operations risk management and financial risk management as a safety net, I think that's quite important. Yeah, absolutely.
00:17:05:14 - 00:17:27:10 Unknown And on one hand, I think, you know, you're really not just at the cutting edge. You're you're driving this market, right? You're creating an entirely new category. But, you know, when you start seeing VCs entering this market, model providers coming to you as clients, it's also a sign that this market is starting to really pick up and reach a level of maturity.
00:17:28:14 - 00:17:42:04 Unknown What does the next two years look like for insurer I, I'm not going to ask beyond two years, because it's almost impossible to predict beyond two years. But, what does it look like for you? Are there particular segments you're going to go after? You know, give me a view as to what that looks like, please.
00:17:42:06 - 00:18:03:21 Unknown Yeah, sure. So we believe that ACI adoption is growing so fast, and also the exposure towards AI risks. And so our risks are then realizing, I think the need for insurance is also growing. So we believe that there's a strong growth ahead of us from an AI insurance perspective. And this will, go over many different industries.
00:18:03:23 - 00:18:22:14 Unknown However, we believe that especially the financial industry, might be one of be. Yeah, where we could see a big, adoption of insurance because the financial institutions are used to dealing with model risk. And then the, you know, the uncertainty which comes with an AI is quite close to the model risks there. So we believe that.
00:18:22:14 - 00:18:42:14 Unknown Yeah. Here insurance and the understanding of, of the different risks that this can be seen quite, quite a bit. You know, quite, quite strongly and therefore that this will also drive a then the, you know, the adoption of insurance as a complement to the governance and AI governance process. Yeah, absolutely. I can see we're a technology vendor.
00:18:42:14 - 00:19:08:16 Unknown Obviously, I could see technology vendors maybe even offering this as a service, potentially with their own technology stack in the near future. With that in mind, is is there any advice you would give to, tech vendors, and clients or other insurance companies? What should they be doing now to prepare for this? You know, apart from picking up the phone and calling you and your team, but how can they compare?
00:19:08:16 - 00:19:33:14 Unknown What should they be doing? Yeah. So I believe having honest conversations about the risks, not just the opportunities. So I believe that with AI there come tremendous opportunities. But this at the same time there's no free lunch. There also comes comes risk. But the nature of an AI system being a statistical model. So I think having honest conversations about what those risks are, I think that's a starting point.
00:19:33:16 - 00:19:57:13 Unknown Then, I think working with insurance companies like Munich Re and my team to see can we now quantify this risk, I think insurance in this respect is quite honest, because the premium is a very nice quantitative reflection of what the true level of risk is. And if, potentially also the premium of one I insurance is too high, this might also send a very strong signal that perhaps.
00:19:57:13 - 00:20:20:10 Unknown Yeah, the AI as it's currently designed, might not be fitted well for the respective use case. And the purpose for that. So I think he also insurance can act as a signal of how risky a certain deployment of AI is, and better than the total costs for the AI. So being basically the usage costs plus the costs of risk, for example, the form of the premium for AI insurance.
00:20:20:10 - 00:20:41:08 Unknown But it's, you know, worth it considering the opportunity. I think having this more honest discussions, I think this will help. Yeah, it's interesting because, you know, you've probably seen the other MIT report where it talks about the fact that more than 80% of AI pilot students of AI pilots fail. They never see the light of production. And my team is obsessed about the quantification of AI impact.
00:20:41:08 - 00:21:03:10 Unknown So it was almost we're focus on what is the value that AI generates. But we never really looked at the risk parts. I seen that we should be. What I think I find quite interesting is, actually what you and your team are providing. I mean, a lot of clients who are very hesitant about taking that leap of faith to invest in AI because it's uncertainty.
00:21:03:12 - 00:21:34:05 Unknown But by bringing you and your team and your service in earlier, they're lowering that risk profile, making it easier to make that decision to invest in AI or not. The case may be, yeah, absolutely fascinating. And just on that, are you where are you engaging in this sort of product development or adoption lifecycle? Are you finding that your clients are bringing you in quite early in the adoption lifecycle, or is it an afterthought or we've launched it now we better communicate and find out what our risk is.
00:21:34:11 - 00:21:58:09 Unknown Where are you typically getting engaged? And so I would say it's a combination of many of the AI vendors. We see that we are getting approached quite early. So there's an AI system already developed, but now they would like to sell, this AI system to many companies and really scale up. And then having insurance, which backs a contractual guarantee that they are giving, towards the AI users around the.
00:21:58:09 - 00:22:22:23 Unknown Yeah, accuracy, rates around the effectiveness of of the AI solution. And this can then be a really powerful. Yeah, a powerful tool. When it comes to the AI uses. So, there we also see that, basically once the AI model is supposed to be more adopted at scale, that then also discussions around risks are truly becoming, yeah.
00:22:23:14 - 00:22:45:17 Unknown A preeminent focus. And there are also, discussions around ensuring this kind of risks. Then also something what we are seeing then where we are getting engaged. Excellent. Okay. Michael, do you mind if I ask you a little bit about your background? Because I'm sure a lot of our, our viewers are sitting here thinking, how does someone like you end up in a role and running a business like this?
00:22:45:17 - 00:23:09:10 Unknown So what is your background, if you don't mind me asking? The journey? Yeah. So I have a PhD in finance and a master degree in, in data science. So it's a combination of, essentially understanding the practical as well as the theoretical, side of AI models. So I've built transformer models myself. Yeah. Which is not always easy.
00:23:09:12 - 00:23:36:16 Unknown Yeah. And at the time, it was still, it's like coding everything in Python and looking up stochastic overflow. And I was stuck somewhere rather than using a byte coding up in the AI system. It's nowadays and then, yeah, understanding the finance world, and basically seeing that when we evaluate in finance any investment project, we always look at the opportunities, but then also the costs of risk to really find out for the net present value of the project is really worth it.
00:23:36:22 - 00:23:58:19 Unknown And I think for AI, I've seen this, you know, the considerations around usage costs, and then the opportunities which come with it, but I haven't really seen yet discussion around the risks and, yeah, from, from my perspective, really combining then the AI knowledge with the finance knowledge. That's yeah, really excites me. And this is what brought me into this role.
00:23:58:19 - 00:24:18:00 Unknown Well, it's almost as well you, tailor made for this role of this business. Okay. It's not often you see someone with a PhD in finance and data science, but I can see it's absolutely work for you. Okay. Definition. Dash, if it's okay, we're going to do a little quick look around. Just a bit of fun. Okay? I'm gonna ask you 5 or 6 questions.
00:24:18:13 - 00:24:41:02 Unknown So the idea is one sentence or one word. Okay. And I think we're going to time this to like, 30s. Okay. Just a bit of fun. So let's go, shall we? Okay. One works I today uncertainty one one word for I in five years statistics I love it. Agents hype or real shift. Real shift I get biggest.
00:24:41:04 - 00:25:10:09 Unknown This is perfect for you biggest risk in AI right now inaccuracies. So my biggest opportunity automation one thing leaders consistently get wrong. Just focusing on the operational risk management dimension rather than the underlying uncertainty. Okay, I've started, so I'll finish last one. One I term you wish people would stop using.
00:25:10:11 - 00:25:35:01 Unknown Atlantic AI systems. That has been the top response from everyone we've interviewed. Very happy about that. So you're oh, you're on the money power okay. Myth versus reality. Okay, Michael. So just sort of building on this fun thing of quickfire. Now let's talk about hype versus reality. Can I give you a couple of statements and then just say hype or reality?
00:25:35:05 - 00:26:07:06 Unknown You got to choose one okay? Okay. So first up I will replace most jobs hype hype okay. Bigger models are always better. Hype AI is already delivering ROI at scale. And so it depends. I think that one I let you get away with. Yeah, yeah, I think so. Every company needs an AI strategy. Reality reality yeah I agents we will place applications.
00:26:07:08 - 00:26:33:20 Unknown Of this depends. Yeah I would say I would tend to be an application. Okay. I think it's very important to think about applications. Yeah. Well, then just let's say the tool itself, because at the end, the value is in the application, not how fancy the tool. Tools. Absolutely. Well said. Okay. You passed. I look for the name of this video series is I Changes everything.
00:26:33:22 - 00:27:03:02 Unknown But is there something that comes to mind that I shouldn't change or couldn't change? So I believe when it comes to true social interactions, that yeah, we are human beings. We also crave social interaction. So I think, yeah, replacing those social interactions, I think that's really difficult. And I believe that we need to have a genuine human being there, not a machine which mimics, human behavior.
00:27:03:03 - 00:27:26:02 Unknown So I think that's that's something which is a key element for me. I completely agree. As you know, as technologists, it's always going to be still a human element. And there are certain things just because we can replace them, it shouldn't we shouldn't. Correct. Yeah. I prot of the week. So Michael, one of the questions I always ask is do you have a favorite prompts or is there a technique that you use when prompting that you'd like to share with the audience, please.
00:27:26:04 - 00:27:44:05 Unknown So I find it quite effective to define the role that the model is, or should, take first, because then I feel that you have the models giving me more precise responses to what I'm asking. Okay. That's interesting. Can you give me an example of the type of role that you would prompt the model to take on?
00:27:44:07 - 00:28:13:04 Unknown Yeah. So for example, if I'm asking about let's say any statistical question, then I would ask it, you know, assume now the role of yeah, a PhD, statistics, student at, say at University of Oxford and now explain this. So you you are having this, this rodent. Now explain this to me in simple terms how you would explain it in your your class if you would be a teaching assistant.
00:28:13:12 - 00:28:31:19 Unknown So that's then let's say, the right level of expertise, which is there. But then from a communication perspective, it's yeah, it's a to see everything. And I find that you are defining those roles and the way of, of communication that this then helps, to provide better, response out of the model to my, to my career.
00:28:32:08 - 00:28:54:13 Unknown That's a great tip, Michael. We've come to the end of our chat. I thoroughly enjoyed it. I found it fascinating. I hope you have two. It was great pleasure. Thank you so much. Thank you so much. And thank you for watching. I hope you've enjoyed it as much as we have. If you're interested in watching more of these podcasts, and I encourage you to go to our Khou.com forward slash, I changes everything.
00:28:54:15 - 00:30:32:04 Unknown You.
