Governance Engineering: From the CIA to the Boardroom

WEBVTT
00:00:00.160 --> 00:00:07.919
Every lawyer, no matter what their practice group, basically does six or seven things over and over and over again.
00:00:08.240 --> 00:00:18.320
And at least 80%, and I'd be nice there, but between 80 and 90% of what a lawyer does today is much better done by AI.
00:00:18.719 --> 00:00:19.359
Oh wow.
00:00:19.600 --> 00:00:27.280
And when you're thinking about bringing AI into your company, you gotta think Iron Man, not the Terminator, right?
00:00:27.519 --> 00:00:32.799
So you're not gonna unplug a person and plug in a bot and everything's fine, right?
00:00:33.119 --> 00:00:41.439
Um it is much more like uh like a lawyer, where eight out of ten of my tasks are better done by AI.
00:00:41.600 --> 00:00:49.439
So let's let the I AI do that thing with the appropriate controls and the appropriate QC.
00:00:49.920 --> 00:00:57.759
Um, but then the human part of what being a lawyer is all about is so essential.
00:01:18.799 --> 00:01:21.200
I'm your host, Madeline Amagashti.
00:01:22.079 --> 00:01:25.519
Season two is about proof of a promise.
00:01:26.159 --> 00:01:38.079
Conversations with people who are not theorizing about responsible AI, but building it, governing it, and being held accountable for it.
00:01:39.280 --> 00:01:46.719
Today's guest has one of the most extraordinary vantage points in the AI governance conversation.
00:01:47.519 --> 00:02:00.879
He was recruited by the CIA in 1992 before the World Wide Web was public to help build one, to help build one of the earliest data analytics programs in history.
00:02:01.760 --> 00:02:17.520
He went on to become a partner at DLA Piper, one of the world's largest law firms, where he was chief data scientist and global AI practice chair, representing the vast majority of the Fortune 100.
00:02:18.479 --> 00:02:22.159
He represented open AI and Anthropic.
00:02:22.719 --> 00:02:30.319
He sat in the White House briefings, he was in the room where the EU AI Act was being negotiated.
00:02:31.280 --> 00:02:35.919
And then at 56, he walked away from all of it.
00:02:36.400 --> 00:02:55.360
Not because he lost faith in AI, but because he decided the most important thing he could do was build a law firm that is also a technology company, one that doesn't just advise on AI governance but engineers it into the systems themselves.
00:02:56.560 --> 00:03:07.360
Bennett Borden is the founder of Clarion AI Partners, and his argument is simple, and I think it is one of the most important ideas in AI right now.
00:03:08.240 --> 00:03:12.319
AI is completely controllable if you do it right.
00:03:13.280 --> 00:03:16.319
Bennett, welcome to Annaya algorithm.
00:03:17.039 --> 00:03:18.319
Thank you so much, Magna.
00:03:18.479 --> 00:03:19.840
I really appreciate being here.
00:03:20.639 --> 00:03:24.800
I'm extremely delighted to have you here, and I'm excited for today's conversation.
00:03:25.120 --> 00:03:35.120
So, Bennett, let's start at the beginning because I don't think anyone listening will have heard an AI governance story that starts quite like yours.
00:03:35.439 --> 00:03:36.560
1992.
00:03:36.960 --> 00:03:42.960
The CIA comes to you whilst you're studying history and philosophy in England.
00:03:43.199 --> 00:03:47.599
What did they actually want and why did they think you were the right person?
00:03:48.000 --> 00:04:03.120
Yeah, it certainly surprised me because, you know, as you said, I what I was studying was really turning points in history, um, but especially how technology, um technological revolutions had affected world society.
00:04:03.520 --> 00:04:26.160
And so everything from, you know, the wheel to the saddle to gunpowder to all the way up through the computer age really was focused on the, you know, the history and and philosophical points, but also sociology and anthropology and archaeology and you know, why do how do societies decide to put themselves together?
00:04:26.319 --> 00:04:30.800
And what is it about the characteristics of a society that make it kind of go one way or the other?
00:04:31.120 --> 00:04:39.839
And so when I was pushed by the agency and they said, hey, we have this new data analytics shop we want you to be a part of, I'm like, you gotta have the wrong guy.
00:04:40.079 --> 00:04:42.959
Like, I'm I'm I don't really do data.
00:04:43.120 --> 00:04:45.040
I'm not a huge fan of math.
00:04:45.199 --> 00:04:48.560
I truly hate statistics, and I failed calculus.
00:04:48.800 --> 00:04:50.639
And they they said, Oh, we know all of that.
00:04:50.720 --> 00:04:52.000
That's not why we want you.
00:04:52.160 --> 00:04:54.639
And so I'm like, Well, why do you want me?
00:04:55.040 --> 00:04:57.439
So it turns out it was very interesting.
00:04:57.759 --> 00:05:10.720
So as you mentioned, in the early 90s, the world wide web and email was just starting to emerge from academia into the real world, and and cell phones were just coming on the market.
00:05:10.959 --> 00:05:20.319
And so we were creating as a species this very personalized new kind of data trail, right?
00:05:20.399 --> 00:05:22.800
This little contrail that we were putting off.
00:05:23.040 --> 00:05:32.560
And the agency wanted to know what we could learn about societies, about people, about groups of people based on this new kind of digital trail.
00:05:32.879 --> 00:05:39.439
Um, and they rightfully saw that this trail was only going to get more voluminous and more varied as time went by.
00:05:39.600 --> 00:05:44.480
And so, you know, and this was at a time when this was a very, very new idea.
00:05:44.720 --> 00:05:53.759
So today, every social media company, every app, every everything basically tracks everything you do and makes a lot of inferences about you.
00:05:53.920 --> 00:05:58.079
But at the time, it was really cool cutting-edge science.
00:05:58.399 --> 00:06:08.399
And so our our goal was can you identify people whose interests align or don't align with you the US?
00:06:08.720 --> 00:06:11.759
And then can we understand their behavior?
00:06:12.000 --> 00:06:13.600
Can we predict their behavior?
00:06:13.680 --> 00:06:15.680
And then, of course, can we influence their behavior?
00:06:15.920 --> 00:06:18.800
And so, really terribly interesting things to learn.
00:06:18.879 --> 00:06:22.240
And that's where I kind of became involved in data.
00:06:22.480 --> 00:06:22.800
Yeah.
00:06:23.040 --> 00:06:24.079
Quite accidentally.
00:06:24.240 --> 00:06:26.720
I didn't set out to become a data scientist.
00:06:27.519 --> 00:06:35.279
But it also, my entire kind of viewpoint on data is that it is a reflection of humanity, right?
00:06:35.360 --> 00:06:39.759
It's how we go about being humans in all cultures across the world.
00:06:39.920 --> 00:06:56.399
And so I've always kind of seen data as almost like this sacred trust that we have about people, that even though this data is spewing off about us, it's still ours and it still reveals some of the most precious elements of our lives.
00:06:56.720 --> 00:07:02.160
And so about eight years into the agency, so in 2000, I decided I'd always wanted to go to law school.
00:07:02.240 --> 00:07:13.120
So I went on to Georgetown Law and then went and got a graduate degree in data analytics from NYU because the technology was evolving so quickly.
00:07:13.439 --> 00:07:17.439
So I I worked in kind of what they call big law, right?
00:07:17.600 --> 00:07:28.480
And one of the top 10 firms my whole career, kind of at that intersection of data, law, and technology across every industry you can imagine.
00:07:28.800 --> 00:07:34.959
So that's eight years of working with the best minds in data and analytics data analytics.
00:07:35.199 --> 00:07:45.920
So I could imagine that looking now at what every social media company does today that you're you were doing in 1992, that was cutting edge size back then.
00:07:47.839 --> 00:07:48.399
Yeah.
00:07:48.720 --> 00:07:57.360
And you were essentially building what became the blueprint for modern digital intelligence before most people knew.
00:07:58.399 --> 00:08:11.040
And even when you know when I went to work and in law, so this was 2003, right after law school, you know, it was really the era of social media and kind of the attention economy.
00:08:11.279 --> 00:08:11.759
Yeah.
00:08:12.079 --> 00:08:18.319
Where trying to keep um a customer's attention on your screen as long as you could, um, that's really where you made all your money.
00:08:18.480 --> 00:08:25.199
And nobody really thought at the time that there would be any kind of negative effects to this, right?
00:08:25.279 --> 00:08:31.439
It was just in the early days of Facebook, you know, before there was even the others Instagram and X and TikTok.
00:08:31.600 --> 00:08:33.200
And it was cool, right?
00:08:33.360 --> 00:08:37.039
You could you could stay in touch with your friends, and you know, it had this.
00:08:37.120 --> 00:08:42.240
Um, there was quite a lot of huge positivity around it at the time.
00:08:42.399 --> 00:08:42.879
Yeah.
00:08:43.200 --> 00:08:56.799
And it wasn't until much later, a generation later, that we started to understand the really damaging effects that social media can have and or the attention economy generally can have.
00:08:57.039 --> 00:09:00.159
And so it's interesting to see how that's been playing out now.
00:09:00.399 --> 00:09:00.559
Yeah.
00:09:00.879 --> 00:09:05.200
That we have this generative AI.
00:09:05.840 --> 00:09:09.759
This technology is truly almost miraculous.
00:09:09.919 --> 00:09:17.679
It is the most transformative, powerful, and disruptive technology since at least electricity.
00:09:17.919 --> 00:09:18.480
Yeah.
00:09:19.120 --> 00:09:21.919
Way more than the internet, way more than computers.
00:09:22.000 --> 00:09:25.679
Like this is this is something actually very new in the world.
00:09:25.840 --> 00:09:33.759
And man, trying to keep our hands around it is a challenge, but something that's absolutely essential.
00:09:34.080 --> 00:09:34.559
Absolutely.
00:09:34.720 --> 00:09:49.120
I love the fact that you've drawn our attention to the that context of understanding what data could do to human behavior long before most organizations even had a I don't know, a data strategy.
00:09:50.559 --> 00:09:51.200
Yeah.
00:09:51.679 --> 00:10:01.759
And you were chief data scientist and global AI practice chair at DLA Piper, moving on from the CIA, obviously one of the biggest law firms in the world.
00:10:01.919 --> 00:10:07.759
So tell us a bit about the study you ran on what AI could do to the practice of law.
00:10:08.000 --> 00:10:09.360
It was wonderful, right?
00:10:09.519 --> 00:10:11.440
Like I had worked my way up.
00:10:11.600 --> 00:10:13.600
This is the pinnacle of my career.
00:10:13.919 --> 00:10:21.360
Um and I was there before Gen AI, mostly what I worked on was algorithmic fairness.
00:10:21.679 --> 00:10:28.000
And so, you know, there's algorithms, good old-fashioned, what they call classical AI now, you know, they work pretty much the same.
00:10:28.159 --> 00:10:35.919
They're they're lines of code that kind of have if, then, if this, then this, then, if, you know, and very controllable.
00:10:36.159 --> 00:10:44.799
And the whole point of it was to group people into groups that are similar enough that you could take an action on them.
00:10:44.960 --> 00:10:51.759
So, like when somebody applies for a mortgage or a credit card or a loan, you're looking for similar credit risk.
00:10:52.080 --> 00:10:58.799
Or a job, you're looking for job fit, or you know, social media, what are you going to click on and advertising, what's going to entice you?
00:10:58.960 --> 00:10:59.200
Yeah.
00:10:59.519 --> 00:11:08.879
And that was, you know, what my job was is to help companies design those and build those correctly, but really keep them compliant with the law.
00:11:09.039 --> 00:11:09.519
All right.
00:11:09.840 --> 00:11:14.240
So then Jenai came out in the fall of 22 and the whole world changed.
00:11:14.399 --> 00:11:18.000
My because this was a vastly different technology.
00:11:19.840 --> 00:11:28.080
And even in the early days, you know, we didn't really think we're like, oh, this is cute, you know, it was like I can talk to it and it'll talk back.
00:11:28.639 --> 00:11:33.039
And, you know, the early days when there was a lot of hallucinations and a lot of, right?
00:11:33.120 --> 00:11:34.799
It wasn't as good as it is now.
00:11:34.960 --> 00:11:47.600
And but even then, I remember the very first time I started to really look into this, just like goosebump kind of like this is going to change everything in the world.
00:11:47.919 --> 00:11:57.679
So as you said, we got to it represent OpenAI early on in Anthropic, and it already represented Google and Microsoft and Amazon for decades.
00:11:57.840 --> 00:12:09.919
And so in January, so like three months after Gen AI came out, the head of the firm came to me and said, Um, hey, what do you think this Gen AI thing is going to have on the effect it's gonna have on the practice of law?
00:12:10.639 --> 00:12:13.360
And and so I'm like, that's that's a pretty interesting question.
00:12:13.600 --> 00:12:16.879
So, you know, luckily we had a great data set, right?
00:12:17.039 --> 00:12:24.320
Because lawyers write down what we do every tenth of an hour, the buildable hour being the bane of our existence.
00:12:25.440 --> 00:12:27.039
But there's a ton of stuff.
00:12:27.279 --> 00:12:37.600
So we took 10 years worth of 14,000 attorneys, time entries, where they described what they did, right?
00:12:37.759 --> 00:12:46.799
So we we were able to across every practice area in the entire firm, what steps go into creating a particular legal product or service?
00:12:47.360 --> 00:12:49.759
And this was a massive data analysis.
00:12:49.840 --> 00:12:52.480
We teamed up with the Imperial College in London.
00:12:52.559 --> 00:12:58.159
I mean, it was a really big kind of picking apart how people do their jobs.
00:12:58.720 --> 00:13:10.559
And what came back to me was just astonishing that every lawyer, no matter what their practice group, basically does six or seven things over and over and over again.
00:13:10.879 --> 00:13:20.879
And at least 80%, and I'd be nice there, but between 80 and 90% of what a lawyer does today is much better done by AI.
00:13:21.440 --> 00:13:21.840
Wow.
00:13:22.320 --> 00:13:32.960
When those results came out, and this is the same time you're getting reports from like Mackenzie and Gartner, like that they're saying similar things about knowledge workers, and lawyers are always at the top one, two, or three of disruption.
00:13:33.120 --> 00:13:35.279
I kind of decided two things at once.
00:13:35.360 --> 00:13:42.559
Because as you said, this is the same time that we're having these meetings at the White House and in Congress and staff briefings.
00:13:42.639 --> 00:14:00.080
And I I was I am on the scientific advisory board to NIST, which wrote one of the AI risk management frameworks that's very popular over in the EU, involved in those negotiations and um working with at least a dozen countries, different ministries of AI.
00:14:00.320 --> 00:14:05.759
And so I became very, very passionate about we have to get this right.
00:14:06.080 --> 00:14:06.399
Yeah.
00:14:06.639 --> 00:14:09.039
This because I'm a huge AI optimist.
00:14:09.120 --> 00:14:14.159
I think the world, what we the good we can do is just almost incalculable.
00:14:14.320 --> 00:14:18.559
But the evil it can do is also not so great either, right?
00:14:18.960 --> 00:14:33.279
And so I I got very focused on public policy, like let's get the regulation correctly, but then really focusing on how do you technologically control AI.
00:14:33.519 --> 00:14:36.960
And so I came to kind of two conclusions at once.
00:14:37.200 --> 00:14:47.919
One, the billable hour and you know, the way that especially larger law firms practice law by throwing bodies at a problem and charging by the hour, that world is simply debt.
00:14:48.240 --> 00:14:51.120
It will not survive the Gen AI revolution.
00:14:51.279 --> 00:15:01.679
And the billable hour is so utterly entrenched in big law firm culture that it's it's gonna take a lot to rip that out and put it in a new.
00:15:01.840 --> 00:15:09.519
And, you know, I decided that I didn't want to spend the last 10 years of my career basically trying to save a sinking ship.
00:15:09.759 --> 00:15:10.399
Yeah.
00:15:10.879 --> 00:15:26.720
Because I realized the opposite was also true that a law firm that was based on the power of AI from the very beginning and taking advantage of everything that it can do, yeah, that could be a completely 22nd century kind of law firm.
00:15:27.200 --> 00:15:36.080
And so at the right bulb age of 56, which is the perfect time to start your own thing, you know, I left the pinnacle of my career and formed Clarion.
00:15:36.399 --> 00:15:42.159
And we're very we're a strange little amalgamation of services because we are lawyers.
00:15:42.320 --> 00:15:47.919
All the lawyers are lawyerslash something, data scientist, computer scientist, and things.
00:15:48.080 --> 00:15:52.960
And so fortunately, a great number of my clients came with me, which was very nice.
00:15:53.120 --> 00:16:05.279
And so, like a client will call up and be like, hey, we're thinking about doing this, and we can help talk through the laws that might apply and the risks, and how do you mitigate those and can you prove it?
00:16:05.440 --> 00:16:12.159
But the two things we do that I love the most are one is helping companies figure out their AI strategy.
00:16:12.799 --> 00:16:21.120
Because this really is you can completely transform your company in beautiful, flourishing ways.
00:16:21.440 --> 00:16:22.080
Definitely.
00:16:22.320 --> 00:16:25.679
But step by step, and how do you do that and what do you start?
00:16:25.840 --> 00:16:28.320
Like that, that is really important.
00:16:29.039 --> 00:16:34.639
And then the third thing we do is we are, as you mentioned, a development shop.
00:16:34.799 --> 00:16:42.000
So we actually help build AI systems for clients, everything from tiny little knowledge bots.
00:16:42.559 --> 00:17:00.399
And so it's a it's a really interesting perspective on bringing a company's purpose kind of to life, figuring out what AI can do, yeah, intermessed with with humanity to make this incredibly more powerful entity out of it.
00:17:00.799 --> 00:17:01.360
I love it.
00:17:01.519 --> 00:17:02.320
I love this.
00:17:02.559 --> 00:17:05.119
So I'll take it one step back.
00:17:05.279 --> 00:17:09.680
So 80, that is a huge number, you know, from your own firm's data.
00:17:09.839 --> 00:17:19.680
I think most leaders listening know that that number is true for their own organizations, but very few have actually done the analysis to prove it.
00:17:19.759 --> 00:17:26.000
And I'm talking about how you could save so much time on repetitive tests.
00:17:26.160 --> 00:17:27.039
But you did.
00:17:27.279 --> 00:17:30.240
And then you did something even rarer.
00:17:30.480 --> 00:17:31.599
You acted on it.
00:17:31.920 --> 00:17:33.839
For good or ill, yes, for good.
00:17:34.079 --> 00:17:35.359
No, it's been fantastic.
00:17:35.759 --> 00:17:48.640
Like, and that's really, you know, and it's also because of how I came to studying data through a very human lens that data really is this representation of what it means to be human.
00:17:48.799 --> 00:17:56.079
And so I naturally incline myself toward how does the technology develop human flourishing.
00:17:56.319 --> 00:17:56.720
Yes.
00:17:56.960 --> 00:18:01.119
And so we talk to our clients, we always use this analogy, right?
00:18:01.279 --> 00:18:11.440
When we first start working with companies, we say when you're thinking about bringing AI into your company, you got to think Iron Man, not the Terminator, right?
00:18:11.680 --> 00:18:16.559
So you're not gonna unplug a person and plug in a bot and everything's fine, right?
00:18:16.720 --> 00:18:23.839
It is much more like a lawyer, where eight out of ten of my tasks are better done by AI.
00:18:24.000 --> 00:18:30.799
So let's let the AI do that thing with the appropriate controls and the appropriate QC.
00:18:30.960 --> 00:18:37.519
But then the human part of what being a lawyer is all about is so essential.
00:18:37.759 --> 00:18:59.359
And so, how do you weave those two knowledge sets and experience sets together so that the AI is empowering that that human part of it and takes away inconsistencies and you know, uh, and actually can bring together the knowledge and experience of an entire firm as opposed to this partner or that partner.
00:18:59.519 --> 00:19:34.960
And that's true of really every company, that you can you can take its experience and its acumen and its years of learning and growing and you know, what worked and what didn't, and bring that into an AI agenc system and just watch that hard-earned um experience be leveraged into something that makes the humans more powerful, that makes the product much better, much more consistent, much more, right, that that has this overlap of learnings, right, as opposed to just coming out of one group or another.
00:19:35.119 --> 00:19:38.160
It's really one of my favorite things to do with companies.
00:19:38.559 --> 00:19:39.039
Love it.
00:19:39.279 --> 00:19:42.559
I love the Iron Man analogy a lot.
00:19:42.799 --> 00:19:46.720
So think Iron Man, not the Terminator, right?
00:19:46.880 --> 00:19:57.359
And you, I was gonna ask you to unpack that for the leaders listening who are wrestling with what AI transformation actually looks like inside their organization.
00:19:58.480 --> 00:20:03.680
Yeah, and so the some of the mistakes that we see that companies make early on.
00:20:03.920 --> 00:20:11.920
One is that they spend too much time defending how they have historically been successful.
00:20:12.559 --> 00:20:21.680
And so even some of the biggest companies in the world, they've they've got a recipe they've been around for 100 years, and they do stuff a certain way.
00:20:21.759 --> 00:20:28.480
And they're, you know, it's it's a CEO's job feels like they are the steward of that prior success.
00:20:28.799 --> 00:20:37.680
And so when something new comes along, the pressure of what do I do I change what we've done for 100 years?
00:20:37.839 --> 00:20:42.000
So to like the pressure of that stewardship can be really, really difficult.
00:20:42.799 --> 00:21:02.319
But this is an age where you simply cannot not address this technology, even at the very tiniest levels where you're using the general AI models to let your people use it, that will give you some marginal efficiency gains, sometimes 20 or 30%, which is fantastic, right?
00:21:02.640 --> 00:21:11.279
But without a concentrated pot like AI, you'll never achieve this true transformative effect.
00:21:11.920 --> 00:21:22.880
And so the other mistake we see is that most companies don't actually understand deeply the steps that they take to do whatever it is they do.
00:21:23.119 --> 00:21:28.000
They just kind of do stuff and they have, you know, sales does this and marketing does this and product does that.
00:21:28.319 --> 00:21:32.240
But really breaking it down into these steps, here's what we do.
00:21:32.799 --> 00:21:39.759
And so without that knowledge, it's very difficult to say, let's put AI in here, here, and here.
00:21:41.039 --> 00:21:46.960
And so this does take some self analysis, right?
00:21:47.039 --> 00:21:48.480
Of what the company does.
00:21:48.559 --> 00:21:50.880
And moreover, what do we want it to do?
00:21:51.039 --> 00:21:59.440
Because you can consciously harness the not just the productivity or the financial success of a firm.
00:22:00.160 --> 00:22:01.839
But what is its culture?
00:22:02.079 --> 00:22:03.759
What is its ethos, right?
00:22:04.000 --> 00:22:09.599
What is what do I want my company to be like, or what legacy has it had that I want to continue?
00:22:10.000 --> 00:22:15.440
Because those things are all capturable into these AI systems.
00:22:15.599 --> 00:22:16.000
Yeah.
00:22:16.240 --> 00:22:20.640
And in a way that no other technology has ever been able to do.
00:22:21.519 --> 00:22:22.000
Thank you.
00:22:22.160 --> 00:22:22.720
This is great.
00:22:22.880 --> 00:22:30.400
And I'm gonna go back to Clarion AI Partners, because I'm excited about the concept that you shared.
00:22:30.480 --> 00:22:40.079
Um you introduced the concept, which is governance engineering, because I think it it reframes something that most organizations are getting wrong.
00:22:40.240 --> 00:22:44.079
So what is governance engineering and why does it matter?
00:22:44.480 --> 00:22:50.720
This concept of governance engineering is really a very powerful one once you sort it out.
00:22:50.880 --> 00:22:53.359
We loved the phrase so much that we trademarked it.
00:22:53.519 --> 00:23:02.240
There's kind of a level one of governance where you've got the basics, you've got a an AI steering committee or group or something, right?
00:23:02.319 --> 00:23:05.200
That's well represented from the different parts of the business.
00:23:05.440 --> 00:23:10.880
You've got a policy and an acceptable use policy, and we will do this or that with data.
00:23:11.039 --> 00:23:14.880
But then it kind of stops there and you kind of go, okay, now what do I do?
00:23:15.359 --> 00:23:18.319
So then you move into the use case, right?
00:23:18.480 --> 00:23:24.559
So that's where you start to take apart your business, figure out where good low-hanging fruit is.
00:23:24.720 --> 00:23:37.680
And so, you know, tasks that are data-centric, highly repetitive, and you're applying the same judgments over and over again is perfectly optimizable by AI.
00:23:38.880 --> 00:23:44.480
But then you get to the point of, well, how do I one make it work, right?
00:23:44.720 --> 00:23:46.559
Like we wouldn't prefer these things to work.
00:23:46.720 --> 00:23:46.960
Yeah.
00:23:47.200 --> 00:23:52.960
But then how do I prove that they're working the way I want them to?
00:23:53.440 --> 00:24:02.240
And so this idea of governance engineering is taking the words from laws or regulations or even industry best practices.
00:24:02.400 --> 00:24:13.599
And for instance, if you think of most of the risk management frameworks or even the EU AI Act, they talk about concepts like fair, accurate, reliable, transparent.
00:24:14.559 --> 00:24:15.599
Totally agree.
00:24:15.839 --> 00:24:17.039
But what does that mean?
00:24:17.200 --> 00:24:19.680
And it means different things in different contexts.
00:24:19.839 --> 00:24:20.160
Yeah.
00:24:20.480 --> 00:24:47.279
And so governance engineering is the ability to take those words and convert them into technological specifications to build into the application that not only ensure that the model is doing what you want it to do, but it spits out the evidence moment by moment that it's doing what it's supposed to do.
00:24:47.599 --> 00:24:54.400
Because right now, AI regulation is all over the place and it's gonna stay pretty fractured.
00:24:54.640 --> 00:24:57.759
It's not gonna settle down for four or five years, I would say.
00:24:57.920 --> 00:24:58.640
It just can't.
00:24:58.880 --> 00:25:05.519
Um it's moving so quickly, so much faster than regulators or legislatures can keep up with.
00:25:05.680 --> 00:25:09.680
And so the best you're gonna get out of regulation is hey, don't screw up the world.
00:25:09.759 --> 00:25:20.319
And you know, and so the this idea of you're gonna have to come up with the way to defend yourself.
00:25:20.799 --> 00:25:28.559
And the law eventually falls back on this very fundamental principle of reasonableness.
00:25:29.359 --> 00:25:34.960
So there's a there's a legal standard called the reasonable person standard or the reasonable company standard.
00:25:35.119 --> 00:25:43.279
And it basically says, what would the average company do based on the information it had at the time?
00:25:43.680 --> 00:25:53.759
And so what that's been interpreted to mean is if I'm introducing AI into a workflow, what new risks am I introducing into the market?
00:25:54.000 --> 00:25:54.559
Yeah.
00:25:54.880 --> 00:25:58.559
What did I do to reasonably mitigate those risks?
00:25:58.720 --> 00:26:03.759
And then, and this is where everybody always forgets, how do I prove it?
00:26:04.079 --> 00:26:04.799
Exactly.
00:26:05.039 --> 00:26:14.720
And so by being able to, it's it's that analysis, like, hey, because builders want to build and they're like, oh, I could do this awesome thing, and and they're like, look, it'll do this thing.
00:26:14.880 --> 00:26:20.319
But builders haven't been trained in risk or especially legal risk.
00:26:20.480 --> 00:26:20.799
Yeah.
00:26:21.039 --> 00:26:25.359
And the same problems on the other side with the lawyers, like the lawyers have been trained in nothing but risk.
00:26:25.519 --> 00:26:26.480
Like, that's what we do.
00:26:26.640 --> 00:26:34.079
But they don't know the technology well enough to ask the questions, even to to, and so that's one of the reasons why we decided to just be both.
00:26:34.240 --> 00:26:54.319
We'll tell you how to do it, and we'll help you do it, and you know, but this you be if because if you have this fundamental evidentiary telemetry of that you're acting within the bounds of the law or regulation or best practices have set, then you can defend yourself whoever comes knocking.
00:26:54.640 --> 00:27:03.039
Um whether it's a court or a regulator or the New York Times, whatever it is, you've gets look, we we thought this through defensibly.
00:27:03.119 --> 00:27:06.319
We we did all these things to try to make it safe.
00:27:06.480 --> 00:27:08.559
And here, look, you can look at the record.
00:27:08.720 --> 00:27:11.200
We we did all the things that we said we would do.
00:27:11.839 --> 00:27:12.640
I love that.
00:27:12.880 --> 00:27:19.440
That distinction between having controls and being able to prove your controls work.
00:27:19.599 --> 00:27:23.759
That is the gap most organizations are living in right now.
00:27:23.920 --> 00:27:26.640
They have the policy, they don't have the proof.
00:27:26.799 --> 00:27:34.559
And the proof is what the EU AI art or the FCA or the PRA, all of them, are actually going to ask for.
00:27:35.119 --> 00:27:36.000
Exactly.
00:27:36.319 --> 00:27:40.319
And especially because they don't know what to ask for now, right?
00:27:40.720 --> 00:28:03.200
And so, you know, especially in the US, you've got this really strange kind of tug-of-war between the states and the federal government uh where because we aren't seeing federal legislation and we aren't seeing really clear guidance coming out of Washington, the states are stepping in, and so you've got, you know, eight or nine or ten states passing AI laws.
00:28:03.359 --> 00:28:18.960
And it it's difficult for a company because you you basically have to engineer to the strictest state, because most companies aren't going to go through all the effort to, you know, treat California one way and Illinois another way and New York another way, right?
00:28:19.039 --> 00:28:20.000
That's really hard to do.
00:28:20.160 --> 00:28:25.039
Um and so they just, well, let's just build to the strictest standard.
00:28:26.880 --> 00:28:41.039
But the the nice thing is if you look at how technology regulation has developed in the past, it's because industry has kind of gotten together and kind of said these are best practices.
00:28:41.200 --> 00:28:51.599
And so even in the headlines lately, where you're seeing, you know, anthropic and open AI and some of the others saying, hey, we're a little worried uh that these things are getting really, really smart.
00:28:51.839 --> 00:28:55.200
And yet out of Washington, what you hear is, no, it'll be fine.
00:28:55.359 --> 00:28:56.000
It'll be fine.
00:28:56.240 --> 00:28:59.200
And it won't be fine unless we make it fine on purpose.
00:28:59.359 --> 00:29:12.480
And so what's beautiful to see is that all of these frontier model makers from you know open AI, Anthropic, Google, Amazon, Microsoft, they are they do talk a lot.
00:29:12.640 --> 00:29:26.240
They are trying to make controls that actually work because you can, especially when you're building, like when we build for clients, the best way to control AI is with AI, right?
00:29:26.480 --> 00:29:28.640
Humans just you're not gonna keep up.
00:29:28.720 --> 00:29:30.240
Our little brains aren't gonna keep up.
00:29:30.400 --> 00:29:38.640
And so building an AI agent, but then we build right beside it another agent whose whole whole job it is, is to oversee the first agent, right?
00:29:38.880 --> 00:29:44.559
And so, and often we base those on different models because different models see things the world a little bit differently.
00:29:44.799 --> 00:29:49.359
And so there's all kinds of ways you can control AI.
00:29:49.680 --> 00:29:54.880
And sometimes I'll work with CEOs and they'll say, but the whole thing's a black box.
00:29:55.039 --> 00:29:58.000
Like even the people who made it don't know how it works.
00:29:58.319 --> 00:30:13.200
Doesn't matter because when a company uses generative AI, there's some result, like it does something, it touches the world somewhere, whether internally inside the company or externally.
00:30:13.440 --> 00:30:17.039
You can always measure that touch.
00:30:17.599 --> 00:30:23.519
And that touch is where you are regulated or have where your liability runs.
00:30:23.759 --> 00:30:31.920
And so, like, think like an HR algorithm that sorts through resumes and decides who gets an interview.
00:30:32.160 --> 00:30:43.359
Well, I can look at the people who were recommended for an interview and the pile of people who are recommended for not, and I can compare those two groups and figure out if there's discrimination built in somewhere.
00:30:43.839 --> 00:30:44.240
Right.
00:30:44.559 --> 00:30:46.160
Or all kinds of things.
00:30:46.240 --> 00:30:58.799
So even if you think about where these models touch the world, that is where you need to measure, and that is where you'll protect yourself the most from liability.
00:30:59.119 --> 00:31:00.319
I agree completely.
00:31:00.559 --> 00:31:01.759
Thanks for sharing that.
00:31:02.000 --> 00:31:07.759
So let's go and let's continue with the conversation we had before the session.
00:31:08.079 --> 00:31:12.720
You talked about every company having a secret source.
00:31:13.039 --> 00:31:18.799
So by that we mean expertise buried in their data and in their people.
00:31:19.599 --> 00:31:25.200
And the real competitive advantage coming from from codifying that into AI systems.
00:31:25.519 --> 00:31:29.440
So could you walk us through one of my favorite things, Magda?
00:31:29.920 --> 00:31:48.880
So because like I said, if if companies buying or licensing this general technology and letting their people use it with the right controls and you know, the whitelists and blacklists and you know policies and around it, that's really great, honestly.
00:31:49.039 --> 00:31:52.480
You'll you'll get these marginal utility gains.
00:31:52.720 --> 00:31:56.799
The problem is that they're available to everybody.
00:31:57.039 --> 00:32:03.839
And so maybe first movers will get an advantage for a little while, but then everybody catches up.
00:32:04.000 --> 00:32:06.880
And then when everybody's special, nobody's special, right?
00:32:07.039 --> 00:32:07.359
Yeah.
00:32:07.599 --> 00:32:15.519
But the thing that really sets a company apart is its acumen, its experience, its secret sauce, right?
00:32:16.079 --> 00:32:21.519
And the artifacts of that acumen are buried within its data.
00:32:21.680 --> 00:32:22.799
And I mean buried.
00:32:22.880 --> 00:32:28.319
Like most companies have very siloed systems, they've they keep everything forever.
00:32:28.480 --> 00:32:36.319
No, it, you know, it's not organized in a way that is easy to get to, or it's locked in the brains of their best people, right?
00:32:36.400 --> 00:32:39.200
That's where they've learned through experience and they're applying judgment.
00:32:39.599 --> 00:32:45.599
The cool thing about Gen AI systems is you can capture knowledge from both of those places.
00:32:45.759 --> 00:32:51.759
And so one of the things we love to do is go find these pockets of acumen, right?
00:32:52.000 --> 00:32:56.000
And it's not like we try to, it used to be, you know, the data lake was the big thing.
00:32:56.160 --> 00:33:01.039
Throw all your data into a big giant data lake, and then, you know, you can look at it.
00:33:01.279 --> 00:33:21.519
Yes, but uh what we have found, especially with agentic AI, is you find pockets of expertise around a particular subject or workflow, and you find your best data on that, the most well curated that represents the best work you have.
00:33:21.759 --> 00:33:31.279
And that's what you use as the knowledge base for these AI agents using retrieval augmented generation, you know, stick them in a folder somewhere.
00:33:31.440 --> 00:33:48.880
And then you point a model at it, and then you constitutionalize that model, um, constitutional AI being a discipline that really anthropic came up with originally that says you you're you give the agent a persona.
00:33:49.519 --> 00:33:59.759
And this is what kind of gets a little freaky because old-fashioned coders who are used to writing like 10,000 lines of code, you write the code, it does the code.
00:33:59.839 --> 00:34:00.400
You alright?
00:34:00.480 --> 00:34:02.160
It's it's perfectly controlled.
00:34:02.400 --> 00:34:03.039
Yeah.
00:34:03.440 --> 00:34:05.599
And Gen AI is not like that.
00:34:05.759 --> 00:34:12.800
Like it is so much more like forming the psychology of the way this agent thinks.
00:34:13.199 --> 00:34:15.199
And so you give it a persona.
00:34:15.280 --> 00:34:21.599
So let's say we were making a chatbot for HR, because they get the same 17 questions over and over and over again.
00:34:21.760 --> 00:34:35.840
And so you take all your HR policies and you put them into a database, you point a model at it and say, Hey, agent, you're you are a chatbot who's going to be interacting with employees, asking questions about HR.
00:34:36.480 --> 00:34:43.360
And then you program in these constitutional principles about how you act.
00:34:43.599 --> 00:34:46.079
Thou shalt this, thou shalt not that, right?
00:34:46.159 --> 00:34:50.079
It's it's just these basic rules of how you're gonna live.
00:34:50.239 --> 00:34:59.679
And with that, you all of a sudden people can query, you know, and um, it's a very the risk is really low, right?
00:34:59.760 --> 00:35:00.719
Because it's internal.
00:35:00.800 --> 00:35:06.000
And then if you answer the question wrong, well, okay, it's a little bit of frustration, but it's not the end of the world.
00:35:06.239 --> 00:35:13.519
But the same concept works when you're building systems that face the public, yeah.
00:35:13.920 --> 00:35:21.440
And that are multi-agent works, because you're basically creating an ensemble of experts, right?
00:35:21.519 --> 00:35:34.079
And this is where China has really, you know, because we constrained the most powerful chips from getting to China, they were very, very smart and came up with, well, I don't need all the chips.
00:35:34.400 --> 00:35:38.880
I don't need to use the whole model's brain for every question.
00:35:39.119 --> 00:35:43.920
If it's a question about a little thing, well, I have that part of that little expert, right?
00:35:44.000 --> 00:35:49.039
And so this ensemble of experts is a very, very powerful technique.
00:35:49.360 --> 00:35:57.199
But the most interesting and challenging part is how do you orchestrate the behavior of all of those?
00:35:57.360 --> 00:36:03.599
Like you've seen like the drone shows where there's like 10,000 drones doing the light shows and stuff like that.
00:36:03.679 --> 00:36:07.199
And I'm like, how in the world are those things not just all crash into each other?
00:36:07.280 --> 00:36:09.760
And right, but it's because of very clever programming.
00:36:09.920 --> 00:36:16.079
And it's the same way with these expert or ensemble agent workflows.
00:36:16.239 --> 00:36:21.119
You really have to think about them because they really do have little objectives all their own.
00:36:21.280 --> 00:36:27.920
The objectives that you've given them, but you have to be really careful what you tell them what their optimization function is, right?
00:36:28.079 --> 00:36:33.679
Like what definitely what makes them a good boy or a bad boy, right?
00:36:34.159 --> 00:36:45.199
So we like one, we were building a pretty complex system for a client, and there was this kind of one corner of the system that just kept this weird behavior.
00:36:46.000 --> 00:36:47.840
It there wasn't a pattern to it.
00:36:47.920 --> 00:36:49.679
There was just it just was odd.
00:36:49.840 --> 00:37:12.480
And so as we really delved into its logs and what it was doing, yeah, we figured out that it was withholding information or even lying to the agent next to it because it interpreted it, it's yeah, hey, do the best you can be to be okay, well, then I'll screw up everybody else around me too, so they'll all look even better, right?
00:37:12.800 --> 00:37:20.239
And so we had to go back in and rewire its optimization and got cooperation a little better, you know.
00:37:20.480 --> 00:37:25.039
But it's it really is like nothing we've ever dealt before.
00:37:25.119 --> 00:37:35.519
And when you really are defining the psychology of how these systems work, and that's where all the fun is too.
00:37:36.320 --> 00:37:37.280
I love this.
00:37:37.440 --> 00:37:44.000
I just love how you talked about the secret source concept and elaborated it so beautifully.
00:37:44.239 --> 00:37:48.159
It reframes the ROI question completely.
00:37:48.400 --> 00:37:48.800
Right?
00:37:48.960 --> 00:37:55.599
Um, the conversation in most boardrooms is about efficiency gains from AI tools.
00:37:56.639 --> 00:38:10.159
What you're describing is something categorically different using AI to make your organization's own knowledge, compounds, and skills in a way that was previously impossible.
00:38:10.960 --> 00:38:11.679
Exactly.
00:38:12.000 --> 00:38:12.800
Amazing.
00:38:13.199 --> 00:38:23.519
And how you can pull in what's really cool is you can use your knowledge as a base, but then you could enrich that data with outside data sources.
00:38:23.760 --> 00:38:31.679
And so we were working with one company who had a very complex supply chain across the whole world, right?
00:38:32.000 --> 00:38:32.239
Yeah.
00:38:32.480 --> 00:38:34.960
And it was all very just in time.
00:38:35.119 --> 00:38:39.039
Like if any one part of it fell down, the whole thing kind of fell down.
00:38:39.280 --> 00:38:42.320
And we were brought in to see if we could make it even more accurate.
00:38:42.559 --> 00:38:52.800
And so it was based on like all the demand, who bought what at what stores, and you know, like really detailed information about demand and location and all that kind of stuff.
00:38:52.960 --> 00:39:01.119
But we brought in weather patterns and geopolitical conflict and things like that to inform those things.
00:39:01.280 --> 00:39:09.360
So, like if there's a hurricane in the Sea of Japan that's gonna mess up traffic, you know that your stuff's gonna get there late.
00:39:09.519 --> 00:39:18.000
And so automatically switching a supplier track to somewhere that's safer that gets the supplies to where they need to go.
00:39:18.239 --> 00:39:36.239
And so it's really this bringing in outside knowledge, which has been accessible in completely new ways with generative AI, and then combining it with your own acumen, it's really amazing how you can transform a company.
00:39:36.480 --> 00:39:48.239
And we always help companies try to understand that when you're doing this transformative work, you should don't start with like the biggest, most important thing in your company.
00:39:48.480 --> 00:39:50.800
Let's go to the HR bot first, right?
00:39:50.960 --> 00:40:16.159
Let's use something that's fairly easy, yeah, you know, clear ROI, you know, but frees up my HR people to do other things and answer the same 17 questions, you know, and because you've got this governance engineering is a discipline that you you teach to the company, and you know, and our goal is that we teach it sufficiently that they don't need us anymore, like that we can do other things.
00:40:16.320 --> 00:40:24.960
But it's really figuring out that uh almost cadence of how do we build upon this and then this and then this and then this.
00:40:25.199 --> 00:40:41.840
At the same time, realizing that, and this goes back to my view of data as being a reflection of humanity, is that trans bringing AI into your company to the point that it transforms it is a 20% technological problem.
00:40:42.000 --> 00:40:45.840
I can build you anything and turn, you know, super suits all over the place, right?
00:40:46.000 --> 00:40:47.840
But it's 80% psychological.
00:40:48.000 --> 00:40:48.239
Yeah.
00:40:48.400 --> 00:40:55.280
Like really, individual employees have very different views on is this a good thing?
00:40:55.360 --> 00:40:56.159
Is it a bad thing?
00:40:56.239 --> 00:40:57.440
Is it going to replace me?
00:40:57.519 --> 00:41:00.480
Is it am I training it to take over my job?
00:41:00.800 --> 00:41:16.000
And so by properly communicating the company's objectives and saying that the the people are the center of how I use AI, that changes their mindset.
00:41:16.159 --> 00:41:17.920
Everybody wants a super suit, right?
00:41:18.079 --> 00:41:25.840
And and so um, even as we work with companies, we we dive pretty deeply into their workings and talk to a lot of people.
00:41:26.000 --> 00:41:44.239
And much of that is for the purpose of, you know, figuring out how the company works and stuff, but a whole lot of it seeing where people are psychologically and trying to lay this foundation of empowerment and flourishing as opposed to, oh my gosh, I gotta take my job.
00:41:44.400 --> 00:41:44.639
Right.
00:41:44.880 --> 00:41:48.800
And so it's, you know, it goes back to my CIA days, right?
00:41:48.880 --> 00:41:52.960
Of trying to help people understand a message and internalize it.
00:41:53.119 --> 00:42:04.239
And but also if they feel like they have a hand in building it, then you'll give much more loyalty and much more excitement about doing so.
00:42:04.639 --> 00:42:17.679
I love the fact that you talked about their transformation being 20% technical and 80% psychological, and also the fact that people are excited, scared, and uncertain at the same time.
00:42:17.840 --> 00:42:22.800
Um, so getting them into the mindset of, no, I'm getting a super suit.
00:42:23.199 --> 00:42:25.199
It unlocks creativity, right?
00:42:25.440 --> 00:42:29.119
And capacity rather than creating resistance.
00:42:30.159 --> 00:42:34.480
So I'm gonna go into this last last but one question.
00:42:34.800 --> 00:42:39.280
So you made a comparison that I want to bring to the audience.
00:42:39.760 --> 00:42:41.920
AI and social media.
00:42:42.639 --> 00:42:45.119
You said we got social media wrong.
00:42:45.360 --> 00:42:46.960
What do you mean by that?
00:42:47.119 --> 00:42:53.039
And what does getting AI wrong look like if we repeat the same mistake?
00:42:53.440 --> 00:42:59.519
Yeah, you know, social media, it wasn't even conceivable that this would be negative somehow.
00:42:59.679 --> 00:43:06.880
Like at the time that that social media was being created and and it was so exciting, right?
00:43:06.960 --> 00:43:13.280
You could keep in touch with your friends, you could see what their your high school friends and their kids were doing.
00:43:13.360 --> 00:43:16.159
Like it was this wonderful way to stay in touch.
00:43:16.320 --> 00:43:20.400
But then it it turned a little evil, right?
00:43:20.559 --> 00:43:27.039
Like what came out of it was it turned our children into screen starers, right?
00:43:27.199 --> 00:43:44.079
Like, so instead of having real experiences out in the world, they had these fake experiences online, and then these things are run by algorithms that are meant to catch your attention and keep you on the screen.
00:43:44.320 --> 00:43:49.920
And so the algorithms naturally figured out well, what keeps you on your screen?
00:43:50.079 --> 00:44:00.239
And yeah, a puppy, you know, video for a little while, but it's the things that are are shocking or provocative or Right?
00:44:00.320 --> 00:44:04.159
It's those things where you go, you can't help but look, right?
00:44:04.480 --> 00:44:11.760
And left unchecked, you just have this entropy that brings people down, right?
00:44:12.159 --> 00:44:12.559
Yeah.
00:44:12.800 --> 00:44:22.000
And it I think it has a l had a lot to do with increasing the extreme polarization we're seeing in the United States, but also worldwide.
00:44:23.199 --> 00:44:32.079
And it wasn't until we realized the dangers it was doing that anything started being done about it.
00:44:32.239 --> 00:44:46.239
And so it's it's not a coincidence that the world's most connected generation in the history of the world is also the loneliest and has the greatest mental health crisis.
00:44:46.880 --> 00:44:49.119
And so we're working on that, right?
00:44:49.280 --> 00:44:57.760
State laws are being passed, even the social media companies themselves, they see the writing on the wall and they're putting in, you know, better protections and things like that.
00:44:58.000 --> 00:45:03.519
But AI has potential to be the best thing for for a child ever, right?
00:45:03.679 --> 00:45:12.800
Like definitely it has the potential of putting into the hands of every child on earth the best education ever in history.
00:45:13.280 --> 00:45:17.199
But at the same time, it'll do what you tell it to do for the most part.
00:45:17.360 --> 00:45:23.119
And it there are nefarious actors out there, and it they are much better.
00:45:23.199 --> 00:45:28.880
It's like the the social media influencing you to do stuff is almost passive, right?
00:45:29.039 --> 00:45:34.320
Because of what it shows you and what the AI systems will straight up talk to you, right?
00:45:34.400 --> 00:45:36.639
And and convince you of things.
00:45:36.800 --> 00:45:46.320
And and so this is where public policy and especially technological controls have to be at the forefront of what you're making.
00:45:46.559 --> 00:45:49.519
We are seeing startup after startup after startup.
00:45:49.679 --> 00:45:50.719
Oh, they've got this great idea.
00:45:50.880 --> 00:45:51.840
We got this great idea.
00:45:51.920 --> 00:45:53.440
Yes, great idea.
00:45:53.760 --> 00:45:55.920
But stop and think a minute.
00:45:56.159 --> 00:45:57.440
How could it go wrong?
00:45:57.760 --> 00:46:04.800
And some startup founders, they're like, I don't want to think about what my thing could do wrong, you know, but but it can't, you know.
00:46:05.039 --> 00:46:12.639
So thinking through what it's meant to do, how do I control it so it does exactly that?
00:46:12.800 --> 00:46:14.400
And then how do I prove it?
00:46:14.559 --> 00:46:14.880
Right?
00:46:14.960 --> 00:46:24.639
You've got to have that thinking going into it so that your product is on the flourishing side and not on the manipulative side.
00:46:24.960 --> 00:46:25.519
I agree.
00:46:25.679 --> 00:46:32.000
This is one of the sensitive conversations as well, because it touches everyone.
00:46:32.239 --> 00:46:40.960
I was in Geneva for the UN AI for good summit recently, and what stayed with me most was that very question.
00:46:41.280 --> 00:46:44.880
What are we leaving behind for the next generation?
00:46:45.280 --> 00:46:47.599
And you framed it beautifully.
00:46:48.239 --> 00:46:54.719
We have the opportunity to get this right in a way we didn't with social media.
00:46:55.679 --> 00:46:59.440
So that the question is whether we will take it.
00:47:00.719 --> 00:47:02.639
It is absolutely our challenge.
00:47:02.800 --> 00:47:08.480
And it is what we decide today will have incredible ramifications for the future.
00:47:08.800 --> 00:47:09.199
Yeah.
00:47:09.440 --> 00:47:13.119
And that's why even governments are going to be slow to act.
00:47:13.280 --> 00:47:20.159
They just every time there's a giant technological advancement, right, the the law doesn't keep up, right?
00:47:20.239 --> 00:47:25.119
And so the difference between what we can do and what the law talks about gets bigger and bigger.
00:47:25.280 --> 00:47:45.519
And that means that we who work in AI, as builders, as lawyers, as leaders, we are the ones who are responsible for creating those standards, for getting together and developing frameworks and um helping the regulators come up with good public policy that makes sense, right?
00:47:45.679 --> 00:47:53.360
That is fair to these wonderful and unlocks this innovation, but also does so in a safe and effective manner.
00:47:53.519 --> 00:48:01.599
And that's really one of the big holdups is everybody goes, okay, let's be fair, let's be accurate, let's be reliable.
00:48:01.840 --> 00:48:06.559
But nobody agrees on how do I mathematically measure that, right?
00:48:06.800 --> 00:48:28.960
And so some of the biggest research project we are involved in at Clarion are around the different ways to design an AI system and the different kinds of controls that work best in different kinds of industries, and the hope that that will be picked up by academia, be picked up by public policy makers.
00:48:29.679 --> 00:48:30.559
Thank you.
00:48:31.199 --> 00:48:40.480
Well, your key message now, and it's on your business card, is AI for good and AI for all.
00:48:41.360 --> 00:48:43.920
In one minute, make the case.
00:49:03.679 --> 00:49:06.880
And that means only the wealthiest companies in the world could afford it.
00:49:07.679 --> 00:49:12.079
And I decided that the equation needed to be balanced.
00:49:12.320 --> 00:49:27.280
And so we work with some of the world's biggest companies and little tiny little mom and pop stores, like just because this AI technology is one of the most wonderful democratizing technologies in the world.
00:49:27.519 --> 00:49:34.320
Anyone who has a really good idea can bring that idea to life without having to be a technological guru, right?
00:49:34.639 --> 00:49:45.920
And so I love unlocking an idea of a little startup or a, you know, even a business that's been like we're working with some construction companies and some like landscaping companies.
00:49:46.079 --> 00:49:48.159
They're like, how can I AI help me?
00:49:48.239 --> 00:49:49.840
I'm like, you watch this, brother.
00:49:50.000 --> 00:49:56.880
And so AI for all, like it's not it's not just meant to be in the hands of the most powerful in the world.
00:49:57.039 --> 00:50:03.440
It's meant to be in everybody's hands and is probably better in the hands of more people than fewer.
00:50:05.280 --> 00:50:16.079
Bennett, this has been one of the most wide-ranging and genuinely important conversations I have had on this show.
00:50:17.920 --> 00:50:23.039
From the CIA to the courtroom to the engine room of AI governance.
00:50:23.840 --> 00:50:27.199
The threat running through all of it is the same.
00:50:27.840 --> 00:50:31.679
AI is controllable if you do it right.
00:50:32.480 --> 00:50:39.679
And the cost of not doing it right is not just a compliance problem, but it is a human one.
00:50:40.000 --> 00:50:40.719
I could not agree more.
00:50:40.960 --> 00:50:42.800
Magda, thanks so much for having me.
00:50:42.880 --> 00:50:45.920
It's a wonderful show, and I hope that we get to talk again.
00:50:46.320 --> 00:50:47.440
I'm looking forward to it.
00:50:47.599 --> 00:50:51.280
So thank you once again, and on to our listeners.
00:50:52.239 --> 00:50:57.199
Governance engineering, as you understand, has been trademarked by Clarence AI.
00:50:57.920 --> 00:51:02.400
It's not a technical concept, it is a leadership decision.
00:51:03.360 --> 00:51:11.599
The choice to build the proof layer in from the start to supersuit your people rather than replace them.
00:51:12.079 --> 00:51:20.239
And to ask yourself, honestly, can I demonstrate that my AI is doing what I designed it to do?
00:51:21.519 --> 00:51:22.639
That is the question.
00:51:23.039 --> 00:51:28.400
And if you cannot yet answer it, Viridian was built to help you get there.
00:51:28.960 --> 00:51:30.880
AI governance, you can prove.
00:51:30.960 --> 00:51:35.679
And it's available on the Microsoft Azure Marketplace, and the link will be in the show notes.
00:51:36.320 --> 00:51:45.280
We will also share the details for Claire on AI Partners and Bennett so you can connect with Bennett and they will be happy to support you.
00:51:45.679 --> 00:51:49.760
So thank you for tuning in to the Anaya algorithm.
00:51:50.000 --> 00:51:53.679
Until next time, keep leading intentionally.
00:51:54.000 --> 00:51:54.800
Thank you.