AI as Normal Technology: On superintelligence delusion, bogus claims and a humanistic AI future with Prof. Arvind Narayanan

WEBVTT
00:00:01.360 --> 00:00:05.251
Welcome to Poets Thinkers, the podcast where we explore the future of humanistic business leadership.
00:00:05.251 --> 00:00:24.410
I'm your host, ben, and today I'm speaking with Arvind Narayanan, Named on Time's inaugural list of the 100 most influential people in AI.
00:00:24.410 --> 00:00:25.795
Arvind is a professor of computer science at Princeton University.
00:00:25.795 --> 00:00:27.460
Named on Time's inaugural list of the 100 most influential people in AI.
00:00:27.460 --> 00:00:30.839
Arvind is a professor of computer science at Princeton University and the director of the Center for Information Technology Policy.
00:00:30.839 --> 00:00:40.432
He co-authored the highly praised book AI Snake Oil and continues to publish a newsletter of the same name which is read by over 60,000 researchers, policymakers, journalists and AI enthusiasts.
00:00:40.432 --> 00:00:47.691
He also led the Princeton Web Transparency and Accountability Project to uncover how companies collect and use our personal information.
00:00:47.691 --> 00:00:52.250
His work was among the first to show how machine learning reflects cultural stereotypes.
00:00:53.539 --> 00:01:00.613
Arvind's latest essay, titled AI is Normal Technology and co-authored with Sayash Kapoor, applies the same rigor and sharp thinking as their previous book.
00:01:00.613 --> 00:01:05.325
The essay lays out a counter-narrative to the hype messaging pushed by the big tech companies.
00:01:05.325 --> 00:01:07.709
It's been making waves across tech and media.
00:01:07.709 --> 00:01:15.015
I found Arvind's work and writing both refreshing and necessary, as it adds the desperately needed nuance to the AI discussion.
00:01:15.015 --> 00:01:26.933
So I'm thrilled we got to speak about the many bogus claims made by AI companies, why you can't listen to the tech CEOs when it comes to superintelligence, and what a more humanistic vision for the application of AI looks like.
00:01:27.781 --> 00:01:39.028
In our conversation, arvind challenges leaders to move beyond automation fantasies towards human AI augmentation and explains why current AI benchmarks fail catastrophically at predicting real-world performance.
00:01:39.028 --> 00:01:48.069
He makes the case for why flexible bottom-up innovation, new management approaches and labor arrangements will determine which organizations thrive in the AI era.
00:01:48.069 --> 00:02:00.248
His perspective bridges computer science rigor with deep humanistic values, showing how thoughtful, humanistic design and governance frameworks can help us navigate the transformation while keeping human agency at the center.
00:02:00.248 --> 00:02:02.325
All right, let's dive in.
00:02:02.325 --> 00:02:06.141
If you like the show, make sure you like, subscribe and share this podcast.
00:02:06.141 --> 00:02:09.568
Arvind, where does the podcast find you.
00:02:09.769 --> 00:02:11.272
Hi Benedict, it's good to be here.
00:02:11.272 --> 00:02:13.467
I'm joining you from Princeton, New Jersey.
00:02:13.870 --> 00:02:14.573
To kick us off.
00:02:14.573 --> 00:02:21.472
Why don't you tell us a little bit about yourself, what you do, what drives you, before we get into the questions I have for you?
00:02:21.632 --> 00:02:30.629
Certainly, I'm a computer science professor and the topic that I study primarily these days has been for the last many, many years is AI.
00:02:30.629 --> 00:02:44.622
But, more broadly, what I've done in the course of my career is taken a computer science perspective to the question of what are the ways in which the tech industries promises and the claims that they make to the public?
00:02:44.622 --> 00:02:51.812
How do those, how are those in keeping with or where do they diverge from the ways in which tech is actually impacting society?
00:02:51.812 --> 00:03:04.653
So if you take that lens broadly bringing accountability to the tech industry, you know, 10, 15 years ago, the most important area in which I felt I could have impact when it comes to this broad set of topics is privacy.
00:03:04.653 --> 00:03:15.127
So my grad students and I did a lot of work to build tools to find all the hidden ways in which the apps and websites that we use are tracking us.
00:03:15.187 --> 00:03:22.838
Where that data is going, what is it being used for beyond just targeted advertising, which is, of course, the reason why this data is being collected?
00:03:22.838 --> 00:03:24.504
How could it potentially be misused?
00:03:24.504 --> 00:03:25.909
What are the risks to society?
00:03:25.909 --> 00:03:26.944
What should we do about it?
00:03:26.944 --> 00:03:29.087
What technical defenses should we build?
00:03:29.087 --> 00:03:31.266
How should we perhaps regulate this?
00:03:31.266 --> 00:03:34.509
So that's where my head was, you know, again 10 or 15 years ago.
00:03:34.509 --> 00:03:51.461
But then, since then, gradually, it's been on the topic of AI, which, you know, while we have, throughout, we've acknowledged the benefits to society, there's also a clear, I think, divergence between the interests of the industry and the interests of the public, and my research tries to fill that gap a little bit.
00:03:51.723 --> 00:04:08.164
And that's one of the many reasons I'm so excited that we get to speak, because your work has been, as a former tech executive and designer, been incredibly inspiring, thought-provoking for me and it's probably a good segue in my first question that I had for you.
00:04:08.164 --> 00:04:15.508
Your book called AI Snake Oil, was one of those pieces of work that I thought was so necessary.
00:04:15.508 --> 00:04:33.456
Before we dive into the specifics, the book was published in late 2024, and you illustrate with great clarity at the very beginning of the book on essentially how imprecise and messy the use of the term artificial intelligence is and all the problems that come with that.
00:04:33.456 --> 00:04:42.927
It means everything and nothing to everyone and that creates confusion, uncertainty and anxieties around the topic and I think, societally, but also on the business side.
00:04:42.927 --> 00:04:44.632
That's in itself a huge problem.
00:04:44.632 --> 00:04:46.305
I think societally, but also on the business side.
00:04:46.305 --> 00:04:47.007
That's in itself a huge problem.
00:04:47.007 --> 00:04:53.528
So, to give listeners a better understanding around the term, maybe let's talk briefly about the biggest myths, risks and opportunities when it comes to AI from your perspective.
00:04:53.809 --> 00:04:54.511
Yeah for sure.
00:04:54.511 --> 00:05:03.786
So let's tease apart the different kinds of technologies and applications that go under the umbrella term AI, and that's what the book starts by doing.
00:05:03.786 --> 00:05:20.605
On the one hand, there's generative AI, of course, which we're all familiar with, but when we look at the kind of AI that's perhaps having the biggest impact over people's lives, or at least traditionally has, that's not generative AI, it's predictive AI.
00:05:20.865 --> 00:05:27.283
And predictive AI, you know, is broadly about using AI to predict the future in order to make some decision.
00:05:27.283 --> 00:05:29.208
Sometimes that can be innocuous.
00:05:29.208 --> 00:05:31.233
You know you have to have analytics.
00:05:31.233 --> 00:05:36.182
Whether you're a website or a store, you want to know how many customers you're going to have the following day.
00:05:36.182 --> 00:05:38.271
The following week, you have to manage inventory.
00:05:38.331 --> 00:05:51.728
There are so many applications of these predictive technologies in normal business operations, but there are much more morally dubious ways in which predictive AI is used everywhere, and that's a big part of what the book gets into.
00:05:51.728 --> 00:05:55.911
It's talking about the use of AI in criminal risk prediction, for instance.
00:05:55.911 --> 00:06:10.752
Today in the United States, in the majority of jurisdictions, when someone is arrested, they are assessed by an automated system to analyze the risk of if they were to go free until their trial, you know, which can be months or years away how risky are they?
00:06:10.752 --> 00:06:15.432
Are they going to come back for their court hearing or, whatever, are they going to commit another crime?
00:06:15.432 --> 00:06:33.791
So these kinds of risk prediction technologies are used to deny people their freedom and, to be clear, this is, you know, jailing someone not based on a determination of guilt, but based on a prediction that they might be risky in the future, and this is a very new kind of logic that has permeated the justice system.
00:06:33.791 --> 00:06:35.654
It has permeated education.
00:06:35.654 --> 00:06:36.985
It's permeated hiring.
00:06:36.985 --> 00:06:39.004
When we apply for a job.
00:06:39.024 --> 00:06:40.168
Yeah, yeah.
00:06:40.689 --> 00:06:44.911
Yeah, I mean insurance, though I mean that's been about risk kind of since the beginning, right.
00:06:44.911 --> 00:07:02.096
So there it's traditional, and we under there's there's a social understanding of what, what, what insurance involves, right, but when we talk about justice, it's supposed to be about what you've done in the past, not what you might do in the future, and that's where there's a disconnect to, let's say, the health care system.
00:07:02.096 --> 00:07:14.815
There are so many horror stories, some of which we discuss in the book, of how, and this is where insurance starts to take on new forms, where it becomes unpredictable.
00:07:14.815 --> 00:07:35.291
As opposed to, let's say, car insurance, right, which is your car insurance premium, doesn't necessarily change your life opportunities, but health insurance does, and so there are algorithms, for instance, that predict that when someone is hospitalized, they will need like 16.6 days to recover, right, and it's an oddly precise number.
00:07:35.291 --> 00:07:41.329
These statistical algorithms will always spit out a number, right, but what is the uncertainty around that?
00:07:41.329 --> 00:07:43.541
That might not be clear to the decision makers.
00:07:43.581 --> 00:07:53.009
So you have cases where this patient can't even walk on the 17th day, but nonetheless her insurance payments have stopped, right, and so these can have really terrifying consequences.
00:07:53.009 --> 00:07:55.762
So that's yeah, that's predictive AI.
00:07:55.762 --> 00:08:08.036
We talk about other kinds of AI, like robotics and self-driving cars, and our point is that if we talk about all these kinds of AI with the same breath, then we're missing the fact that A these are very different technologies, you know.
00:08:08.036 --> 00:08:11.584
In one case, it's plain old statistic that's rebranded as AI.
00:08:11.584 --> 00:08:16.021
In another case, these are genuinely new things, foundation models that have new capabilities.
00:08:16.021 --> 00:08:20.360
And we're also missing that the consequences in people's lives are very, very different.
00:08:20.480 --> 00:08:36.162
Yeah, yeah, and I think that's why I would recommend and I have recommended reading the book, even as a primer, to get a sense for what do we mean when we use this term, or even at least have an understanding and some level of literacy to actually even ask some probing questions as a really good starting point.
00:08:36.482 --> 00:08:43.005
So then, in April this year, you published an essay, together with your co-author, titled AI is Normal Technology.
00:08:43.005 --> 00:09:14.696
I read it the first time a few days after it came out and I've read it since two, three more times, and I've shared it with many people, because it's one of those pieces that and I think even from the public perception, it offers a counter-narrative, and I think you describe it as a worldview, and it adds what I only describe as a desperately needed nuance to the discussion that's predominantly driven by essentially two handful of companies who have a big capitalistic and potentially ideologic interest in perpetuating the hype.
00:09:14.696 --> 00:09:19.404
And I also found that your worldview is a deeply humanistic one.
00:09:19.404 --> 00:09:30.964
I think it gives humans a lot more agency in this AI world, rather than the very often post-human narrative of superintelligence.
00:09:30.964 --> 00:09:37.811
And all of that For those folks who have not read it, although I really suggest everyone to read it and we'll link to it in the show notes.
00:09:37.811 --> 00:09:43.452
Can you give a summary of the essay's key thesis and the kind of key points that you're making?
00:09:43.740 --> 00:09:44.302
Yeah for sure.
00:09:44.302 --> 00:09:49.364
So the points you made, I think, are really central to our way of thinking.
00:09:49.364 --> 00:09:50.625
So let's start with that.
00:09:50.625 --> 00:09:53.851
The role of human agency AI.
00:09:53.851 --> 00:09:59.129
Snake oil was about AI here and now and what works now and what is overhyped.
00:09:59.250 --> 00:10:08.884
Ai as normal technology kind of picks up with the same co-author, sayash Kapoor, picks up where the book left off and is kind of a framework for thinking about the next few decades of AI.
00:10:08.884 --> 00:10:11.890
And it's not a work of AI skepticism, literally.
00:10:11.890 --> 00:10:21.115
The second sentence of the paper is even technologies such as the electricity and the internet, general purpose technologies of that nature, are normal in our conception.
00:10:21.115 --> 00:10:38.248
So we're not trying to be AI skeptics, but it is, as you say, a counter narrative to superintelligence that says this technology is going to undergo recursive self-improvement and then superintelligence leave humans in the dust and if that happens, there's no telling what might happen.
00:10:38.248 --> 00:10:40.333
It might be utopia or dystopia.
00:10:40.333 --> 00:10:44.663
So the best thing we can do is put the brakes on this technology.
00:10:44.663 --> 00:11:15.942
So what we're saying is look, we have a lot of experience from really powerful past general purpose technologies and what we can see from technologies let's say, to use electricity as one example the logic behind the development of the technology itself and the timelines on which the technology develops are different from the logic behind how that technology diffuses into society, gets adopted by businesses, governments and others and the timelines on which that happens.
00:11:15.942 --> 00:11:24.094
These are two very different processes and I'll say specific things about superintelligence in a second.
00:11:24.114 --> 00:11:32.806
But to stick with the electricity example, we distinguish between four different stages invention, innovation, early user adoption and adaptation of businesses, societies and so forth.
00:11:32.806 --> 00:11:35.615
So let me very briefly tell you what those four stages are about.
00:11:35.615 --> 00:11:42.160
And that four-stage framework is the central intellectual, conceptual contribution of the paper for thinking about especially AI.
00:11:42.160 --> 00:11:52.879
But really, you know, this framework has long existed for past technologies and we take that and we adapt it to the case of AI and elaborate upon it.
00:11:52.879 --> 00:11:58.220
This goes back to a classic work by Everett Rogers and many others more than more than half a century ago.
00:11:58.220 --> 00:12:05.884
So in the case of electricity, things like electromagnetic loss, direct and alternating currents, those are the invention stage.
00:12:06.330 --> 00:12:09.556
But people don't use electricity directly, they use electrical appliances.
00:12:09.556 --> 00:12:13.120
So that's the innovation stage, sometimes called complementary innovation.
00:12:13.120 --> 00:12:16.139
You have to figure out the grid, the infrastructure.
00:12:16.139 --> 00:12:17.576
You have to figure out appliances.
00:12:17.576 --> 00:12:18.793
So that's product development that gradually happens.
00:12:18.793 --> 00:12:19.046
People have to learn how the grid, the infrastructure.
00:12:19.046 --> 00:12:20.238
You have to figure out appliances, so that's product development that gradually happens.
00:12:20.238 --> 00:12:28.582
People have to learn how to use this safely and then the final and slowest stage is organizations productively adopting it.
00:12:28.582 --> 00:12:37.044
And here there is powerful work by, I think, paul A David is the name of an economist that comes to mind looking at the adoption of electricity.
00:12:37.549 --> 00:12:41.778
It took factories many decades to figure out how to productively use this.
00:12:41.778 --> 00:12:51.035
At first they tried replacing their giant steam boilers with a big electric generator, but it turned out that didn't bring productivity improvements.
00:12:51.035 --> 00:13:02.724
What they had to do was rearrange the whole layout of factories, because electricity is much more portable, it can be generated and moved wherever you need it, and so to rearrange factories around the logic of the assembly line.
00:13:02.724 --> 00:13:13.510
And so that's not just rearranging factories, but it also changes the specialization of labor, the way that workers are trained right, and so the whole relationship between the firm and the worker changes.
00:13:13.510 --> 00:13:24.066
So we talk about these are all kinds of adaptations that are going to be necessary to reap the economic benefits of AI, and these things do not happen on the same time scale.
00:13:24.066 --> 00:13:26.956
That capabilities might improve from one model to another.
00:13:26.956 --> 00:13:30.196
When models improve, that has to be translated to products.
00:13:30.196 --> 00:13:39.929
And we talk about how, in the case of software engineering, people are figuring out how to do this, you know, with software like Cloud Code and Cursor and other ways to use LLMs for software engineering.
00:13:39.929 --> 00:13:41.254
And then people have to.
00:13:41.254 --> 00:13:42.438
There's a user learning curve.
00:13:42.438 --> 00:13:49.043
But then the final step, which hasn't even really started happening yet how does the nature of software firms itself have to change.
00:13:49.043 --> 00:13:54.202
What are the deep structural changes that need to happen to best take advantage of all of this?
00:13:54.202 --> 00:13:55.797
So that's our four-stage framework.
00:13:56.190 --> 00:13:58.697
Just a few quick words on superintelligence.
00:13:58.977 --> 00:14:18.360
You know, some people who have a different view will say yes, ai today is normal technology, but what's going to happen is, once recursive self-improvement hits, all of these bottlenecks are going to go out the window, because it's going to be incomprehensibly faster and superior to the human intellect, and so whatever are these bottlenecks that are holding us back are not going to hold AI back.
00:14:18.700 --> 00:14:20.523
So there's a lot we say about this.
00:14:20.523 --> 00:14:30.217
We have a whole section on superintelligence, but the fundamental reason we're skeptical about all this is that I think these arguments misunderstand the nature of human intelligence.
00:14:30.217 --> 00:14:32.990
Human intelligence is not static and fixed.
00:14:32.990 --> 00:14:40.876
Human intelligence in the modern economy primarily comes from our tools and our mastery over those tools, and AI is actually one of those tools.
00:14:40.876 --> 00:14:47.998
So, in our view, improvements in AI capabilities are improving what we should consider to be part of human intelligence.
00:14:47.998 --> 00:15:06.441
It's not a human versus AI comparison, so we have a lot more to say about it, but I'll stop there, and I do think that these long established patterns are going to continue, even if there are technical breakthroughs that allow AI systems to build future AI systems and allow some form of recursive self-improvement.
00:15:06.769 --> 00:15:15.135
Yeah, thank you for laying that out, and we'll dive deeper into some of these parts, as I have some more specific questions.
00:15:15.135 --> 00:15:22.018
But it's really, I like this last part that you said about you know, really changing the lens and the perspective of this.
00:15:22.018 --> 00:15:37.336
If we use this tool, this general purpose technology, right, then this can really unlock human ingenuity to levels that we haven't seen before and can all be in the service of how we want to live in societies and how we want to live on this planet.
00:15:37.336 --> 00:15:39.563
But it is not.
00:15:39.563 --> 00:15:47.508
You know, we are essentially, we live in a world that is dominated by the super intelligence.
00:15:47.508 --> 00:15:55.484
We can't do anything about it, which I think is not only hopeful, but it's also, I think, the way you ground it in.
00:15:55.484 --> 00:15:57.339
You know the history of humankind and the technology evolution.
00:15:57.339 --> 00:15:59.309
I think it way you ground it in you know the history of humankind and technology evolution.
00:15:59.309 --> 00:16:04.162
I think it makes a really strong and solid case for that.
00:16:04.429 --> 00:16:15.289
So you, as you just described when you spoke about superintelligence, you seem to be one of the very few people saying that AI adoption is actually moving much slower than what's being propagated.
00:16:15.289 --> 00:16:17.155
I know you have in the paper some examples.
00:16:17.155 --> 00:16:23.956
I think also on LinkedIn, you've been sharing some examples, from the Dynamo to the PC usage compared to GPT usage.
00:16:23.956 --> 00:16:35.577
Sam Altman doesn't seem to get tired telling people that AGI will be here by 2027, but also he's now calling it a bubble, and neither does Elon Musk, both being idolized as innovators.
00:16:35.577 --> 00:16:38.756
You know of their generation, from shareholders and markets.
00:16:38.756 --> 00:16:57.259
You know you make a really credible case and argument that that's not the case, that AGI is not something we need to worry about anytime soon, and in fact, there's other parts that we should actually be thinking about, both in terms of opportunities, but also in terms of being very smart about regulations where safety is actually important.
00:16:57.259 --> 00:16:59.893
So can you speak a little bit more with me about that?
00:17:00.033 --> 00:17:00.695
Yeah for sure.
00:17:00.695 --> 00:17:02.139
So there are many aspects to that.
00:17:02.139 --> 00:17:08.180
Let's start with one small thing, which is the predictions made by tech CEOs.
00:17:08.180 --> 00:17:10.510
You know, how much stock do we have to put into that?
00:17:10.510 --> 00:17:18.839
I mean, look, I think at a broad level, people understand that these CEOs, you know they have a reason for hyping up their products.
00:17:18.839 --> 00:17:26.854
Their predictions are not necessarily reliable, but, you know, is it the case that they know something that the rest of us don't?
00:17:26.854 --> 00:17:37.270
I mean, it's certainly true that they know more about their own product roadmap than they are publicly revealing, and so you know, they might know what's coming a few months down the road.
00:17:37.571 --> 00:17:54.266
But our whole perspective is that these are there, are these barriers that exist that are not about the speed of development of the technology but the way that it integrates into society, and when it comes to that aspect, these CEOs are not by any means the authorities we should be listening to.
00:17:54.266 --> 00:18:02.775
From everything I've seen, their mental models of the interaction between technology and society are less sophisticated than the rest of us, and you kind of almost.
00:18:02.775 --> 00:18:12.516
You know, I've spent some time doing tech startups and I think you need a certain kind of naivete to try the same things that many other people have tried before and it's not you know, so it's not a criticism of them.
00:18:12.516 --> 00:18:26.718
It's good that people are trying these really bold things, but they're not the people to listen to when it comes to what are the you know, diffusion barriers, as we call them, that we'll have to overcome in order to productively make use of these technologies.
00:18:26.718 --> 00:18:37.212
And specifically when it comes to AGI, I think the key reason why there's something intellectually incoherent about the way it's defined and the predictions that are made on the basis of it.
00:18:38.654 --> 00:18:40.897
There's, broadly, I think, two ways to define it.
00:18:40.897 --> 00:18:45.083
I mean there's like 20, but we can, you know, we can cluster it into two things.
00:18:45.083 --> 00:18:47.826
One is something about the internals of the system.
00:18:47.826 --> 00:18:55.236
Is it like truly thinking like a human, is it able to be flexible and in general, and all that.
00:18:55.256 --> 00:19:01.150
But another aspect you can focus on and this is the aspect that the companies like to focus on a lot is is it going to automate the majority of tasks in the economy?
00:19:01.150 --> 00:19:02.556
Is it capable of doing so?
00:19:02.556 --> 00:19:10.769
And that's the definition that's actually more relevant to us, because we're less interested in the cognitive science aspect of what is happening inside the AI systems.
00:19:10.769 --> 00:19:15.422
We're more interested in what are their economic effects going to be and on what timescales.
00:19:15.422 --> 00:19:16.994
So that's fine.
00:19:16.994 --> 00:19:24.320
We adopt for the most part those companies' definition, but where we differ is okay.
00:19:24.320 --> 00:19:32.133
So if the concept you're really interested in is is this thing going to be able to do all those messy things in the real world that people have to do?
00:19:32.133 --> 00:19:38.498
How can you make any claims about whether AGI has been achieved or not based on some silly benchmarks, right?
00:19:38.549 --> 00:19:50.875
And a lot of our technical work has been, just you know, relentlessly pounding on the serious, serious limitations of these benchmarks, how they completely fail to capture what is actually hard about doing those jobs in the real world.
00:19:50.895 --> 00:19:51.920
Yeah, real world application.
00:19:52.450 --> 00:19:53.313
Yeah, exactly.
00:19:53.313 --> 00:20:15.885
So when ChatGPT came out, you know there was so much hype around the fact that it can pass the bar exam, and that might be true, but a lawyer's job is not to answer bar exam questions all day, and we've addressed repeatedly how attempts to draw conclusions about real world applicability or, you know, job displacement based on these benchmark numbers are not very illuminating.
00:20:15.885 --> 00:20:22.740
They don't really tell you what the downstream bottlenecks are, and to do that you have to do it on a sector by sector basis.
00:20:22.740 --> 00:20:35.512
You have to do it with the involvement of those legal experts or medical experts or whoever it is in those specific sectors who know what those bottlenecks are, and so, basically, the AI CEOs are not the ones I would be listening to on this front.
00:20:35.713 --> 00:20:41.315
I had a follow-up question, which kind of is a good tie-in and then we go from there.
00:20:41.315 --> 00:20:51.786
So, specifically, you know Silicon Valley loves to dismiss critics and while you are not an AI expert, critic or skeptic, you know, you.
00:20:51.786 --> 00:20:58.502
I think your view of the world as you're describing it, it could be understood as that.
00:20:58.502 --> 00:21:02.031
So Silicon Valley doesn't love that and loves to dismiss it.
00:21:02.031 --> 00:21:09.961
But you are not just a person that's pointing out a new worldview, you are one of the world's leading experts on the topic.
00:21:09.961 --> 00:21:15.390
So when industry leaders, when you speak with them and they tell you you don't get it, what's your response?
00:21:15.390 --> 00:21:33.172
Maybe in a more broad sense, because, on the one hand, there are these benchmarks that are failing to resemble the real world, but on the other hand, there are real world negative consequences that we're already seeing at scale because of this technology that is also being dismissed, often as part of the general dismissal.
00:21:33.172 --> 00:21:35.545
So talk to me a little bit about that.
00:21:36.527 --> 00:21:41.486
So we have an interesting I would say kind of relationship with the tech companies.
00:21:41.486 --> 00:21:53.626
They're not necessarily dismissive of us, which I have been pleasantly surprised to see, and this might be because we're computer scientists, so it's not easy for them to say, oh, you don't understand the technology.
00:21:53.626 --> 00:21:55.349
Well, that's clearly not true.
00:21:55.349 --> 00:22:09.333
We understand the technology, we teach the technology, we build AI for a lot of things, and we also have lots of empirical efforts aimed at improving the state of benchmarking, and I think the companies realize that that's also in their own interests.
00:22:09.333 --> 00:22:26.641
I think the poor state of benchmarks today, while it might enable some amount of AI hype, ultimately it's in nobody's interest, and so some of the tech companies go so far as to give us API credits in order to test their models with our improved benchmarking and evaluation systems.
00:22:26.641 --> 00:22:28.984
So I think all of that is nice.
00:22:28.984 --> 00:22:35.835
It would be harder to do this work if CEOs were out there dismissing us as idiots.
00:22:35.835 --> 00:22:37.041
So that is helpful.
00:22:37.041 --> 00:22:53.696
But at the same time, we do think, yeah, they have been not nearly as responsible as they should be in acknowledging the limitations of these tools, and that's, I think, not something that's necessarily going to change through hoping for AI companies to do the right thing.
00:22:53.696 --> 00:23:08.001
I think they have to be essentially incentivized or forced into acknowledging and mitigating the limitations of these tools through public pressure, investigative journalism, research efforts such as yours, and regulation when necessary.
00:23:08.001 --> 00:23:12.847
So, to the extent possible, we try to contribute to those efforts as well.
00:23:13.048 --> 00:23:28.193
I do think there are some areas that are pretty good in terms of regulation, so, like you know, when you're talking about the use of AI in medicine or any other regulated area, you don't specifically need AI regulation.
00:23:28.193 --> 00:23:30.087
A lot of it is already regulated right.
00:23:30.087 --> 00:23:46.441
So any medical device, for instance, has to be approved by the FDA, and so, gradually, the FDA has built up expertise in testing AI systems as part of their medical devices, and so there's a healthier relationship, I think, between tech and medical applications.
00:23:46.441 --> 00:23:50.151
But in some other areas, we're seeing that there is clearly a crisis.
00:23:50.151 --> 00:23:56.528
These AI chatbots that people are trying to rely on as companions are asking for mental health advice.
00:23:56.528 --> 00:24:02.851
So that's an area where existing regulatory frameworks are not sufficient, and so there's an urgent need for action.
00:24:03.560 --> 00:24:21.702
Yeah, that's a really good point which I wanted to get to and I know you're also pointing out very clearly in the essay, together with these other parts around regulation and how, as societies, we got to hold these things in check Before we get there.
00:24:21.702 --> 00:24:33.911
Because you mentioned responsibility, one thing that you write in the essay, which is a quote that I highlighted in the first reading you write the normal technology frame is about the relationship between technology and society.
00:24:33.911 --> 00:24:52.539
As a designer, this reminded me very strongly of the way the Laszlo Moholy-Nagy, the founder of the new Bauhaus, articulated their mission as they started in the 1930s in Chicago, where they said we accept the challenge of technical progress with its recognition of social responsibility.
00:24:52.539 --> 00:24:53.462
So that is.
00:24:53.462 --> 00:24:55.650
I don't want to say word for word, but it's very aligned.
00:24:55.650 --> 00:25:26.491
So I couldn't read the essay without seeing design's role implied throughout your framework as you walked earlier through the stages, because underlying all of that is the understanding of the technology and what it can do and what it cannot do, and it is the understanding of human needs and then bringing those things together in a way that you know design translates need, application, and in a way that it actually resonates on an emotional level with people.
00:25:26.539 --> 00:25:39.827
So my question that kind of came up for me was whether we need a new form of industrial design that comes with this general purpose technology in the sense of the Bauhaus, that could become the driver of this diffusion engine that you're talking about.
00:25:39.827 --> 00:25:45.527
You know, as I've been going through history to kind of research some of this, what would this humanistic design look like?
00:25:45.527 --> 00:25:47.779
It's really interesting because there are parallels.
00:25:47.779 --> 00:25:58.330
You know you mentioned the industrial revolution, electrification, and I've, you know, read deep into the streamlined locomotive design, which had zero.
00:25:58.330 --> 00:26:03.202
This beautiful streamlined design had zero aerodynamic benefit.
00:26:03.202 --> 00:26:05.166
It was purely to sell people.
00:26:05.166 --> 00:26:21.695
A dream of this is the future of transportation, even if you go to the Macintosh, which was packaging technology that existed at Xerox PARC, but in a way that it actually created some sort of attachment and made it useful and approachable and accessible for humans.
00:26:21.695 --> 00:26:26.412
So does that make sense in the context of what you're seeing?
00:26:26.412 --> 00:26:33.430
And is that part of the way we need to rethink the workplace and these companies, how we're dealing with these, but there's no technology.
00:26:33.650 --> 00:27:00.291
Yeah, that's a great point, and I think we've already repeatedly seen with AI that the extent to which people will take advantage of this technology and the way in which they will take advantage of the technology and the extent to which they will be able to understand and avoid the pitfalls, all of those are deeply affected, not necessarily only by the capabilities themselves, but to an at least equal degree, perhaps to a greater degree, by design elements, and there are so many aspects of this.
00:27:00.291 --> 00:27:08.574
Yes, creating attachment in some sense is important, but with AI perhaps especially, there's the new risk of what does it mean to create too much attachment?
00:27:08.574 --> 00:27:21.402
We've seen controversies recently with these bots having kind of sycophantic behavior, you know, flattering users, creating emotional dependency, and users coming to over, rely on those things We've seen.
00:27:21.402 --> 00:27:27.574
You know, even simple things like will the user see this as a tool?
00:27:27.574 --> 00:27:29.547
Will they see it as a coworker?
00:27:29.547 --> 00:27:33.609
Will they see it as you know, an intern or another kind of person?
00:27:33.609 --> 00:27:56.771
All of that is strongly affected by the design elements of these tools and, in particular, I think the user interface of these AI tools themselves are the best way to communicate to users the limitations of these tools, and that's one area where companies I think can do a lot better and again, I think right now they're not doing a great job.
00:27:56.852 --> 00:28:31.555
So I'm going to give you one example of something that's been on my mind, based on a controversy that was just very recent, as we're recording this in late August, and this relates to Google's upcoming Pixel 10 phones, and they have a very interesting feature where you can dramatically zoom into an image but then AI kind of fills in the details generative AI, I guess sort of you know these are upscaling models and they work by to put it metaphorically, by imagining what might have been in that little part of the image.
00:28:31.555 --> 00:28:33.742
I mean, it's a cool feature I can imagine.
00:28:33.742 --> 00:28:35.588
It certainly has a lot of entertainment value.
00:28:35.588 --> 00:28:40.819
It might also be useful to make certain images much more aesthetically pleasing.
00:28:40.819 --> 00:29:11.915
But you can also very easily imagine the risks and downsides of this, because AI is making up details and images and you know people are probably going to be using this to zoom into, let's say, people's faces, and then you know it's completing that based on what is most statistically probable, and so it might put in an image of a celebrity there, or it might generate a person there based on what is statistically most likely based on racial, cultural stereotypes.
00:29:12.140 --> 00:29:21.250
So these are all things that might happen and I think companies are missing an opportunity to use these user interfaces to educate people about what can go wrong.
00:29:21.250 --> 00:29:31.124
And when I've seen screenshots, the phones are not out yet, but when I see screenshots of how the feature looks, there is none of that opportunity to educate the user being taken advantage of.
00:29:31.124 --> 00:29:40.505
And perhaps most depressingly, google, after informing users that this feature is best used for things like landscapes, you know, nature and stuff where you know.
00:29:40.505 --> 00:29:41.750
Yeah, it makes it prettier.
00:29:41.750 --> 00:29:43.380
The details are maybe not that important.
00:29:43.840 --> 00:29:59.301
What they actually demoed it on is to zoom in on a car that was like two pixels in the original image and then the zoomed in image has this very beautiful looking specific make and model of a car which you know, as far as anyone knows, were details that were made up by AI.
00:29:59.301 --> 00:30:15.310
So clearly the PR people here, or whichever you know division within Google, is coming up with that example to advertise the feature, is doing it in a very irresponsible way, which I'm not sure the engineers would support, based on what they have said about what you want to use the feature for.
00:30:15.310 --> 00:30:16.492
So these kinds of things.
00:30:16.492 --> 00:30:24.890
Right, there's a long way to go in getting the design right to really communicate to users what are appropriate and inappropriate ways to use these tools.
00:30:25.551 --> 00:30:55.063
Yeah, those are great examples and really it brings me back to conversations that I've already had on this podcast, just the recent one that was with a writer in the AI team at Microsoft and she studied poetry and we had a long conversation about how there was a time when it was very clear that you're talking to a computer system and it was weird and the weirdness was good to communicate to the user that you're talking to computer system and there's some strange stuff going on.
00:30:55.103 --> 00:31:21.544
So there's a built-in emotional barrier and I think in a lot of ways, over the last 24 months at least, the marketing messaging has been the exact opposite right, and there's a need for an ethical code because we also, as you mentioned earlier, we're working so close now to the human organism, right, that we can and unfortunately and we'll get to that in a second unfortunately it has already led to loss of life.
00:31:21.984 --> 00:31:58.810
We can impact people's physiology, biochemical reactions in the body with these systems and I think your call for responsibility and these interfaces, being clear about that and having a really humanistic approach here, I think is going to be essential, aside from also overall helping lower the barriers of entry and increasing literacy and regulatory frameworks, but I think from the product maker side, on the product designers and engineering side and then the marketing, it's essential that there needs to be a code of conduct which I think also is kind of built into your worldview.
00:31:58.810 --> 00:32:11.248
It's much more you know in service of humanity in a positive sense, rather than you know whatever capabilities seem cool on the onset but then no one really is looking at the unintended consequences down the line.
00:32:11.867 --> 00:32:17.536
Yeah, which is probably a good segue into the next question.
00:32:17.536 --> 00:32:22.509
So you know we are in this massive hype cycle and there's no arguing really over that.
00:32:22.509 --> 00:32:28.449
But we talked about responsibility and I can't help but look back at the mass adoption of social media.
00:32:28.449 --> 00:32:55.355
We got billions of people connected and we made them be more connected with social media and social media algorithms and we didn't call them AI back then, but there was a very similar kind of let's call it optimism about democratizing information, connecting people, and that didn't work out so great because we are very clear on social media's impact on teen mental health, character AI and meta AI, but leading to tragic loss of life, as I said before.
00:32:55.355 --> 00:32:57.788
So those are real negative consequences.
00:32:57.788 --> 00:33:19.859
And you talk in the paper about governance and you mentioned now before too several times you know, regulation, responsibility, what makes you think we'll get AI governance right or more right than we did with prior, at least IT-related general purpose technologies, but you can also go general purpose technology as a whole.
00:33:20.161 --> 00:33:29.711
Yeah, I mean, I'm certainly not confident of that and, especially when it comes to policymaking, I think it's really hard to be confident that we'll get anything right.
00:33:29.711 --> 00:33:32.548
There are so many reasons why things can go wrong.
00:33:32.548 --> 00:33:36.688
I do think there are some reasons for cautious optimism.
00:33:36.688 --> 00:33:44.833
I think in many cases when it comes to AI, like I was saying earlier, many of the risks don't necessarily require new regulation.
00:33:44.833 --> 00:33:55.511
They require appropriate enforcement of long existing regulations, right Regulations that long predate AI when it comes to those specific domains financial, medical, legal, etc.
00:33:55.511 --> 00:34:05.891
And that's an easier challenge in many ways than with social media, where this whole attention economy this was a new set of questions that regulators weren't used to thinking about.
00:34:05.891 --> 00:34:10.731
We do, of course, have that same set of questions with AI as well when it comes to attention.
00:34:10.731 --> 00:34:14.300
And, yes, these are things we very much need to be concerned about.
00:34:14.300 --> 00:34:21.704
And just to clarify, when we say, excuse me, when we say AI is normal technology, the point is not nothing to see here.
00:34:21.704 --> 00:34:22.347
Move along.
00:34:22.527 --> 00:34:27.829
General purpose technologies in the past have, over and over, had unpredictable social effects.
00:34:27.829 --> 00:34:32.813
That is something that it means to be a normal technology, like with cars, for instance.
00:34:32.813 --> 00:34:39.300
My colleague, zeynep Tufekci likes to point out that one of the things people were worried about were things like accidents.
00:34:39.300 --> 00:34:42.168
You know they were regarding them as faster horses.
00:34:42.168 --> 00:34:43.552
Essentially that was a risk.
00:34:43.552 --> 00:34:54.083
But things like restructuring our whole geography right and the rise of suburbanization these were not things people were thinking about, could have thought about in the early days of automobiles.
00:34:54.083 --> 00:35:04.130
So, similarly, the ways in which AI will deeply restructure our society, those are things that I'm not claiming I have any way to anticipate sitting here right, that's not at all the approach.
00:35:04.150 --> 00:35:33.532
Instead, when it comes to policymaking, what we can offer, I think, is some cautious optimism, based on the fact that, with social media, there seems to be a recognition among US policymakers that they got it wrong and there is at least a desire not to repeat the mistakes of the previous generation of technology.
00:35:33.532 --> 00:35:34.521
And that's, you know.
00:35:34.521 --> 00:35:43.172
That is something I think we can take advantage of, we as in civil society, everybody who's interested in a healthier relationship between technology and society.
00:35:43.172 --> 00:35:54.492
I think with social media, you know it was until it took, until this technology was very, very entrenched in our lives and kids' lives, the research even really got going.
00:35:54.492 --> 00:36:00.353
With AI, at least there is a recognition, because ChatGPT was such a surprise to so many people.
00:36:00.353 --> 00:36:12.344
It does seem to have kicked into high gear a lot of research on trying to understand how people are using it, so there are some reasons to be somewhat optimistic compared to social media, but it is going to take a lot of work.
00:36:12.344 --> 00:36:13.746
Nothing can be taken for granted.
00:36:13.967 --> 00:36:22.012
Yeah, and I think I share your cautious optimism because we do seem, in fact, to learn some lessons from some of those mistakes.
00:36:22.514 --> 00:36:36.947
And the other thing that I thought was really interesting, you know, in this context is just there is the do no harm on the individual level, right, asking ourselves continuously is this something that can really lead to immediate and very quick?
00:36:37.086 --> 00:36:43.829
I think that's the thing the cycles seem very quick in terms of seeing some of those really grave consequences.
00:36:43.829 --> 00:36:48.586
The other part is interesting Kissinger and Schmidt in the latest book, genesis.
00:36:48.586 --> 00:36:58.034
They talk about how we need to figure out how to handle these technologies that are so powerful and keep societies intact and bring them into the future intact.
00:36:58.034 --> 00:37:13.213
And that intact is a it stuck with me because it's such a powerful thing, it's such a powerful word, there's so much packed into it, and I think what you're saying is we don't even know exactly what that entails, but we need to do a lot more research.
00:37:13.213 --> 00:37:15.103
We don't even know exactly what that entails, but we need to do a lot more research.
00:37:15.103 --> 00:37:21.293
And I see a lot of the responsibility also on the makers of the technology, not just the rest of society, to see what that might look like.
00:37:21.594 --> 00:37:22.255
Yeah, definitely.
00:37:22.255 --> 00:37:27.952
I mean, I don't know if intact is going to be something we're going to be able to achieve.
00:37:27.952 --> 00:37:34.173
Again, when we look at past general purpose technologies, I mean we survived, but we didn't survive intact.
00:37:34.173 --> 00:37:39.309
We survived in a very different form as a society, stronger in many ways but actually weaker in some ways.
00:37:39.309 --> 00:37:44.413
And I think that might happen with AI and we can try to mitigate those negative effects.
00:37:44.413 --> 00:37:47.987
But I think there will be negative effects and I don't think there's any way to sugarcoat that.
00:37:48.900 --> 00:37:50.461
So switching gears.
00:37:50.561 --> 00:38:09.213
You talked at the beginning about how, when we're looking at general purpose technology, not only is the direct application of the capabilities important, but often it requires to rework pretty much everything around it Structures of companies, how companies are being led, how organizations are being led, skill sets and so on and so forth.
00:38:09.440 --> 00:38:17.760
And what I loved about both the book and the essays, that you apply the critical thinking, but also this moral, humanistic worldview you know, and everything that you write.
00:38:17.760 --> 00:38:29.179
So, in the context of that, what do you think are some of the key changes that need to be made by leaders around the way they lead themselves and then how the organizations might function?
00:38:29.179 --> 00:38:36.070
Because I remember and I took a quote out of the book here where you write broken AI is appealing to broken institutions.
00:38:36.070 --> 00:38:44.961
I also kind of thought that a little bit further, where you know you can only apply AI to the extent in which you allow your organization to change.
00:38:44.961 --> 00:38:48.931
So talk to me a little bit about you know what are those leadership skills.
00:38:48.931 --> 00:38:56.501
How do leaders need to look at AI beyond just bolting on the next chat interface to whatever exists already?
00:38:56.722 --> 00:38:57.545
Yeah, definitely.
00:38:57.545 --> 00:39:08.572
So we can make a distinction between uses of especially generative AI, but any kind of AI to replace workers versus augment their skills, right?
00:39:08.572 --> 00:39:11.260
I mean it seems clear that there is such a distinction.
00:39:11.260 --> 00:39:21.177
I think a lot of leaders tend to be very focused on the former Mm-hmm For obvious reasons, cost-cutting being one of them.
00:39:21.177 --> 00:39:29.157
But I think even from a purely profit and revenue-focused perspective, it's not clear that that's the best approach.
00:39:29.605 --> 00:39:44.369
Generative AI in particular has many serious limitations as an automation technology Because it's unreliable, you know, because it has some failure rate, as opposed to a much higher failure rate than traditional digital automation, which tends to do the same thing every time.
00:39:44.369 --> 00:39:58.476
You're not necessarily realizing that much of a cost saving through automation, because you still need layers of human oversight for when things go wrong and they will, and many companies have found this out to their regret.
00:39:58.476 --> 00:40:03.628
A simple example is replacing customer service representatives with chatbots.
00:40:03.628 --> 00:40:07.648
I mean, many people, including me, thought this was the first thing that was going to happen.
00:40:07.648 --> 00:40:28.710
I mean chatbot, it's right there in the name, but pretty soon we've started hearing many comical stories, like there was that one with Air Canada where the chatbot made up a policy that didn't exist and the case went all the way to the Canadian Supreme Court, which forced the company to abide by this fake policy that it's chatbot made up right.
00:40:28.730 --> 00:40:34.728
So there are many financial consequences of rushing headlong into automation, but this role of augmenting human workers, I think that has a much greater potential.
00:40:34.728 --> 00:40:47.688
In many ways, the capabilities of generative AI tools in humans are complementary, and the thing that I always advise business leaders about this kind of thing is it's not something you can figure out from the top down and impose upon your workers.
00:40:47.688 --> 00:41:01.813
Workers are excited to try these types of things, of recognizing these complementarities, and the thing that leaders can do is to create the conditions under which that kind of bottom-up innovation can flourish and then kind of get out of the way.
00:41:01.813 --> 00:41:11.168
And that's a very different approach from automation, where you have to figure out where can you realize the cost savings and then implement that and then do the hard work of firing those workers right.
00:41:11.168 --> 00:41:16.847
So this is a very, very different kind of approach and it is something I think that is super important to consider.
00:41:16.847 --> 00:41:18.750
So that's one big thing that comes to mind.
00:41:18.971 --> 00:41:37.940
Yeah, I was going to ask you is this the moment where the kind of parallels to the industrial revolution actually break, where, you know, it was all about consistency, predictability, automation, and that required a certain type of leadership and management style and setting up organization in a certain way, incentive structures and so on and so forth.
00:41:37.940 --> 00:41:49.413
But now we're actually at a point where it's almost the opposite is what needs to become the default, and we've always had very innovative organizations that were led differently and they fostered ingenuity.
00:41:49.413 --> 00:42:28.173
But when we're talking about, you know, augmenting human qualities, then it seems like we really need to rethink the leadership and management playbook in a way that, for most leaders, is extremely foreign, because everything but still being taught and propagated is the industrial revolution management playbook 101 rather than what you're describing, which which really is this how do you cultivate an entire organization that is based on embracing failure, experimentation, experimentation, creativity to really maximize their own human ingenuity, and then augmented and accelerated by these systems?
00:42:28.414 --> 00:42:29.257
Yeah, completely agree.
00:42:29.257 --> 00:42:30.710
That's a really nice way to put it.
00:42:30.931 --> 00:42:41.056
That's very exciting and seemingly a lot still needs to be figured out, because what's happening out in the world is not that yet, but that's exciting.
00:42:41.056 --> 00:42:50.121
So then, to finally wrap us up, you teach and your students are fortunate to get to tap into your wisdom on a regular basis.
00:42:50.121 --> 00:42:57.387
For our students collectively and the young people in the world, they're facing real, at least short-term, challenges.
00:42:57.387 --> 00:43:00.112
You know, you spoke about jobs being eliminated.
00:43:00.112 --> 00:43:05.259
A lot of entry-level junior jobs are being eliminated, at least for now, and we'll see how that goes.
00:43:05.259 --> 00:43:13.148
But there's also a massive uncertainty about the state of the world, and that comes from the misinformation and attacks on institutions and democracies.
00:43:13.148 --> 00:43:18.068
What advice do you have for the young people that we teach, that we get to work with?
00:43:18.068 --> 00:43:21.376
How can they set themselves up to thrive in this world?
00:43:21.724 --> 00:43:22.387
Yeah, definitely.
00:43:22.387 --> 00:43:26.557
And I would say, if I were to boil this down to a word, I would say flexibility.
00:43:26.557 --> 00:43:40.090
I mean, I don't have all the answers, I don't know that anyone has all the answers, but it's clear that things are not only in a period of flux now, but that they probably will continue to be for the foreseeable future.
00:43:40.090 --> 00:43:55.530
So I think you know, what's going to be most successful is the underlying ability to adapt to those changing conditions right, rather than trying to predict what those conditions are going to be a few years from now, because I think no one knows the answer to that.
00:43:55.710 --> 00:44:09.731
I think, just extrapolating a little bit from the experience of software engineering, what a typical software engineer does in their day-to-day is changing quite rapidly because a lot of the mundane stuff can be left to AI.
00:44:09.731 --> 00:44:10.835
But it doesn't mean you can.
00:44:10.835 --> 00:44:14.547
It's far from implying that you can outsource your entire job to AI.
00:44:14.547 --> 00:44:25.572
In fact, deep understanding of software systems has become more essential, not less, because AI can do some of the things that are more of the kind of rote work involved in software engineering.
00:44:25.572 --> 00:44:55.280
So it's, I think, reasonable to extrapolate that something similar will play out in many domains where deep human understanding very much remains essential, but when paired with the ability to quickly change your workflows, to take advantage of the advancing frontier of AI capabilities and, similarly, flexibility, not just in terms of learning new tasks in your day-to-day workflows but also being willing to explore new labor arrangements.
00:44:55.724 --> 00:45:25.090
So it's possible again in the example of software, that AI is going to advantage startups compared to big tech companies, because if it becomes possible to sit down with one particular customer, deeply understand their requirements in software and to custom build software for them, let's say, within a few weeks it's not quite possible yet, but imagine a future where advancing AI capabilities make this possible and that implies a sea change in the entire way that the industry is structured.
00:45:25.090 --> 00:45:42.833
It's not a matter of a big company creating one giant piece of software that everybody is forced to use and is forced to adapt their workflows to whatever you know Jira or Slack or whatever your enterprise software does for you but instead being nimble and actually creating software for a thousand different customers.
00:45:42.833 --> 00:45:49.418
So what kinds of labor arrangements will best be suited to take advantage of that kind of business model and revenue model?
00:45:49.418 --> 00:45:52.168
I don't know, but it requires a certain degree of flexibility.
00:45:52.389 --> 00:45:54.885
That's a fantastic way to wrap up the conversation, arvind.
00:45:54.885 --> 00:45:56.969
We covered a lot of ground.
00:45:56.969 --> 00:46:12.139
I appreciate your time, your insight, the work that you continuously put out there is, as I mentioned at the beginning, incredibly inspiring, and so I'm really looking forward to the next essays coming out, which I believe, from some LinkedIn posts, might lead into a new book, which is also exciting.
00:46:12.139 --> 00:46:17.117
So we'll link to all of the material we talked about in the show notes.
00:46:17.117 --> 00:46:20.074
So thank you so much again for your time and we'll talk soon.
00:46:20.253 --> 00:46:20.856
Thank you so much.
00:46:20.856 --> 00:46:25.617
This has been really fun and yes, AI as normal technology is going to be our next book.
00:46:27.565 --> 00:46:28.949
And that's a wrap for this week's show.
00:46:28.949 --> 00:46:31.376
Thank you for listening to Poets and Thinkers.
00:46:31.376 --> 00:46:36.692
If you liked this episode, make sure you hit follow and subscribe to get the latest episodes wherever you listen to your podcast.