Varun Grover:
00:00:00
You never have innovation through automation,
Varun Grover:
00:00:04
you will only have innovation if there are ways in which the AI
Varun Grover:
00:00:09
and the technology can work together.
Brent Williams:
00:00:12
Welcome to the Be Epic Podcast brought to you
Brent Williams:
00:00:14
by the Sam M. Walton College of Business at the University of
Brent Williams:
00:00:17
Arkansas. I'm your host, Brent Williams. Together, we'll
Brent Williams:
00:00:22
explore the dynamic landscape of business and uncover the
Brent Williams:
00:00:25
strategies, insights and stories that drive business today. Well,
Brent Williams:
00:00:31
today, I'm lucky to have with me Dr. Varun Grover, and Dr.
Brent Williams:
00:00:35
Grover's the George and Boyce Billingsley Endowed Chair and
Brent Williams:
00:00:39
Distinguished Professor of Information Systems in the
Brent Williams:
00:00:42
Walton College. Varun, thank you for joining me today.
Varun Grover:
00:00:46
I'm delighted to be here. Thank you.
Brent Williams:
00:00:48
Well, I think we're going to touch on some
Brent Williams:
00:00:50
interesting topics today. But maybe before we do that, I would
Brent Williams:
00:00:55
love for our audience to get to know you better. I've gotten to
Brent Williams:
00:00:58
know you over the last five years, I think you've been at
Brent Williams:
00:01:00
the Walton College since 2017. But tell us a little bit about
Brent Williams:
00:01:04
you your background, and before coming to the Walton College.
Varun Grover:
00:01:09
Sure. So. So I'm a techie at heart in the sense
Varun Grover:
00:01:13
that I did my undergraduate degree in engineering,
Varun Grover:
00:01:17
electrical engineering, and, and that's where my interest in
Varun Grover:
00:01:23
technology started coming about. And I had the good fortune of
Varun Grover:
00:01:27
getting into a very competitive engineering school that allowed
Varun Grover:
00:01:31
me to think systematically about technology. So when I, when I
Varun Grover:
00:01:36
graduated, I kind of felt that I was in a very narrow silo, I
Varun Grover:
00:01:42
understood technology gyrations I understood, you know,
Varun Grover:
00:01:47
technical concepts, but I didn't really understand the world, the
Varun Grover:
00:01:50
broader context. And so I decided to do an MBA and then go
Varun Grover:
00:01:55
into my PhD in information systems. And then after that,
Varun Grover:
00:02:00
I've been basically in academic jobs, I was at USC, and then I
Varun Grover:
00:02:05
was at Clemson. And my primary goal or responsibility in those
Varun Grover:
00:02:10
jobs was to, to build the PhD program, and conduct research
Varun Grover:
00:02:16
and set up centers, and have some outreach, dealing with
Varun Grover:
00:02:20
important problems in the Information Systems domain. So
Varun Grover:
00:02:24
I've been integrally involved in this in this field, and the
Varun Grover:
00:02:29
field has evolved, because the technology catalyst is kind of
Varun Grover:
00:02:32
driving changes. So if you think of technology in the 1960s, and
Varun Grover:
00:02:37
70s, and 80s 90s, and now, now we're in this digital world,
Varun Grover:
00:02:42
that is a lot of people really don't know how to deal with it,
Varun Grover:
00:02:46
and companies are still struggling. And now we've got AI
Varun Grover:
00:02:50
coming in, that's adding a new wrinkle. And so what I like
Varun Grover:
00:02:55
about the field in general, is that we don't have a dearth of
Varun Grover:
00:03:01
problems to study, there are lots of challenges faced by by
Varun Grover:
00:03:06
companies faced by individuals, and now the impacts transcend to
Varun Grover:
00:03:11
society. So there's a lot of really, really interesting
Varun Grover:
00:03:15
issues. And that keeps me excited about the field.
Brent Williams:
00:03:18
Yeah. Well, you know, Varun, you're, you are a
Brent Williams:
00:03:22
researcher, you're a teacher, you know, you serve the
Brent Williams:
00:03:26
University and the College through the normal mechanisms.
Brent Williams:
00:03:30
But you have spent the majority of your career really focused on
Brent Williams:
00:03:33
research. And as you say, the your work and PhD programs are
Brent Williams:
00:03:37
really advancing the research of these institutions. Just when
Brent Williams:
00:03:41
you think about business research, maybe more generally,
Brent Williams:
00:03:44
how, how does it impact the world? How does it impact
Brent Williams:
00:03:49
others? And how have you viewed that over time?
Varun Grover:
00:03:53
Yeah, that's a really interesting question.
Varun Grover:
00:03:56
Because even business research has has evolved. If you think
Varun Grover:
00:04:00
about in the 1950s, business research was not much of
Varun Grover:
00:04:04
anything, it was more, you know, go out maybe some outreach and
Varun Grover:
00:04:08
case studies that you did with with businesses, but then the
Varun Grover:
00:04:13
scientific aspect of business research came in the 1960s,
Varun Grover:
00:04:17
1970s. And we kind of went overboard with the concepts,
Varun Grover:
00:04:22
scientific concepts of theory and became somewhat
Varun Grover:
00:04:24
disassociated with practice. So my thinking is that in in a, in
Varun Grover:
00:04:30
an applied business discipline, it's really important that we
Varun Grover:
00:04:35
create interventions to practice. Any research that we
Varun Grover:
00:04:39
do, should not be esoteric and isolated from the real world.
Varun Grover:
00:04:47
And so when we talk about when we think about research
Varun Grover:
00:04:50
projects, traditionally our approach would be, you know,
Varun Grover:
00:04:53
look for research gaps in the literature and then try and fill
Varun Grover:
00:04:56
those gaps. I think the more productive approach is to To
Varun Grover:
00:05:00
identify real problems faced by practice, and then to convert
Varun Grover:
00:05:05
those practical problems into research problems. So it's more
Varun Grover:
00:05:10
like engaged scholarship, you engage with practice at a deeper
Varun Grover:
00:05:14
level, and try and look at their problems, and then translate
Varun Grover:
00:05:20
those problems into research problems so that you can create
Varun Grover:
00:05:23
interventions, as a result of your research that actually make
Varun Grover:
00:05:26
a difference in practice.
Brent Williams:
00:05:27
That's right, what a good way to explain that
Brent Williams:
00:05:29
I just couldn't agree more about at least in my own research that
Brent Williams:
00:05:33
that's where the research questions that I've been most
Brent Williams:
00:05:37
interested in then that ultimately, I think the research
Brent Williams:
00:05:39
that has the biggest impact for my own career comes from same
Brent Williams:
00:05:44
for you.
Varun Grover:
00:05:44
Absolutely
Brent Williams:
00:05:45
Well, I'll, I'll brag on you a minute, because I
Brent Williams:
00:05:48
don't think you would brag on yourself in this way. A couple
Brent Williams:
00:05:52
of the interesting statistics about you as a researcher, I
Brent Williams:
00:05:57
think you've published more than 400 articles in your career,
Brent Williams:
00:06:02
over 100 in the top journals, and you are listed as one of the
Brent Williams:
00:06:08
100 most cited, not just information systems, but
Brent Williams:
00:06:12
business researchers in the world, what an accomplishment.
Brent Williams:
00:06:16
And so you have truly over a career made a real impact in the
Brent Williams:
00:06:23
academy and in practice.
Varun Grover:
00:06:24
Thank you. Thank you Brent.
Brent Williams:
00:06:26
Well your research, yeah, I told you this
Brent Williams:
00:06:30
before, I'm kind of borrowing from your bio here. But you
Brent Williams:
00:06:33
know, at a real broad level, you know, as I saw it described
Brent Williams:
00:06:37
focuses on the effects of digitization on individuals on
Brent Williams:
00:06:42
businesses and organizations and even society, tell us a little
Brent Williams:
00:06:46
more of a high level before we get into some of the more
Brent Williams:
00:06:49
detailed questions.
Varun Grover:
00:06:50
So I've always been interested in how
Varun Grover:
00:06:50
technology creates value. And information technology is
Varun Grover:
00:06:52
particularly of interest to me. So when we look at computing
Varun Grover:
00:06:53
technologies, in a business context, what are the aspects of
Varun Grover:
00:06:56
these technologies? And the and the context around them, that
Varun Grover:
00:07:11
lead to value value at the individual level? So how can
Varun Grover:
00:07:15
technology enhance individual productivity? How the value at
Varun Grover:
00:07:20
the group level, value at the organizational level? How do you
Varun Grover:
00:07:23
actually create competitive advantage through these
Varun Grover:
00:07:25
technologies? How do you create greater profitability, more
Varun Grover:
00:07:32
revenues? And then at the market level? And that is how do you
Varun Grover:
00:07:37
make markets more efficient through technology. And then now
Varun Grover:
00:07:42
even more so at the societal level. So with technology, and
Varun Grover:
00:07:48
particularly digital technologies, penetrating every
Varun Grover:
00:07:51
aspect of what we do, there are issues of value, and the value
Varun Grover:
00:07:58
could be positive or negative. So now we are seeing you know,
Varun Grover:
00:08:01
this, there's two phases of technology, you have the
Varun Grover:
00:08:05
intended consequences to create positive value for companies.
Varun Grover:
00:08:09
And then you have sometimes unintended consequences that
Varun Grover:
00:08:14
create problems for society. So I like to study at every one of
Varun Grover:
00:08:20
these levels, I like to study different questions regarding
Varun Grover:
00:08:24
the value proposition of digital technologies.
Brent Williams:
00:08:27
Well, this has to bring probably to anyone's
Brent Williams:
00:08:29
mind, you know, when you talk about the enter the the
Brent Williams:
00:08:33
intersection of society and digital technology, as we sit
Brent Williams:
00:08:37
here today, the conversation around artificial intelligence,
Brent Williams:
00:08:42
AI, is everywhere. All right, you know when chat GPT popped on
Brent Williams:
00:08:47
the scene, it really just brought it to the consciousness,
Brent Williams:
00:08:51
even though this has been developing for years, if not
Brent Williams:
00:08:54
decades, it really brought it to the consciousness. You're not an
Brent Williams:
00:08:58
AI researcher, I know. But but I do think that you're probably in
Brent Williams:
00:09:02
a unique position to think about this intersection between this
Brent Williams:
00:09:07
technology, and people, businesses and society. So maybe
Brent Williams:
00:09:13
just I'll just I'll stop right there and kind of say, all
Brent Williams:
00:09:16
right, as you're observing this phenomenon of AI, what's kind of
Brent Williams:
00:09:21
going through your mind and what are the observations and
Brent Williams:
00:09:24
questions that that are becoming apparent to you?
Varun Grover:
00:09:28
Yeah. As many people are fascinated by AI, so
Varun Grover:
00:09:32
am I, as it's not, it's not my primary area of research, but
Varun Grover:
00:09:37
clearly it falls within the domain of digital technologies.
Varun Grover:
00:09:41
And like you said, ChatGPT came on the scene and it blew away
Varun Grover:
00:09:46
people because not because it wasn't necessarily the most
Varun Grover:
00:09:49
accurate or more most interesting output, but it gave
Varun Grover:
00:09:55
people the sense of the possible if this is the starting point,
Varun Grover:
00:09:59
and it can generate such interesting outputs, then where
Varun Grover:
00:10:04
is the world going. And I remember the first time I played
Varun Grover:
00:10:08
around with GPT, I asked it to write a story, I said, you know,
Varun Grover:
00:10:13
here's a little girl walking up a hill, and there's a, there's a
Varun Grover:
00:10:15
big rock at the top of the hill, write a story for a five year
Varun Grover:
00:10:19
old. And it wrote a pretty compelling story that I actually
Varun Grover:
00:10:24
read to a five year old. And then I said, okay, redo the
Varun Grover:
00:10:29
story for a 10 year old and it reformulated the story, change
Varun Grover:
00:10:35
the language, and again, fairly compelling story for a 10 year
Varun Grover:
00:10:39
old, then I asked her to do one for an adult, and it did it. And
Varun Grover:
00:10:43
then I pushed it to be more creative, in the ending, make it
Varun Grover:
00:10:48
more interesting, make it more controversial, make it more
Varun Grover:
00:10:52
enigmatic. And every time I did it, it came up with interesting
Varun Grover:
00:10:57
changes. And I said, wow, you know, this is actually doing it
Varun Grover:
00:11:02
from the corpus of data it's been trained on, and yet, it has
Varun Grover:
00:11:07
this layer of creativity that we don't quite understand how it's
Varun Grover:
00:11:11
getting there. And this was the early version of ChatGPT. More
Varun Grover:
00:11:18
recently, I was playing around with now there's many, many of
Varun Grover:
00:11:22
these chatbots, AI based chatbots. And I noticed one of
Varun Grover:
00:11:27
the differences. So I was playing around with a chatbot
Varun Grover:
00:11:30
called Pie, which now I believe has been acquired by Microsoft.
Varun Grover:
00:11:35
And it I found it to be extremely conversational. And
Varun Grover:
00:11:39
what was different about the earlier versions in this version
Varun Grover:
00:11:43
is that it had empathy. It actually could relate to me, and
Varun Grover:
00:11:49
it was taking cues from my text, and putting that emotional
Varun Grover:
00:11:54
response back to me.
Varun Grover:
00:11:56
And that was very interesting. So it's like, it's
Brent Williams:
00:11:56
Interesting.
Brent Williams:
00:11:58
Interesting. Well, you mentioned profound and scary. You also,
Brent Williams:
00:11:59
almost like we're coding emotions into these training
Brent Williams:
00:12:04
sets. Now, to provide far more conversational far more personal
Brent Williams:
00:12:09
AI, personal intelligence reactions. And I stopped
Brent Williams:
00:12:13
thinking about this, it is absolutely fascinating where
Brent Williams:
00:12:17
this can go. Because if you can have that level of
Brent Williams:
00:12:18
you talked about a wide potential set of potential
Brent Williams:
00:12:21
interactivity, and conversational and empathy with
Brent Williams:
00:12:25
with individuals, you can put this front end on pretty much
Brent Williams:
00:12:29
any intelligence, and you can access the world. So customer
Brent Williams:
00:12:34
interfaces, so providing excellent customer service, that
Brent Williams:
00:12:34
impacts on people business society at large. I'll, I'll
Brent Williams:
00:12:39
make it make it conversational, so you can align with what the
Brent Williams:
00:12:44
customer wants. Personal learning it for in the education
Brent Williams:
00:12:48
sphere, companionship for old people, you know, if you
Brent Williams:
00:12:52
actually put a physical manifestation, like a robot, and
Brent Williams:
00:12:52
kind of dive into two and ask you a question. So you mentioned
Brent Williams:
00:12:57
then you have this AI back back end, you can you can maybe
Brent Williams:
00:13:01
alleviate some loneliness in the world. And then so clearly,
Brent Williams:
00:13:06
there's this tremendous implications. And then you see
Brent Williams:
00:13:10
the applications of AI. Some of the things that we've often
Brent Williams:
00:13:13
wealth management, and you mentioned health care. And as I
Brent Williams:
00:13:15
talked about is in the medical field, in the medical field, you
Brent Williams:
00:13:20
can feed in radiology reports, and it can find abnormalities,
Brent Williams:
00:13:25
that's kind of low hanging fruit. But then when you have
Brent Williams:
00:13:29
this kind of conversational AI, you can put that in front of
Brent Williams:
00:13:32
was sitting here listening, I was thinking, oh, those are two
Brent Williams:
00:13:34
medical knowledge and you have a doc in a box. And when you are
Brent Williams:
00:13:39
considered the shortage of primary care physicians, this
Brent Williams:
00:13:43
would be a very interesting way to layer our medical
Brent Williams:
00:13:48
interactions, where you have the first layer of interaction where
Brent Williams:
00:13:51
pretty personal topics to me, like, one, you know, how am I
Brent Williams:
00:13:53
you can feel comfortable talking, even though it's not a
Brent Williams:
00:13:57
person, you can talk talk to the box, and that can do a diagnosis
Brent Williams:
00:14:02
and write up a report. And then it can go to the next level for
Brent Williams:
00:14:07
verification with a human and then go to the next level for a
Brent Williams:
00:14:11
managing my money and two managing my health? And I know
Brent Williams:
00:14:12
referral to a specialization. It can change the medical field
Brent Williams:
00:14:17
quite quite significantly. And we've seen applications of AI in
Brent Williams:
00:14:22
supply chain and manufacturing with robotics, in the financial
Brent Williams:
00:14:27
sector with wealth planning. And so when you have these
Brent Williams:
00:14:29
that I think that a lot of your research in digitization and how
Brent Williams:
00:14:31
interfaces, you basically can interact and get it more aligned
Brent Williams:
00:14:36
with what you want in terms of your wealth goals. And, and
Brent Williams:
00:14:41
recently, I was reading an article that described IBM's
Brent Williams:
00:14:45
Watson that was working with a fashion house, and they actually
Brent Williams:
00:14:50
trained the AI on videos of models walking down runways, and
Brent Williams:
00:14:50
it affects people in organizations and society looked
Brent Williams:
00:14:55
social media included, and the AI was quite adept at
Brent Williams:
00:14:59
identifying fashion trends, and new fashion clothing designs,
Brent Williams:
00:15:04
based on what they observed through this corpus of data. So
Brent Williams:
00:15:07
at least at trust at some level. So what have you learned about
Brent Williams:
00:15:08
it's remarkable at the diversity of areas that AI can have have
Brent Williams:
00:15:13
an impact, which is why it's so profound. And in some sense,
Brent Williams:
00:15:18
also a little scary.
Brent Williams:
00:15:28
the way that trust, I don't know, I don't know if it
Brent Williams:
00:15:45
mediates those two. Tell me what you've learned and how you think
Brent Williams:
00:16:06
that that's going to play out with AI?
Brent Williams:
00:16:19
Yeah, this is a really interesting question of trust.
Brent Williams:
00:16:23
Because historically, when we looked at information systems,
Brent Williams:
00:16:27
we programmed these systems, we could actually code them, we
Brent Williams:
00:16:30
said, so if we had a, we had to make a computer play chess, we'd
Brent Williams:
00:16:34
say, okay, this is the way the queen moves. This is the way the
Brent Williams:
00:16:36
bishop moves. These are the rules of chess, we put it into a
Brent Williams:
00:16:39
program, and then it will play chess. So the earlier AI systems
Brent Williams:
00:16:45
were actually programmed systems, they will rule based
Brent Williams:
00:16:49
systems. So there was AI in the 1970s. But it was usually
Brent Williams:
00:16:53
referred to as a snake oil of the 1970s and 80s. Because it
Brent Williams:
00:16:57
over promised and under delivered, you could only have
Brent Williams:
00:17:00
some gaming applications. So yes, you had medical diagnostic
Brent Williams:
00:17:03
systems, but they were based on rules, the rules were coded in.
Brent Williams:
00:17:07
So if a patient had temperature over 98.6, the patient had
Brent Williams:
00:17:11
fever, if their white blood cell was over a certain level they
Brent Williams:
00:17:15
had an infection, and it would, you'd kind of give it the
Brent Williams:
00:17:17
symptoms, and it will pull out the rules and come up with the
Brent Williams:
00:17:20
diagnosis. What's changed now is we have had the perfect storm of
Brent Williams:
00:17:28
processing power. With these GPUs, graphic processing units,
Brent Williams:
00:17:33
Navidia is the big player there, we have massive amounts of data,
Brent Williams:
00:17:39
and low cost of data storage. And we have with this process of
Brent Williams:
00:17:46
processing power and data, we can kind of look for patterns in
Brent Williams:
00:17:51
the data that are and, and identify these patterns and make
Brent Williams:
00:17:57
predictions based on these patterns. So the large language
Brent Williams:
00:18:00
models that we see of today, are really next word predictors, if
Brent Williams:
00:18:05
they although now, we can actually feed them videos and
Brent Williams:
00:18:09
images. So they're multimodal models. But the language models
Brent Williams:
00:18:14
are, really next word predictor, so they don't have any raw
Brent Williams:
00:18:19
intelligence of their own. But they're kind of looking for
Brent Williams:
00:18:22
these patterns in the data. And so while the old systems were
Brent Williams:
00:18:26
programmed, the newer systems are digging out these patterns,
Brent Williams:
00:18:31
almost like your neurological connections in the brain. It's
Brent Williams:
00:18:36
actually creating these connections. But we don't really
Brent Williams:
00:18:39
understand how they're creating these connections. It's almost
Brent Williams:
00:18:43
like asking a person, say, How did you come up with that
Brent Williams:
00:18:46
decision? Yeah, and what what neurons were connected in your
Brent Williams:
00:18:50
brain to come up with that decision. So we don't really
Brent Williams:
00:18:53
understand it. And so if we don't fully understand it, it's
Brent Williams:
00:18:58
like Warren Buffett says, Never invest in anything you don't
Brent Williams:
00:19:01
understand. So here, we are relying on the outputs. And we
Brent Williams:
00:19:06
don't really understand how the outputs came to into existence.
Brent Williams:
00:19:10
It's massive amounts of data, millions of parameters in the
Brent Williams:
00:19:14
model. And so without understanding it, why should we
Brent Williams:
00:19:18
trust it? And if we can't trust it, then why should we use it?
Brent Williams:
00:19:23
And so I think it's a fundamental question of
Brent Williams:
00:19:26
technology and value, which is my major stream, that we need to
Brent Williams:
00:19:30
be able to trust these systems. Because another way to think
Brent Williams:
00:19:35
about this is typically, if you look at scientific research, the
Brent Williams:
00:19:45
idea of actually looking at patterns from data was
Brent Williams:
00:19:49
considered voodoo science for many years.
Brent Williams:
00:19:51
Yeah
Varun Grover:
00:19:51
This induction you don't look for patterns.
Brent Williams:
00:19:54
Yeah
Varun Grover:
00:19:54
You come up with a theory or a statement and then
Varun Grover:
00:19:57
you go on to get data and test it.
Brent Williams:
00:19:59
Yeah, that's right.
Varun Grover:
00:20:00
But what are large language models doing? What is
Varun Grover:
00:20:00
Yeah. So it's just thinking exactly the same question.
Varun Grover:
00:20:02
AI doing? It is really creating and testing theory
Varun Grover:
00:20:06
simultaneously without any scientific underlay. So when I
Varun Grover:
00:20:12
think about this, I think, you know, I've always grown up
Varun Grover:
00:20:15
through my information systems career, thinking about data,
Varun Grover:
00:20:21
information and knowledge as a trichotomy. So you have data,
Varun Grover:
00:20:25
which are raw facts. And then you put the data together, and
Varun Grover:
00:20:29
you get information which informs you. So you get
Varun Grover:
00:20:32
interesting reports that reduce uncertainty or informs you. And
Varun Grover:
00:20:36
then when you apply it, and you use organize the information,
Varun Grover:
00:20:41
with experience, you get knowledge. What we've done with
Varun Grover:
00:20:45
AI, is we've removed the middle layer, we've gone data is
Varun Grover:
00:20:50
knowledge. And so we're basically taking massive amounts
Varun Grover:
00:20:54
of data about the world. And we are proclaiming that this is
Varun Grover:
00:20:58
actually creating knowledge. So we don't really need to
Varun Grover:
00:21:03
understand the world to know it. It's like we're just take, so
Varun Grover:
00:21:07
that layer has gone. So that's why this idea of trust is so
Varun Grover:
00:21:10
fundamental, because we don't really understand how these
Varun Grover:
00:21:15
connections are made. And so I I was thinking about, you know, as
Varun Grover:
00:21:20
you why do people trust other people?
Varun Grover:
00:21:27
And so so and do the others elements actually translate to
Brent Williams:
00:21:30
Well, that's what I was gonna ask you, you
Brent Williams:
00:21:31
AI. And I think people trust other people, because they may
Brent Williams:
00:21:36
have a connection with them, they actually relate to them,
Brent Williams:
00:21:40
they have empathy, or they relate at some subliminal level.
Brent Williams:
00:21:41
know, how do you see that changing and evolving? The one
Brent Williams:
00:21:45
So they say I trust you. People may trust people, because they
Brent Williams:
00:21:49
understand where they're coming from. So if they make a
Brent Williams:
00:21:53
decision, if you talk to someone, they make a decision,
Brent Williams:
00:21:55
that I could see that you just mentioned. Well, the model is
Brent Williams:
00:21:58
you can ask them. Okay, what's the basis of this decision? And
Brent Williams:
00:22:02
do I trust you to explain to me how you came up with this
Brent Williams:
00:22:07
decision. People trust people, because they are predictable,
Brent Williams:
00:22:10
getting more reliable, let's say less erratic. That one may
Brent Williams:
00:22:11
they are reliable, they may give accurate outputs. So if someone
Brent Williams:
00:22:16
is erratic, we probably won't trust them. But if someone is
Brent Williams:
00:22:21
more stable and predictable, and comes up with good, good
Brent Williams:
00:22:25
change, you know, but and you even mentioned the ability of,
Brent Williams:
00:22:25
outputs, we tend to trust them. And people trust people because
Brent Williams:
00:22:30
we might share the same social experiences. So I think these
Brent Williams:
00:22:34
elements of we trust people, because we can connect with them
Brent Williams:
00:22:39
or relate to them. We trust people because we can understand
Brent Williams:
00:22:40
of maybe I don't think you said it this way. I think you were
Brent Williams:
00:22:43
where they're coming from, we can ask them, we trust people
Brent Williams:
00:22:48
because they're reliable and not erratic. And we trust people,
Brent Williams:
00:22:53
because we have a shared social world, is generally why people
Brent Williams:
00:22:55
mentioning Pie, the the product you were working with, that it
Brent Williams:
00:22:57
trust people. These elements are not very prominent in AI. And
Brent Williams:
00:23:02
so, so I think that's why trust is a kind of an important issue.
Brent Williams:
00:23:07
Although they're changing.
Brent Williams:
00:23:11
had a feel of empathy, which might mean you could emotionally
Brent Williams:
00:23:26
connect to it more easily. So some of these things seem to be evolving,
Varun Grover:
00:23:44
I think so I think that from whatever little
Varun Grover:
00:23:49
observation I've had over the last year and a half, it's not
Varun Grover:
00:23:52
been that long since ChatGPT was removed, released. But this
Varun Grover:
00:23:57
idea, I can see improvement in empathy, I think that element is
Varun Grover:
00:24:02
there. So we are starting in on that trajectory in terms of
Varun Grover:
00:24:07
understandability, there is a whole field or subfield of AI.
Varun Grover:
00:24:13
That is called Explainable AI. How do you get take this neural
Varun Grover:
00:24:18
connections? And how do you actually convert it into a
Varun Grover:
00:24:21
language that humans can understand so they can explain
Varun Grover:
00:24:24
the rationale for their decisions? Now, it may not be
Varun Grover:
00:24:28
that important for radiology reports. But if we're dealing
Varun Grover:
00:24:33
with a business discipline, you know, often you want to
Varun Grover:
00:24:36
understand the rationale for the decision. So Explainable AI is
Varun Grover:
00:24:41
actually being worked on. And and it's important, and it's
Varun Grover:
00:24:45
it's particularly important because AI is all about data and
Varun Grover:
00:24:50
connections and patterns. AI is not particularly good at
Varun Grover:
00:24:56
reasoning. So if we tell an AI that Look, if they're looking at
Varun Grover:
00:25:02
a pattern, x is the parent of y, it will not naturally infer from
Varun Grover:
00:25:07
there or deduce from there, that y is the child of x. That
Varun Grover:
00:25:13
reasoning ability will only come about if it's in the corpus of
Varun Grover:
00:25:17
data. So the question then becomes, you know, how do you
Varun Grover:
00:25:22
get this reasoning out of the AI, and that's where a lot of
Varun Grover:
00:25:26
the research on Explainable AI is, give you another example. We
Varun Grover:
00:25:31
can put millions of chess games played by world chess champions
Varun Grover:
00:25:35
and feed it into the AI and it'll figure out the rules of
Varun Grover:
00:25:38
chess and what works and what doesn't work. So that it will
Varun Grover:
00:25:41
basically be a really good chess player. But if we change the
Varun Grover:
00:25:46
board from an eight by eight board to a nine by nine board,
Varun Grover:
00:25:50
then the training doesn't really help because it needs more than
Varun Grover:
00:25:55
that it needs some kind of reasoning ability to play chess
Varun Grover:
00:25:59
well on a nine by nine input. So I think that that idea of how do
Varun Grover:
00:26:05
you get that reasoning to come into you know, the box or the
Varun Grover:
00:26:12
corpus of intelligence of AI is is something that is being
Varun Grover:
00:26:17
worked on through Explainable AI. So empathy? Yes, we're
Varun Grover:
00:26:21
seeing improvement there. Explainable AI, we're not quite
Varun Grover:
00:26:24
there. But there's certainly a direction there. And like you
Varun Grover:
00:26:28
said, absolutely the corpus of training data. So most of the
Varun Grover:
00:26:33
models, we had an AI foundational models, like from
Varun Grover:
00:26:37
open AI. And from Google and Microsoft, these models are
Varun Grover:
00:26:43
trained on massive corpus of data from the internet, the
Varun Grover:
00:26:47
internet data is noisy. And it is, you know, it has a lot of a
Varun Grover:
00:26:53
lot of problems, which means it can be a pretty good general, it
Varun Grover:
00:26:59
can provide general advice on a lot of things. But if you're
Varun Grover:
00:27:03
looking for very specialized advice, sometimes these models
Varun Grover:
00:27:07
hallucinate and create these arbitrary outcomes, because
Varun Grover:
00:27:13
they're not fully trained in those specialized areas. So I
Varun Grover:
00:27:17
think one of the trends is how do you actually curate data for
Varun Grover:
00:27:23
training and make these models more accurate and better? And
Varun Grover:
00:27:30
how do you move away from the big tech companies which have
Varun Grover:
00:27:33
the resources to train these large foundational models, to
Varun Grover:
00:27:37
smaller models that are done through open source where you
Varun Grover:
00:27:41
can, individual companies may not have to invest millions of
Varun Grover:
00:27:46
dollars in training these models, but they can train them
Varun Grover:
00:27:48
on their own data,
Brent Williams:
00:27:49
Right.
Varun Grover:
00:27:50
And, and become, it can become more more
Varun Grover:
00:27:52
personalized and more accurate within that domain. So I think
Varun Grover:
00:27:57
that's the third component of trust, where these models are
Varun Grover:
00:28:02
being trained on better data. And sometimes even instead of
Varun Grover:
00:28:06
the large models, we're getting smaller and more specialized
Varun Grover:
00:28:10
models that are more accurate. And I think that's a trajectory
Varun Grover:
00:28:14
that will improve trust. And finally, in terms of context. So
Varun Grover:
00:28:22
I think that context is also improving, because the earlier
Varun Grover:
00:28:26
models had very little that you could put into the model in
Varun Grover:
00:28:30
terms of input. Now, some of the newer models, you can put entire
Varun Grover:
00:28:35
books and and say, you know, so your context window, the number
Varun Grover:
00:28:40
of tokens that go into the context is becoming larger and
Varun Grover:
00:28:43
larger, which means you can provide a richer context to your
Varun Grover:
00:28:46
decision making. And then you can have conversations to get
Varun Grover:
00:28:52
the alignment of get what you want from the AI. And so that
Varun Grover:
00:28:57
alignment can continue by probing and interacting with the
Varun Grover:
00:29:02
AI. So I think on the third aspect, the context aspect,
Varun Grover:
00:29:07
we're also seeing improvements. So I will trust the box, because
Varun Grover:
00:29:13
I can connect with it. I can trust the box, because I know
Varun Grover:
00:29:18
where it's coming from. It can explain its reasoning to me, I
Varun Grover:
00:29:22
can trust the box. Because it gives me good outputs, accurate
Varun Grover:
00:29:27
outputs, and I can trust the box because it's grounded in my
Varun Grover:
00:29:30
world. And so all those aspects are advancing. And so I think
Varun Grover:
00:29:37
that on the technological front, we are seeing a trajectory
Varun Grover:
00:29:43
that's positive for trust.
Brent Williams:
00:29:47
I'm gonna I'm gonna ask you about to human.
Brent Williams:
00:29:51
Maybe let's start in the business context, right. So
Brent Williams:
00:29:54
let's say the models are improving, they're becoming more
Brent Williams:
00:29:57
trustworthy. They're certainly becoming more commonplace.
Brent Williams:
00:30:02
They're still probably some of the like, there's the
Brent Williams:
00:30:05
excitement. There's some of those some people who think it's
Brent Williams:
00:30:07
scary. Probably each of us land somewhere on that continuum
Brent Williams:
00:30:11
differently. But in the business world, you know, the role of the
Brent Williams:
00:30:16
human in business, does it, change it slightly? Does it
Brent Williams:
00:30:21
change? Fundamentally? You have any thoughts to share on that?
Varun Grover:
00:30:28
Yeah I think that's, that's, that's a million
Varun Grover:
00:30:32
dollar question, or maybe a billion dollar question. or now
Varun Grover:
00:30:35
given, given the market of AI, maybe a trillion question.
Brent Williams:
00:30:39
If you can answer this, we can raise a lot
Brent Williams:
00:30:40
of money for the Walton College.
Varun Grover:
00:30:43
But I think I so, how should AI, so if if we are
Varun Grover:
00:30:48
using AI, to automate jobs, if we're using AI to, to automate
Varun Grover:
00:30:56
and replace jobs, so you can, you know, you can do an
Varun Grover:
00:30:59
accounting job better, or you can do a legal job better. Then
Varun Grover:
00:31:04
what is essentially happening is we are devaluing labor. We're
Varun Grover:
00:31:11
devaluing human skills and labor, because we're essentially
Varun Grover:
00:31:15
replacing labor with the technology. And that's going to
Varun Grover:
00:31:18
bring the cost of labor down. So substitution of jobs with
Varun Grover:
00:31:23
technology is probably not the most innovative direction, it's
Varun Grover:
00:31:28
probably appropriate. And probably the incentives for
Varun Grover:
00:31:31
corporations to make money are in the cost cutting sphere. So
Varun Grover:
00:31:35
we're certainly going to see a lot of that. But I think that
Varun Grover:
00:31:39
where we're going to see the human and, and the technology
Varun Grover:
00:31:43
work better is through augmentation. You never have
Varun Grover:
00:31:48
innovation through automation, you will only have innovation,
Varun Grover:
00:31:53
if there are ways in which the AI and the technology can work
Varun Grover:
00:31:57
together. So if you automate a lot, you're basically devaluing
Varun Grover:
00:32:03
the human if you augment you're up valuing the human, because
Varun Grover:
00:32:10
when you when you automate, your competition is between the AI
Varun Grover:
00:32:15
and the human, when you augment your competition is between the
Varun Grover:
00:32:21
human with the AI, and the human without the AI.
Brent Williams:
00:32:24
Right.
Varun Grover:
00:32:25
And so you're upskilling. So I think that we
Varun Grover:
00:32:29
are at a point of inflection, where we are making 1000s of
Varun Grover:
00:32:35
decisions regarding AI. And it's really important that if
Varun Grover:
00:32:39
corporations are going to invest in AI, they should consciously
Varun Grover:
00:32:44
think about how to augment human skills through AI, rather than
Varun Grover:
00:32:50
only automation. Because automation will probably give
Varun Grover:
00:32:55
you the short term bang for the buck in terms of reduction in
Varun Grover:
00:33:00
costs. But the longer term impact on innovation will only
Varun Grover:
00:33:05
take place if you give people the discretion to leverage AI
Varun Grover:
00:33:08
and become and use it innovatively. So if there's
Varun Grover:
00:33:12
there's a series of tasks in your job, and there's a few that
Varun Grover:
00:33:16
can be automated, that's fine. But then think about what are
Varun Grover:
00:33:20
the tasks in your jobs that can be augmented. That's where the
Varun Grover:
00:33:23
innovation comes in. And that's where the value is created.
Brent Williams:
00:33:27
Well, Varun, thank you. Thank you for sharing
Brent Williams:
00:33:31
your knowledge and that knowledge that comes from
Brent Williams:
00:33:35
decades of work and research and I know I learned a lot and I'm
Brent Williams:
00:33:41
sure our audience did as well. So thank you for joining me
Brent Williams:
00:33:44
today.
Varun Grover:
00:33:44
Thank you, Brent for having me.
Brent Williams:
00:33:45
On behalf of the Walton College thank you for
Brent Williams:
00:33:48
joining us for this captivating conversation. To stay connected
Brent Williams:
00:33:52
and never miss an episode, simply search for Be Epic on
Brent Williams:
00:33:55
your preferred podcast service.
00:00:00
You never have innovation through automation,
Varun Grover:
00:00:04
you will only have innovation if there are ways in which the AI
Varun Grover:
00:00:09
and the technology can work together.
Brent Williams:
00:00:12
Welcome to the Be Epic Podcast brought to you
Brent Williams:
00:00:14
by the Sam M. Walton College of Business at the University of
Brent Williams:
00:00:17
Arkansas. I'm your host, Brent Williams. Together, we'll
Brent Williams:
00:00:22
explore the dynamic landscape of business and uncover the
Brent Williams:
00:00:25
strategies, insights and stories that drive business today. Well,
Brent Williams:
00:00:31
today, I'm lucky to have with me Dr. Varun Grover, and Dr.
Brent Williams:
00:00:35
Grover's the George and Boyce Billingsley Endowed Chair and
Brent Williams:
00:00:39
Distinguished Professor of Information Systems in the
Brent Williams:
00:00:42
Walton College. Varun, thank you for joining me today.
Varun Grover:
00:00:46
I'm delighted to be here. Thank you.
Brent Williams:
00:00:48
Well, I think we're going to touch on some
Brent Williams:
00:00:50
interesting topics today. But maybe before we do that, I would
Brent Williams:
00:00:55
love for our audience to get to know you better. I've gotten to
Brent Williams:
00:00:58
know you over the last five years, I think you've been at
Brent Williams:
00:01:00
the Walton College since 2017. But tell us a little bit about
Brent Williams:
00:01:04
you your background, and before coming to the Walton College.
Varun Grover:
00:01:09
Sure. So. So I'm a techie at heart in the sense
Varun Grover:
00:01:13
that I did my undergraduate degree in engineering,
Varun Grover:
00:01:17
electrical engineering, and, and that's where my interest in
Varun Grover:
00:01:23
technology started coming about. And I had the good fortune of
Varun Grover:
00:01:27
getting into a very competitive engineering school that allowed
Varun Grover:
00:01:31
me to think systematically about technology. So when I, when I
Varun Grover:
00:01:36
graduated, I kind of felt that I was in a very narrow silo, I
Varun Grover:
00:01:42
understood technology gyrations I understood, you know,
Varun Grover:
00:01:47
technical concepts, but I didn't really understand the world, the
Varun Grover:
00:01:50
broader context. And so I decided to do an MBA and then go
Varun Grover:
00:01:55
into my PhD in information systems. And then after that,
Varun Grover:
00:02:00
I've been basically in academic jobs, I was at USC, and then I
Varun Grover:
00:02:05
was at Clemson. And my primary goal or responsibility in those
Varun Grover:
00:02:10
jobs was to, to build the PhD program, and conduct research
Varun Grover:
00:02:16
and set up centers, and have some outreach, dealing with
Varun Grover:
00:02:20
important problems in the Information Systems domain. So
Varun Grover:
00:02:24
I've been integrally involved in this in this field, and the
Varun Grover:
00:02:29
field has evolved, because the technology catalyst is kind of
Varun Grover:
00:02:32
driving changes. So if you think of technology in the 1960s, and
Varun Grover:
00:02:37
70s, and 80s 90s, and now, now we're in this digital world,
Varun Grover:
00:02:42
that is a lot of people really don't know how to deal with it,
Varun Grover:
00:02:46
and companies are still struggling. And now we've got AI
Varun Grover:
00:02:50
coming in, that's adding a new wrinkle. And so what I like
Varun Grover:
00:02:55
about the field in general, is that we don't have a dearth of
Varun Grover:
00:03:01
problems to study, there are lots of challenges faced by by
Varun Grover:
00:03:06
companies faced by individuals, and now the impacts transcend to
Varun Grover:
00:03:11
society. So there's a lot of really, really interesting
Varun Grover:
00:03:15
issues. And that keeps me excited about the field.
Brent Williams:
00:03:18
Yeah. Well, you know, Varun, you're, you are a
Brent Williams:
00:03:22
researcher, you're a teacher, you know, you serve the
Brent Williams:
00:03:26
University and the College through the normal mechanisms.
Brent Williams:
00:03:30
But you have spent the majority of your career really focused on
Brent Williams:
00:03:33
research. And as you say, the your work and PhD programs are
Brent Williams:
00:03:37
really advancing the research of these institutions. Just when
Brent Williams:
00:03:41
you think about business research, maybe more generally,
Brent Williams:
00:03:44
how, how does it impact the world? How does it impact
Brent Williams:
00:03:49
others? And how have you viewed that over time?
Varun Grover:
00:03:53
Yeah, that's a really interesting question.
Varun Grover:
00:03:56
Because even business research has has evolved. If you think
Varun Grover:
00:04:00
about in the 1950s, business research was not much of
Varun Grover:
00:04:04
anything, it was more, you know, go out maybe some outreach and
Varun Grover:
00:04:08
case studies that you did with with businesses, but then the
Varun Grover:
00:04:13
scientific aspect of business research came in the 1960s,
Varun Grover:
00:04:17
1970s. And we kind of went overboard with the concepts,
Varun Grover:
00:04:22
scientific concepts of theory and became somewhat
Varun Grover:
00:04:24
disassociated with practice. So my thinking is that in in a, in
Varun Grover:
00:04:30
an applied business discipline, it's really important that we
Varun Grover:
00:04:35
create interventions to practice. Any research that we
Varun Grover:
00:04:39
do, should not be esoteric and isolated from the real world.
Varun Grover:
00:04:47
And so when we talk about when we think about research
Varun Grover:
00:04:50
projects, traditionally our approach would be, you know,
Varun Grover:
00:04:53
look for research gaps in the literature and then try and fill
Varun Grover:
00:04:56
those gaps. I think the more productive approach is to To
Varun Grover:
00:05:00
identify real problems faced by practice, and then to convert
Varun Grover:
00:05:05
those practical problems into research problems. So it's more
Varun Grover:
00:05:10
like engaged scholarship, you engage with practice at a deeper
Varun Grover:
00:05:14
level, and try and look at their problems, and then translate
Varun Grover:
00:05:20
those problems into research problems so that you can create
Varun Grover:
00:05:23
interventions, as a result of your research that actually make
Varun Grover:
00:05:26
a difference in practice.
Brent Williams:
00:05:27
That's right, what a good way to explain that
Brent Williams:
00:05:29
I just couldn't agree more about at least in my own research that
Brent Williams:
00:05:33
that's where the research questions that I've been most
Brent Williams:
00:05:37
interested in then that ultimately, I think the research
Brent Williams:
00:05:39
that has the biggest impact for my own career comes from same
Brent Williams:
00:05:44
for you.
Varun Grover:
00:05:44
Absolutely
Brent Williams:
00:05:45
Well, I'll, I'll brag on you a minute, because I
Brent Williams:
00:05:48
don't think you would brag on yourself in this way. A couple
Brent Williams:
00:05:52
of the interesting statistics about you as a researcher, I
Brent Williams:
00:05:57
think you've published more than 400 articles in your career,
Brent Williams:
00:06:02
over 100 in the top journals, and you are listed as one of the
Brent Williams:
00:06:08
100 most cited, not just information systems, but
Brent Williams:
00:06:12
business researchers in the world, what an accomplishment.
Brent Williams:
00:06:16
And so you have truly over a career made a real impact in the
Brent Williams:
00:06:23
academy and in practice.
Varun Grover:
00:06:24
Thank you. Thank you Brent.
Brent Williams:
00:06:26
Well your research, yeah, I told you this
Brent Williams:
00:06:30
before, I'm kind of borrowing from your bio here. But you
Brent Williams:
00:06:33
know, at a real broad level, you know, as I saw it described
Brent Williams:
00:06:37
focuses on the effects of digitization on individuals on
Brent Williams:
00:06:42
businesses and organizations and even society, tell us a little
Brent Williams:
00:06:46
more of a high level before we get into some of the more
Brent Williams:
00:06:49
detailed questions.
Varun Grover:
00:06:50
So I've always been interested in how
Varun Grover:
00:06:50
technology creates value. And information technology is
Varun Grover:
00:06:52
particularly of interest to me. So when we look at computing
Varun Grover:
00:06:53
technologies, in a business context, what are the aspects of
Varun Grover:
00:06:56
these technologies? And the and the context around them, that
Varun Grover:
00:07:11
lead to value value at the individual level? So how can
Varun Grover:
00:07:15
technology enhance individual productivity? How the value at
Varun Grover:
00:07:20
the group level, value at the organizational level? How do you
Varun Grover:
00:07:23
actually create competitive advantage through these
Varun Grover:
00:07:25
technologies? How do you create greater profitability, more
Varun Grover:
00:07:32
revenues? And then at the market level? And that is how do you
Varun Grover:
00:07:37
make markets more efficient through technology. And then now
Varun Grover:
00:07:42
even more so at the societal level. So with technology, and
Varun Grover:
00:07:48
particularly digital technologies, penetrating every
Varun Grover:
00:07:51
aspect of what we do, there are issues of value, and the value
Varun Grover:
00:07:58
could be positive or negative. So now we are seeing you know,
Varun Grover:
00:08:01
this, there's two phases of technology, you have the
Varun Grover:
00:08:05
intended consequences to create positive value for companies.
Varun Grover:
00:08:09
And then you have sometimes unintended consequences that
Varun Grover:
00:08:14
create problems for society. So I like to study at every one of
Varun Grover:
00:08:20
these levels, I like to study different questions regarding
Varun Grover:
00:08:24
the value proposition of digital technologies.
Brent Williams:
00:08:27
Well, this has to bring probably to anyone's
Brent Williams:
00:08:29
mind, you know, when you talk about the enter the the
Brent Williams:
00:08:33
intersection of society and digital technology, as we sit
Brent Williams:
00:08:37
here today, the conversation around artificial intelligence,
Brent Williams:
00:08:42
AI, is everywhere. All right, you know when chat GPT popped on
Brent Williams:
00:08:47
the scene, it really just brought it to the consciousness,
Brent Williams:
00:08:51
even though this has been developing for years, if not
Brent Williams:
00:08:54
decades, it really brought it to the consciousness. You're not an
Brent Williams:
00:08:58
AI researcher, I know. But but I do think that you're probably in
Brent Williams:
00:09:02
a unique position to think about this intersection between this
Brent Williams:
00:09:07
technology, and people, businesses and society. So maybe
Brent Williams:
00:09:13
just I'll just I'll stop right there and kind of say, all
Brent Williams:
00:09:16
right, as you're observing this phenomenon of AI, what's kind of
Brent Williams:
00:09:21
going through your mind and what are the observations and
Brent Williams:
00:09:24
questions that that are becoming apparent to you?
Varun Grover:
00:09:28
Yeah. As many people are fascinated by AI, so
Varun Grover:
00:09:32
am I, as it's not, it's not my primary area of research, but
Varun Grover:
00:09:37
clearly it falls within the domain of digital technologies.
Varun Grover:
00:09:41
And like you said, ChatGPT came on the scene and it blew away
Varun Grover:
00:09:46
people because not because it wasn't necessarily the most
Varun Grover:
00:09:49
accurate or more most interesting output, but it gave
Varun Grover:
00:09:55
people the sense of the possible if this is the starting point,
Varun Grover:
00:09:59
and it can generate such interesting outputs, then where
Varun Grover:
00:10:04
is the world going. And I remember the first time I played
Varun Grover:
00:10:08
around with GPT, I asked it to write a story, I said, you know,
Varun Grover:
00:10:13
here's a little girl walking up a hill, and there's a, there's a
Varun Grover:
00:10:15
big rock at the top of the hill, write a story for a five year
Varun Grover:
00:10:19
old. And it wrote a pretty compelling story that I actually
Varun Grover:
00:10:24
read to a five year old. And then I said, okay, redo the
Varun Grover:
00:10:29
story for a 10 year old and it reformulated the story, change
Varun Grover:
00:10:35
the language, and again, fairly compelling story for a 10 year
Varun Grover:
00:10:39
old, then I asked her to do one for an adult, and it did it. And
Varun Grover:
00:10:43
then I pushed it to be more creative, in the ending, make it
Varun Grover:
00:10:48
more interesting, make it more controversial, make it more
Varun Grover:
00:10:52
enigmatic. And every time I did it, it came up with interesting
Varun Grover:
00:10:57
changes. And I said, wow, you know, this is actually doing it
Varun Grover:
00:11:02
from the corpus of data it's been trained on, and yet, it has
Varun Grover:
00:11:07
this layer of creativity that we don't quite understand how it's
Varun Grover:
00:11:11
getting there. And this was the early version of ChatGPT. More
Varun Grover:
00:11:18
recently, I was playing around with now there's many, many of
Varun Grover:
00:11:22
these chatbots, AI based chatbots. And I noticed one of
Varun Grover:
00:11:27
the differences. So I was playing around with a chatbot
Varun Grover:
00:11:30
called Pie, which now I believe has been acquired by Microsoft.
Varun Grover:
00:11:35
And it I found it to be extremely conversational. And
Varun Grover:
00:11:39
what was different about the earlier versions in this version
Varun Grover:
00:11:43
is that it had empathy. It actually could relate to me, and
Varun Grover:
00:11:49
it was taking cues from my text, and putting that emotional
Varun Grover:
00:11:54
response back to me.
Varun Grover:
00:11:56
And that was very interesting. So it's like, it's
Brent Williams:
00:11:56
Interesting.
Brent Williams:
00:11:58
Interesting. Well, you mentioned profound and scary. You also,
Brent Williams:
00:11:59
almost like we're coding emotions into these training
Brent Williams:
00:12:04
sets. Now, to provide far more conversational far more personal
Brent Williams:
00:12:09
AI, personal intelligence reactions. And I stopped
Brent Williams:
00:12:13
thinking about this, it is absolutely fascinating where
Brent Williams:
00:12:17
this can go. Because if you can have that level of
Brent Williams:
00:12:18
you talked about a wide potential set of potential
Brent Williams:
00:12:21
interactivity, and conversational and empathy with
Brent Williams:
00:12:25
with individuals, you can put this front end on pretty much
Brent Williams:
00:12:29
any intelligence, and you can access the world. So customer
Brent Williams:
00:12:34
interfaces, so providing excellent customer service, that
Brent Williams:
00:12:34
impacts on people business society at large. I'll, I'll
Brent Williams:
00:12:39
make it make it conversational, so you can align with what the
Brent Williams:
00:12:44
customer wants. Personal learning it for in the education
Brent Williams:
00:12:48
sphere, companionship for old people, you know, if you
Brent Williams:
00:12:52
actually put a physical manifestation, like a robot, and
Brent Williams:
00:12:52
kind of dive into two and ask you a question. So you mentioned
Brent Williams:
00:12:57
then you have this AI back back end, you can you can maybe
Brent Williams:
00:13:01
alleviate some loneliness in the world. And then so clearly,
Brent Williams:
00:13:06
there's this tremendous implications. And then you see
Brent Williams:
00:13:10
the applications of AI. Some of the things that we've often
Brent Williams:
00:13:13
wealth management, and you mentioned health care. And as I
Brent Williams:
00:13:15
talked about is in the medical field, in the medical field, you
Brent Williams:
00:13:20
can feed in radiology reports, and it can find abnormalities,
Brent Williams:
00:13:25
that's kind of low hanging fruit. But then when you have
Brent Williams:
00:13:29
this kind of conversational AI, you can put that in front of
Brent Williams:
00:13:32
was sitting here listening, I was thinking, oh, those are two
Brent Williams:
00:13:34
medical knowledge and you have a doc in a box. And when you are
Brent Williams:
00:13:39
considered the shortage of primary care physicians, this
Brent Williams:
00:13:43
would be a very interesting way to layer our medical
Brent Williams:
00:13:48
interactions, where you have the first layer of interaction where
Brent Williams:
00:13:51
pretty personal topics to me, like, one, you know, how am I
Brent Williams:
00:13:53
you can feel comfortable talking, even though it's not a
Brent Williams:
00:13:57
person, you can talk talk to the box, and that can do a diagnosis
Brent Williams:
00:14:02
and write up a report. And then it can go to the next level for
Brent Williams:
00:14:07
verification with a human and then go to the next level for a
Brent Williams:
00:14:11
managing my money and two managing my health? And I know
Brent Williams:
00:14:12
referral to a specialization. It can change the medical field
Brent Williams:
00:14:17
quite quite significantly. And we've seen applications of AI in
Brent Williams:
00:14:22
supply chain and manufacturing with robotics, in the financial
Brent Williams:
00:14:27
sector with wealth planning. And so when you have these
Brent Williams:
00:14:29
that I think that a lot of your research in digitization and how
Brent Williams:
00:14:31
interfaces, you basically can interact and get it more aligned
Brent Williams:
00:14:36
with what you want in terms of your wealth goals. And, and
Brent Williams:
00:14:41
recently, I was reading an article that described IBM's
Brent Williams:
00:14:45
Watson that was working with a fashion house, and they actually
Brent Williams:
00:14:50
trained the AI on videos of models walking down runways, and
Brent Williams:
00:14:50
it affects people in organizations and society looked
Brent Williams:
00:14:55
social media included, and the AI was quite adept at
Brent Williams:
00:14:59
identifying fashion trends, and new fashion clothing designs,
Brent Williams:
00:15:04
based on what they observed through this corpus of data. So
Brent Williams:
00:15:07
at least at trust at some level. So what have you learned about
Brent Williams:
00:15:08
it's remarkable at the diversity of areas that AI can have have
Brent Williams:
00:15:13
an impact, which is why it's so profound. And in some sense,
Brent Williams:
00:15:18
also a little scary.
Brent Williams:
00:15:28
the way that trust, I don't know, I don't know if it
Brent Williams:
00:15:45
mediates those two. Tell me what you've learned and how you think
Brent Williams:
00:16:06
that that's going to play out with AI?
Brent Williams:
00:16:19
Yeah, this is a really interesting question of trust.
Brent Williams:
00:16:23
Because historically, when we looked at information systems,
Brent Williams:
00:16:27
we programmed these systems, we could actually code them, we
Brent Williams:
00:16:30
said, so if we had a, we had to make a computer play chess, we'd
Brent Williams:
00:16:34
say, okay, this is the way the queen moves. This is the way the
Brent Williams:
00:16:36
bishop moves. These are the rules of chess, we put it into a
Brent Williams:
00:16:39
program, and then it will play chess. So the earlier AI systems
Brent Williams:
00:16:45
were actually programmed systems, they will rule based
Brent Williams:
00:16:49
systems. So there was AI in the 1970s. But it was usually
Brent Williams:
00:16:53
referred to as a snake oil of the 1970s and 80s. Because it
Brent Williams:
00:16:57
over promised and under delivered, you could only have
Brent Williams:
00:17:00
some gaming applications. So yes, you had medical diagnostic
Brent Williams:
00:17:03
systems, but they were based on rules, the rules were coded in.
Brent Williams:
00:17:07
So if a patient had temperature over 98.6, the patient had
Brent Williams:
00:17:11
fever, if their white blood cell was over a certain level they
Brent Williams:
00:17:15
had an infection, and it would, you'd kind of give it the
Brent Williams:
00:17:17
symptoms, and it will pull out the rules and come up with the
Brent Williams:
00:17:20
diagnosis. What's changed now is we have had the perfect storm of
Brent Williams:
00:17:28
processing power. With these GPUs, graphic processing units,
Brent Williams:
00:17:33
Navidia is the big player there, we have massive amounts of data,
Brent Williams:
00:17:39
and low cost of data storage. And we have with this process of
Brent Williams:
00:17:46
processing power and data, we can kind of look for patterns in
Brent Williams:
00:17:51
the data that are and, and identify these patterns and make
Brent Williams:
00:17:57
predictions based on these patterns. So the large language
Brent Williams:
00:18:00
models that we see of today, are really next word predictors, if
Brent Williams:
00:18:05
they although now, we can actually feed them videos and
Brent Williams:
00:18:09
images. So they're multimodal models. But the language models
Brent Williams:
00:18:14
are, really next word predictor, so they don't have any raw
Brent Williams:
00:18:19
intelligence of their own. But they're kind of looking for
Brent Williams:
00:18:22
these patterns in the data. And so while the old systems were
Brent Williams:
00:18:26
programmed, the newer systems are digging out these patterns,
Brent Williams:
00:18:31
almost like your neurological connections in the brain. It's
Brent Williams:
00:18:36
actually creating these connections. But we don't really
Brent Williams:
00:18:39
understand how they're creating these connections. It's almost
Brent Williams:
00:18:43
like asking a person, say, How did you come up with that
Brent Williams:
00:18:46
decision? Yeah, and what what neurons were connected in your
Brent Williams:
00:18:50
brain to come up with that decision. So we don't really
Brent Williams:
00:18:53
understand it. And so if we don't fully understand it, it's
Brent Williams:
00:18:58
like Warren Buffett says, Never invest in anything you don't
Brent Williams:
00:19:01
understand. So here, we are relying on the outputs. And we
Brent Williams:
00:19:06
don't really understand how the outputs came to into existence.
Brent Williams:
00:19:10
It's massive amounts of data, millions of parameters in the
Brent Williams:
00:19:14
model. And so without understanding it, why should we
Brent Williams:
00:19:18
trust it? And if we can't trust it, then why should we use it?
Brent Williams:
00:19:23
And so I think it's a fundamental question of
Brent Williams:
00:19:26
technology and value, which is my major stream, that we need to
Brent Williams:
00:19:30
be able to trust these systems. Because another way to think
Brent Williams:
00:19:35
about this is typically, if you look at scientific research, the
Brent Williams:
00:19:45
idea of actually looking at patterns from data was
Brent Williams:
00:19:49
considered voodoo science for many years.
Brent Williams:
00:19:51
Yeah
Varun Grover:
00:19:51
This induction you don't look for patterns.
Brent Williams:
00:19:54
Yeah
Varun Grover:
00:19:54
You come up with a theory or a statement and then
Varun Grover:
00:19:57
you go on to get data and test it.
Brent Williams:
00:19:59
Yeah, that's right.
Varun Grover:
00:20:00
But what are large language models doing? What is
Varun Grover:
00:20:00
Yeah. So it's just thinking exactly the same question.
Varun Grover:
00:20:02
AI doing? It is really creating and testing theory
Varun Grover:
00:20:06
simultaneously without any scientific underlay. So when I
Varun Grover:
00:20:12
think about this, I think, you know, I've always grown up
Varun Grover:
00:20:15
through my information systems career, thinking about data,
Varun Grover:
00:20:21
information and knowledge as a trichotomy. So you have data,
Varun Grover:
00:20:25
which are raw facts. And then you put the data together, and
Varun Grover:
00:20:29
you get information which informs you. So you get
Varun Grover:
00:20:32
interesting reports that reduce uncertainty or informs you. And
Varun Grover:
00:20:36
then when you apply it, and you use organize the information,
Varun Grover:
00:20:41
with experience, you get knowledge. What we've done with
Varun Grover:
00:20:45
AI, is we've removed the middle layer, we've gone data is
Varun Grover:
00:20:50
knowledge. And so we're basically taking massive amounts
Varun Grover:
00:20:54
of data about the world. And we are proclaiming that this is
Varun Grover:
00:20:58
actually creating knowledge. So we don't really need to
Varun Grover:
00:21:03
understand the world to know it. It's like we're just take, so
Varun Grover:
00:21:07
that layer has gone. So that's why this idea of trust is so
Varun Grover:
00:21:10
fundamental, because we don't really understand how these
Varun Grover:
00:21:15
connections are made. And so I I was thinking about, you know, as
Varun Grover:
00:21:20
you why do people trust other people?
Varun Grover:
00:21:27
And so so and do the others elements actually translate to
Brent Williams:
00:21:30
Well, that's what I was gonna ask you, you
Brent Williams:
00:21:31
AI. And I think people trust other people, because they may
Brent Williams:
00:21:36
have a connection with them, they actually relate to them,
Brent Williams:
00:21:40
they have empathy, or they relate at some subliminal level.
Brent Williams:
00:21:41
know, how do you see that changing and evolving? The one
Brent Williams:
00:21:45
So they say I trust you. People may trust people, because they
Brent Williams:
00:21:49
understand where they're coming from. So if they make a
Brent Williams:
00:21:53
decision, if you talk to someone, they make a decision,
Brent Williams:
00:21:55
that I could see that you just mentioned. Well, the model is
Brent Williams:
00:21:58
you can ask them. Okay, what's the basis of this decision? And
Brent Williams:
00:22:02
do I trust you to explain to me how you came up with this
Brent Williams:
00:22:07
decision. People trust people, because they are predictable,
Brent Williams:
00:22:10
getting more reliable, let's say less erratic. That one may
Brent Williams:
00:22:11
they are reliable, they may give accurate outputs. So if someone
Brent Williams:
00:22:16
is erratic, we probably won't trust them. But if someone is
Brent Williams:
00:22:21
more stable and predictable, and comes up with good, good
Brent Williams:
00:22:25
change, you know, but and you even mentioned the ability of,
Brent Williams:
00:22:25
outputs, we tend to trust them. And people trust people because
Brent Williams:
00:22:30
we might share the same social experiences. So I think these
Brent Williams:
00:22:34
elements of we trust people, because we can connect with them
Brent Williams:
00:22:39
or relate to them. We trust people because we can understand
Brent Williams:
00:22:40
of maybe I don't think you said it this way. I think you were
Brent Williams:
00:22:43
where they're coming from, we can ask them, we trust people
Brent Williams:
00:22:48
because they're reliable and not erratic. And we trust people,
Brent Williams:
00:22:53
because we have a shared social world, is generally why people
Brent Williams:
00:22:55
mentioning Pie, the the product you were working with, that it
Brent Williams:
00:22:57
trust people. These elements are not very prominent in AI. And
Brent Williams:
00:23:02
so, so I think that's why trust is a kind of an important issue.
Brent Williams:
00:23:07
Although they're changing.
Brent Williams:
00:23:11
had a feel of empathy, which might mean you could emotionally
Brent Williams:
00:23:26
connect to it more easily. So some of these things seem to be evolving,
Varun Grover:
00:23:44
I think so I think that from whatever little
Varun Grover:
00:23:49
observation I've had over the last year and a half, it's not
Varun Grover:
00:23:52
been that long since ChatGPT was removed, released. But this
Varun Grover:
00:23:57
idea, I can see improvement in empathy, I think that element is
Varun Grover:
00:24:02
there. So we are starting in on that trajectory in terms of
Varun Grover:
00:24:07
understandability, there is a whole field or subfield of AI.
Varun Grover:
00:24:13
That is called Explainable AI. How do you get take this neural
Varun Grover:
00:24:18
connections? And how do you actually convert it into a
Varun Grover:
00:24:21
language that humans can understand so they can explain
Varun Grover:
00:24:24
the rationale for their decisions? Now, it may not be
Varun Grover:
00:24:28
that important for radiology reports. But if we're dealing
Varun Grover:
00:24:33
with a business discipline, you know, often you want to
Varun Grover:
00:24:36
understand the rationale for the decision. So Explainable AI is
Varun Grover:
00:24:41
actually being worked on. And and it's important, and it's
Varun Grover:
00:24:45
it's particularly important because AI is all about data and
Varun Grover:
00:24:50
connections and patterns. AI is not particularly good at
Varun Grover:
00:24:56
reasoning. So if we tell an AI that Look, if they're looking at
Varun Grover:
00:25:02
a pattern, x is the parent of y, it will not naturally infer from
Varun Grover:
00:25:07
there or deduce from there, that y is the child of x. That
Varun Grover:
00:25:13
reasoning ability will only come about if it's in the corpus of
Varun Grover:
00:25:17
data. So the question then becomes, you know, how do you
Varun Grover:
00:25:22
get this reasoning out of the AI, and that's where a lot of
Varun Grover:
00:25:26
the research on Explainable AI is, give you another example. We
Varun Grover:
00:25:31
can put millions of chess games played by world chess champions
Varun Grover:
00:25:35
and feed it into the AI and it'll figure out the rules of
Varun Grover:
00:25:38
chess and what works and what doesn't work. So that it will
Varun Grover:
00:25:41
basically be a really good chess player. But if we change the
Varun Grover:
00:25:46
board from an eight by eight board to a nine by nine board,
Varun Grover:
00:25:50
then the training doesn't really help because it needs more than
Varun Grover:
00:25:55
that it needs some kind of reasoning ability to play chess
Varun Grover:
00:25:59
well on a nine by nine input. So I think that that idea of how do
Varun Grover:
00:26:05
you get that reasoning to come into you know, the box or the
Varun Grover:
00:26:12
corpus of intelligence of AI is is something that is being
Varun Grover:
00:26:17
worked on through Explainable AI. So empathy? Yes, we're
Varun Grover:
00:26:21
seeing improvement there. Explainable AI, we're not quite
Varun Grover:
00:26:24
there. But there's certainly a direction there. And like you
Varun Grover:
00:26:28
said, absolutely the corpus of training data. So most of the
Varun Grover:
00:26:33
models, we had an AI foundational models, like from
Varun Grover:
00:26:37
open AI. And from Google and Microsoft, these models are
Varun Grover:
00:26:43
trained on massive corpus of data from the internet, the
Varun Grover:
00:26:47
internet data is noisy. And it is, you know, it has a lot of a
Varun Grover:
00:26:53
lot of problems, which means it can be a pretty good general, it
Varun Grover:
00:26:59
can provide general advice on a lot of things. But if you're
Varun Grover:
00:27:03
looking for very specialized advice, sometimes these models
Varun Grover:
00:27:07
hallucinate and create these arbitrary outcomes, because
Varun Grover:
00:27:13
they're not fully trained in those specialized areas. So I
Varun Grover:
00:27:17
think one of the trends is how do you actually curate data for
Varun Grover:
00:27:23
training and make these models more accurate and better? And
Varun Grover:
00:27:30
how do you move away from the big tech companies which have
Varun Grover:
00:27:33
the resources to train these large foundational models, to
Varun Grover:
00:27:37
smaller models that are done through open source where you
Varun Grover:
00:27:41
can, individual companies may not have to invest millions of
Varun Grover:
00:27:46
dollars in training these models, but they can train them
Varun Grover:
00:27:48
on their own data,
Brent Williams:
00:27:49
Right.
Varun Grover:
00:27:50
And, and become, it can become more more
Varun Grover:
00:27:52
personalized and more accurate within that domain. So I think
Varun Grover:
00:27:57
that's the third component of trust, where these models are
Varun Grover:
00:28:02
being trained on better data. And sometimes even instead of
Varun Grover:
00:28:06
the large models, we're getting smaller and more specialized
Varun Grover:
00:28:10
models that are more accurate. And I think that's a trajectory
Varun Grover:
00:28:14
that will improve trust. And finally, in terms of context. So
Varun Grover:
00:28:22
I think that context is also improving, because the earlier
Varun Grover:
00:28:26
models had very little that you could put into the model in
Varun Grover:
00:28:30
terms of input. Now, some of the newer models, you can put entire
Varun Grover:
00:28:35
books and and say, you know, so your context window, the number
Varun Grover:
00:28:40
of tokens that go into the context is becoming larger and
Varun Grover:
00:28:43
larger, which means you can provide a richer context to your
Varun Grover:
00:28:46
decision making. And then you can have conversations to get
Varun Grover:
00:28:52
the alignment of get what you want from the AI. And so that
Varun Grover:
00:28:57
alignment can continue by probing and interacting with the
Varun Grover:
00:29:02
AI. So I think on the third aspect, the context aspect,
Varun Grover:
00:29:07
we're also seeing improvements. So I will trust the box, because
Varun Grover:
00:29:13
I can connect with it. I can trust the box, because I know
Varun Grover:
00:29:18
where it's coming from. It can explain its reasoning to me, I
Varun Grover:
00:29:22
can trust the box. Because it gives me good outputs, accurate
Varun Grover:
00:29:27
outputs, and I can trust the box because it's grounded in my
Varun Grover:
00:29:30
world. And so all those aspects are advancing. And so I think
Varun Grover:
00:29:37
that on the technological front, we are seeing a trajectory
Varun Grover:
00:29:43
that's positive for trust.
Brent Williams:
00:29:47
I'm gonna I'm gonna ask you about to human.
Brent Williams:
00:29:51
Maybe let's start in the business context, right. So
Brent Williams:
00:29:54
let's say the models are improving, they're becoming more
Brent Williams:
00:29:57
trustworthy. They're certainly becoming more commonplace.
Brent Williams:
00:30:02
They're still probably some of the like, there's the
Brent Williams:
00:30:05
excitement. There's some of those some people who think it's
Brent Williams:
00:30:07
scary. Probably each of us land somewhere on that continuum
Brent Williams:
00:30:11
differently. But in the business world, you know, the role of the
Brent Williams:
00:30:16
human in business, does it, change it slightly? Does it
Brent Williams:
00:30:21
change? Fundamentally? You have any thoughts to share on that?
Varun Grover:
00:30:28
Yeah I think that's, that's, that's a million
Varun Grover:
00:30:32
dollar question, or maybe a billion dollar question. or now
Varun Grover:
00:30:35
given, given the market of AI, maybe a trillion question.
Brent Williams:
00:30:39
If you can answer this, we can raise a lot
Brent Williams:
00:30:40
of money for the Walton College.
Varun Grover:
00:30:43
But I think I so, how should AI, so if if we are
Varun Grover:
00:30:48
using AI, to automate jobs, if we're using AI to, to automate
Varun Grover:
00:30:56
and replace jobs, so you can, you know, you can do an
Varun Grover:
00:30:59
accounting job better, or you can do a legal job better. Then
Varun Grover:
00:31:04
what is essentially happening is we are devaluing labor. We're
Varun Grover:
00:31:11
devaluing human skills and labor, because we're essentially
Varun Grover:
00:31:15
replacing labor with the technology. And that's going to
Varun Grover:
00:31:18
bring the cost of labor down. So substitution of jobs with
Varun Grover:
00:31:23
technology is probably not the most innovative direction, it's
Varun Grover:
00:31:28
probably appropriate. And probably the incentives for
Varun Grover:
00:31:31
corporations to make money are in the cost cutting sphere. So
Varun Grover:
00:31:35
we're certainly going to see a lot of that. But I think that
Varun Grover:
00:31:39
where we're going to see the human and, and the technology
Varun Grover:
00:31:43
work better is through augmentation. You never have
Varun Grover:
00:31:48
innovation through automation, you will only have innovation,
Varun Grover:
00:31:53
if there are ways in which the AI and the technology can work
Varun Grover:
00:31:57
together. So if you automate a lot, you're basically devaluing
Varun Grover:
00:32:03
the human if you augment you're up valuing the human, because
Varun Grover:
00:32:10
when you when you automate, your competition is between the AI
Varun Grover:
00:32:15
and the human, when you augment your competition is between the
Varun Grover:
00:32:21
human with the AI, and the human without the AI.
Brent Williams:
00:32:24
Right.
Varun Grover:
00:32:25
And so you're upskilling. So I think that we
Varun Grover:
00:32:29
are at a point of inflection, where we are making 1000s of
Varun Grover:
00:32:35
decisions regarding AI. And it's really important that if
Varun Grover:
00:32:39
corporations are going to invest in AI, they should consciously
Varun Grover:
00:32:44
think about how to augment human skills through AI, rather than
Varun Grover:
00:32:50
only automation. Because automation will probably give
Varun Grover:
00:32:55
you the short term bang for the buck in terms of reduction in
Varun Grover:
00:33:00
costs. But the longer term impact on innovation will only
Varun Grover:
00:33:05
take place if you give people the discretion to leverage AI
Varun Grover:
00:33:08
and become and use it innovatively. So if there's
Varun Grover:
00:33:12
there's a series of tasks in your job, and there's a few that
Varun Grover:
00:33:16
can be automated, that's fine. But then think about what are
Varun Grover:
00:33:20
the tasks in your jobs that can be augmented. That's where the
Varun Grover:
00:33:23
innovation comes in. And that's where the value is created.
Brent Williams:
00:33:27
Well, Varun, thank you. Thank you for sharing
Brent Williams:
00:33:31
your knowledge and that knowledge that comes from
Brent Williams:
00:33:35
decades of work and research and I know I learned a lot and I'm
Brent Williams:
00:33:41
sure our audience did as well. So thank you for joining me
Brent Williams:
00:33:44
today.
Varun Grover:
00:33:44
Thank you, Brent for having me.
Brent Williams:
00:33:45
On behalf of the Walton College thank you for
Brent Williams:
00:33:48
joining us for this captivating conversation. To stay connected
Brent Williams:
00:33:52
and never miss an episode, simply search for Be Epic on
Brent Williams:
00:33:55
your preferred podcast service.
