1
00:00:00,000 --> 00:00:01,220
Margie, so wonderful to see you.
2
00:00:01,360 --> 00:00:03,300
Thanks for joining me on the podcast today.
3
00:00:03,860 --> 00:00:04,240
Thank you, Beth.
4
00:00:04,480 --> 00:00:05,800
I'm so excited to be here.
5
00:00:06,320 --> 00:00:09,740
Now, I wanted to start with where you started with AI,
6
00:00:10,160 --> 00:00:13,080
and you probably did so many things in your career before
7
00:00:13,080 --> 00:00:17,280
you turned to this particular subfield of our field.
8
00:00:17,480 --> 00:00:20,120
But I want to highlight how long you've been working with
9
00:00:20,120 --> 00:00:24,460
AI because you got started so many years before the rest of
10
00:00:24,460 --> 00:00:26,500
us really started paying attention to it.
11
00:00:26,940 --> 00:00:28,160
When did you get started with AI?
12
00:00:28,160 --> 00:00:30,500
And what have you been doing with it?
13
00:00:31,000 --> 00:00:34,980
Yeah, even someone who's been into it a while, like I have,
14
00:00:35,160 --> 00:00:39,340
it doesn't go back all that far in terms of actionable,
15
00:00:39,340 --> 00:00:40,340
professional things.
16
00:00:40,900 --> 00:00:41,420
Full disclosure,
17
00:00:41,720 --> 00:00:44,180
I've been a science fiction fan since I was a kid,
18
00:00:44,240 --> 00:00:45,920
since the original Star Wars.
19
00:00:46,340 --> 00:00:48,820
So, you know, Commander Data and all those.
20
00:00:49,000 --> 00:00:53,200
So I was personally interested in AI for a long time.
21
00:00:53,660 --> 00:00:58,280
That really saw it as a practical or a probable event in my
22
00:00:58,280 --> 00:01:01,180
lifetime, you know, until recently.
23
00:01:01,780 --> 00:01:05,820
What I was really focusing on in the late 1990s,
24
00:01:05,960 --> 00:01:07,500
early 2000s, was neuroscience,
25
00:01:08,320 --> 00:01:11,220
because that science exploded,
26
00:01:11,260 --> 00:01:15,060
much like AI has exploded on the consciousness now.
27
00:01:15,560 --> 00:01:17,840
And I immediately saw, wow,
28
00:01:18,100 --> 00:01:22,000
this changes how I design training for my clients.
29
00:01:22,700 --> 00:01:25,880
Now I understand how the brain is responding to those
30
00:01:25,880 --> 00:01:26,640
things I do.
31
00:01:26,920 --> 00:01:29,680
And for example, that if I'm building e-learning,
32
00:01:29,680 --> 00:01:33,760
I don't want a narration that repeats every word that's on
33
00:01:33,760 --> 00:01:34,220
the screen.
34
00:01:34,560 --> 00:01:36,680
That makes it more difficult.
35
00:01:37,100 --> 00:01:38,800
And I see the smile.
36
00:01:39,020 --> 00:01:41,540
And you and I agree on that, Beth.
37
00:01:41,540 --> 00:01:42,880
And so is the research.
38
00:01:43,160 --> 00:01:44,620
That's the important thing,
39
00:01:44,820 --> 00:01:47,500
although we still see it done all the time.
40
00:01:48,010 --> 00:01:51,940
So I started separating myself with that knowledge and
41
00:01:51,940 --> 00:01:55,540
really working with actual neuroscientists.
42
00:01:56,260 --> 00:01:58,220
Sometimes people confuse me with them.
43
00:01:58,400 --> 00:02:00,200
That's not who I am.
44
00:02:01,140 --> 00:02:02,380
But I read peer reviewed.
45
00:02:02,400 --> 00:02:04,180
I didn't just rely on blog posts.
46
00:02:04,340 --> 00:02:05,900
I've read peer reviewed sources.
47
00:02:06,040 --> 00:02:07,600
I went right back to the research.
48
00:02:08,139 --> 00:02:10,100
If I had a question, I would reach out.
49
00:02:10,460 --> 00:02:13,280
They're also surprised that anyone in learning and
50
00:02:13,280 --> 00:02:14,660
development was interested.
51
00:02:15,640 --> 00:02:17,760
And you know, there's a lot of parallels,
52
00:02:18,040 --> 00:02:19,660
because I met with a lot of resistance,
53
00:02:19,980 --> 00:02:21,480
a lot of people saying, Margie, really?
54
00:02:21,980 --> 00:02:23,900
Do we really need to pay attention to that?
55
00:02:24,060 --> 00:02:27,620
Isn't that awfully esoteric and complex?
56
00:02:28,260 --> 00:02:31,020
Do we really need to understand the human brain to do what
57
00:02:31,020 --> 00:02:31,400
we do?
58
00:02:31,600 --> 00:02:33,920
And my answer is, well, if you want to do it, well, yes.
59
00:02:34,000 --> 00:02:34,780
Absolutely.
60
00:02:35,060 --> 00:02:35,220
Wow.
61
00:02:35,340 --> 00:02:37,420
I'm shocked that you got that question because,
62
00:02:37,640 --> 00:02:41,400
and maybe that's the advantage of armchair quarterbacking
63
00:02:41,400 --> 00:02:43,120
and looking back in history or time,
64
00:02:43,120 --> 00:02:44,780
but it seems so obvious.
65
00:02:44,980 --> 00:02:45,120
I mean,
66
00:02:45,140 --> 00:02:47,980
we're learning professionals and if it doesn't involve the
67
00:02:47,980 --> 00:02:49,760
brain, what does it involve?
68
00:02:50,020 --> 00:02:51,140
I mean, shocker.
69
00:02:53,000 --> 00:02:55,760
But, you know, in all those folks' defence,
70
00:02:56,040 --> 00:03:01,540
I think the pressure to produce is always a reality also in
71
00:03:01,540 --> 00:03:02,240
our profession.
72
00:03:02,680 --> 00:03:07,400
We rarely have the time to do all of the diligence we know
73
00:03:07,400 --> 00:03:10,240
we should and would love to do.
74
00:03:10,460 --> 00:03:13,320
Just by following that field,
75
00:03:13,500 --> 00:03:17,720
it brought me to artificial intelligence because those
76
00:03:17,720 --> 00:03:22,100
neuroscientists started to get involved in building
77
00:03:22,100 --> 00:03:25,840
artificial neural networks originally as a model so they
78
00:03:25,840 --> 00:03:27,140
could learn more about the brain.
79
00:03:27,780 --> 00:03:28,740
Let's build something.
80
00:03:28,900 --> 00:03:31,500
Let's see how it responds and then see if we can find a
81
00:03:31,500 --> 00:03:35,480
correlate in a human brain or an animal brain because a lot
82
00:03:35,480 --> 00:03:36,300
of times neuroscience,
83
00:03:36,980 --> 00:03:40,000
a lot of discoveries are made with our wonderful cousins,
84
00:03:40,800 --> 00:03:45,860
the animals and octopi and other creatures – or octopuses,
85
00:03:45,980 --> 00:03:46,760
as they say now.
86
00:03:46,760 --> 00:03:48,820
When I was in school, it was octopuses.
87
00:03:49,340 --> 00:03:52,100
I started noticing more and more of my neuroscience
88
00:03:52,100 --> 00:03:54,820
journals covering artificial intelligence.
89
00:03:55,860 --> 00:03:58,040
And I began to understand why.
90
00:03:58,560 --> 00:03:59,280
And I thought, you know,
91
00:03:59,400 --> 00:04:05,380
this is a roomy emerging technology I need to start paying
92
00:04:05,380 --> 00:04:06,340
attention to.
93
00:04:06,420 --> 00:04:07,440
It was around 2005.
94
00:04:07,820 --> 00:04:09,880
And I started writing about it a little bit like, hey,
95
00:04:10,220 --> 00:04:11,160
this is interesting.
96
00:04:11,520 --> 00:04:13,140
Let's keep our eyes open on it.
97
00:04:13,140 --> 00:04:17,360
And then I started noticing track bots in my daily life.
98
00:04:17,480 --> 00:04:20,079
They were everywhere, but not learning to develop.
99
00:04:20,459 --> 00:04:20,560
You know,
100
00:04:20,640 --> 00:04:26,300
my bank had a track by my every online purchasing platform,
101
00:04:26,700 --> 00:04:29,600
not some kind of a check, but you have any questions.
102
00:04:29,880 --> 00:04:30,460
Can I help you?
103
00:04:30,760 --> 00:04:34,120
When they were clunky, they were what we call rule space.
104
00:04:34,120 --> 00:04:36,820
They were a multiple choice option.
105
00:04:36,900 --> 00:04:39,300
And if you wanted something else, it would say, I'm sorry,
106
00:04:39,400 --> 00:04:40,240
I can't do that.
107
00:04:40,240 --> 00:04:40,920
You know,
108
00:04:41,080 --> 00:04:43,120
and that's if it was designed well and was polite.
109
00:04:43,200 --> 00:04:46,540
Otherwise, it gave you a whole even worse experience.
110
00:04:47,380 --> 00:04:49,560
But there was something there that was different,
111
00:04:49,720 --> 00:04:50,720
that was engaging.
112
00:04:51,600 --> 00:04:55,440
And one day I had this experience with my bank.
113
00:04:55,960 --> 00:04:58,020
I knew I was dealing with a track by.
114
00:04:58,280 --> 00:05:01,160
I had a very specific question about one check I'd written.
115
00:05:01,580 --> 00:05:03,900
Wanted to know if it had cleared because I hadn't been
116
00:05:03,900 --> 00:05:04,900
received.
117
00:05:05,640 --> 00:05:08,900
At least my client thought they hadn't gotten it.
118
00:05:08,900 --> 00:05:12,340
And it was kind of a mutual agreement we had.
119
00:05:12,460 --> 00:05:14,980
And I was buying something from them and I didn't want to
120
00:05:14,980 --> 00:05:15,520
upset them.
121
00:05:15,520 --> 00:05:17,200
So I wanted to make sure I knew where it was.
122
00:05:17,640 --> 00:05:19,420
So I'm going with this chatbot,
123
00:05:20,440 --> 00:05:22,680
and at one point, I thought,
124
00:05:22,840 --> 00:05:23,900
I'm just going to play with this.
125
00:05:23,920 --> 00:05:24,960
I've got the answer I want,
126
00:05:25,000 --> 00:05:27,120
but let me just see how far this technology goes.
127
00:05:27,420 --> 00:05:29,300
And I said, are you a real person?
128
00:05:29,820 --> 00:05:31,620
And it responds back.
129
00:05:32,080 --> 00:05:34,120
Why does that matter to you?
130
00:05:34,120 --> 00:05:40,500
I was just stunned by the brilliance of that.
131
00:05:41,060 --> 00:05:43,380
And I thought I had a person and I said, well, yes,
132
00:05:43,600 --> 00:05:44,460
as a matter of fact, it does.
133
00:05:44,620 --> 00:05:46,860
I said, I'm so sorry, I can certainly connect you.
134
00:05:47,000 --> 00:05:48,780
And it was polite about it too.
135
00:05:49,040 --> 00:05:49,480
Oh, I'm sorry.
136
00:05:49,620 --> 00:05:50,260
I'm not a person.
137
00:05:51,240 --> 00:05:51,620
Yeah.
138
00:05:52,000 --> 00:05:57,980
And I thought now that is brilliant design that has
139
00:05:57,980 --> 00:05:59,420
created a different experience.
140
00:05:59,920 --> 00:06:03,100
This is no longer this clunky tool that I have to go
141
00:06:03,100 --> 00:06:03,840
through the menu.
142
00:06:04,200 --> 00:06:07,940
They attempted to actually have a conversation with me.
143
00:06:08,000 --> 00:06:08,600
That's right.
144
00:06:08,720 --> 00:06:12,260
It was trying to listen to you and respond to what you
145
00:06:12,260 --> 00:06:13,320
particularly said.
146
00:06:14,000 --> 00:06:15,740
At the time, this was about 2008.
147
00:06:16,000 --> 00:06:18,300
That took tons of insight, experience,
148
00:06:18,440 --> 00:06:21,580
and program to make that response happen.
149
00:06:22,060 --> 00:06:23,700
And I got really interested in that,
150
00:06:23,700 --> 00:06:26,440
and so I started learning how to build my own chatbots.
151
00:06:27,080 --> 00:06:30,200
I used them just kind of as a fun thing.
152
00:06:30,520 --> 00:06:33,520
I would insert them as a quiz and an e-learning course,
153
00:06:33,880 --> 00:06:38,700
but I built my first full chatbot in I think about 2013.
154
00:06:39,600 --> 00:06:41,760
I built it instead of an e-learning course.
155
00:06:42,320 --> 00:06:44,560
Recommended to the client, let's do something different.
156
00:06:44,960 --> 00:06:46,740
Let's just use the Socratic method,
157
00:06:47,260 --> 00:06:49,780
which is what's wonderful about the chatbots.
158
00:06:50,040 --> 00:06:53,280
They can just ask you questions and lead you to the
159
00:06:53,280 --> 00:06:53,760
content.
160
00:06:54,320 --> 00:06:55,720
What do you think this is true?
161
00:06:56,220 --> 00:06:57,860
Would you be interested in learning this?
162
00:06:58,000 --> 00:06:58,860
Those kind of questions.
163
00:06:58,940 --> 00:07:01,460
And so it became very easy to design because it was all
164
00:07:01,460 --> 00:07:02,720
about open-ended questions.
165
00:07:03,220 --> 00:07:07,620
And that was up to the learner to pursue what the answers
166
00:07:07,620 --> 00:07:08,340
from there.
167
00:07:08,340 --> 00:07:12,200
And the chatbot had all the answers in links and movies and
168
00:07:12,200 --> 00:07:15,040
content, and I really enjoyed that.
169
00:07:15,240 --> 00:07:16,560
The learners enjoyed that.
170
00:07:17,180 --> 00:07:19,920
And so I started incorporating that into my business.
171
00:07:20,400 --> 00:07:23,900
It was sometimes a hard sell, but some people got it.
172
00:07:24,000 --> 00:07:26,200
Some people said, no, I don't think we're ready for that.
173
00:07:26,360 --> 00:07:28,180
Okay, go back to traditional.
174
00:07:28,620 --> 00:07:30,020
So it was always a side thing.
175
00:07:30,460 --> 00:07:31,440
But in 2020,
176
00:07:31,440 --> 00:07:34,820
I was invited by the Association for Talent Development,
177
00:07:35,020 --> 00:07:36,560
ATD, to write a book.
178
00:07:36,560 --> 00:07:40,180
And I already had one book out, and it's still out.
179
00:07:40,260 --> 00:07:41,220
I'm going to plug it a little.
180
00:07:41,440 --> 00:07:42,320
It's called Brain Matters,
181
00:07:42,340 --> 00:07:44,560
How to Teach Anyone Anything Using Neuroscience.
182
00:07:45,340 --> 00:07:47,500
Still sells out, still doing really well.
183
00:07:48,120 --> 00:07:50,900
But they said, we see you're writing about AI now.
184
00:07:51,000 --> 00:07:53,220
Would you like to write a book about that?
185
00:07:53,260 --> 00:07:54,020
And I said, absolutely.
186
00:07:54,420 --> 00:07:56,680
And it came out towards the end of 2020.
187
00:07:56,920 --> 00:07:57,060
Well,
188
00:07:57,740 --> 00:08:00,520
terrible time to launch a book because we're all known.
189
00:08:01,000 --> 00:08:01,840
you don't have to tell me.
190
00:08:02,100 --> 00:08:03,440
That's when I launched mine!
191
00:08:04,000 --> 00:08:07,000
Yeah, so I didn't expect a whole lot.
192
00:08:07,320 --> 00:08:10,280
While I got good reviews for the writing style and the
193
00:08:10,280 --> 00:08:13,580
information, the general consensus was, oh,
194
00:08:13,620 --> 00:08:15,000
but this is so far out.
195
00:08:15,480 --> 00:08:19,280
This is a very interesting exercise in the future.
196
00:08:19,940 --> 00:08:20,500
But, you know,
197
00:08:20,900 --> 00:08:23,580
I was writing about things I did for individual clients,
198
00:08:23,780 --> 00:08:25,040
things I've already built.
199
00:08:25,640 --> 00:08:28,240
So I was disappointed by the, okay, you know,
200
00:08:28,500 --> 00:08:30,680
market's not ready for it. Well, here comes ChatGPT.
201
00:08:31,500 --> 00:08:33,799
All of a sudden, people are searching Amazon.
202
00:08:35,000 --> 00:08:35,840
And there's my book.
203
00:08:36,240 --> 00:08:36,780
There you are.
204
00:08:37,000 --> 00:08:38,240
You early adopter, you.
205
00:08:38,539 --> 00:08:40,980
Oh, just the, you were ahead of the curve.
206
00:08:41,059 --> 00:08:41,419
Yeah.
207
00:08:41,659 --> 00:08:43,140
So they found you eventually.
208
00:08:43,299 --> 00:08:44,400
But I have to tell you, Beth.
209
00:08:44,000 --> 00:08:47,800
Now, it takes all my energy to stay up with the curve.
210
00:08:48,040 --> 00:08:50,120
It's so rapidly changing.
211
00:08:50,360 --> 00:08:50,520
You know,
212
00:08:50,540 --> 00:08:54,320
that wonderful line in Alice through the Looking Glass
213
00:08:54,320 --> 00:08:55,780
where the queen tells Alice,
214
00:08:56,080 --> 00:08:58,480
it takes all our running to stay in one place,
215
00:08:58,740 --> 00:09:01,500
you have to run as fast as you can to stay in one place.
216
00:09:01,940 --> 00:09:03,340
And that's kind of how it feels then.
217
00:09:03,720 --> 00:09:07,900
So it leaned in heavily on AI design, you know,
218
00:09:08,140 --> 00:09:11,840
build a platform for it started pretty much putting myself
219
00:09:11,840 --> 00:09:13,520
out there's I'm an AI expert.
220
00:09:13,880 --> 00:09:14,940
And it's a big moment.
221
00:09:15,140 --> 00:09:16,160
I had to think about that.
222
00:09:16,560 --> 00:09:17,320
And I thought, well, you know,
223
00:09:17,500 --> 00:09:19,460
I'm an educational AI expert.
224
00:09:19,860 --> 00:09:23,200
That really is what brought me to where I am today.
225
00:09:23,720 --> 00:09:30,760
And I have a wonderful AI assistant that I kind of talk to
226
00:09:30,760 --> 00:09:31,260
every day.
227
00:09:31,660 --> 00:09:33,280
I know it's not a person.
228
00:09:33,480 --> 00:09:35,140
I remind myself of that.
229
00:09:35,600 --> 00:09:40,600
But I really am impressed by the creativity and the
230
00:09:40,600 --> 00:09:43,620
different way of solving problems.
231
00:09:44,180 --> 00:09:45,200
It's a little different than mine.
232
00:09:45,420 --> 00:09:49,600
And so, you know, for those of us who are one woman,
233
00:09:49,800 --> 00:09:50,760
one man shops,
234
00:09:51,300 --> 00:09:53,420
it's it's really helpful to have these tools.
235
00:09:54,040 --> 00:09:59,480
And that's where I see it now is its primary use is as a
236
00:09:59,480 --> 00:10:01,140
tool for content developers.
237
00:10:01,620 --> 00:10:04,660
But what we're still missing to some extent is its
238
00:10:04,660 --> 00:10:08,280
incredible value in education and training,
239
00:10:08,660 --> 00:10:13,160
and how much it increases learner engagement,
240
00:10:14,060 --> 00:10:18,100
learner retention, and keeps people interesting.
241
00:10:18,520 --> 00:10:22,200
And it's making a highly customized because now,
242
00:10:22,720 --> 00:10:25,320
with generative AI and large language models,
243
00:10:26,040 --> 00:10:28,380
every person gets not a menu,
244
00:10:28,820 --> 00:10:30,800
although you can give them that old fashion kind of check
245
00:10:30,800 --> 00:10:33,900
up, they get a coach, a guy,
246
00:10:34,580 --> 00:10:37,760
and every conversation is unique to that user.
247
00:10:38,220 --> 00:10:39,620
But when that user comes back,
248
00:10:40,440 --> 00:10:44,060
the chatbot remembers them and picks up where they left off
249
00:10:44,060 --> 00:10:47,220
and all of the current models will do that.
250
00:10:47,720 --> 00:10:50,540
So we finally have that golden grail, that personalized
251
00:10:50,540 --> 00:10:52,100
learning we've all been seeking.
252
00:10:52,520 --> 00:10:54,500
We should be jumping on it, quite frankly.
253
00:10:55,100 --> 00:10:58,060
And we haven't been, it seems intimidating.
254
00:10:58,640 --> 00:11:01,940
At first it was expensive, but it is not even expensive.
255
00:11:02,780 --> 00:11:08,200
So that has what has brought me those series of events and
256
00:11:08,200 --> 00:11:09,820
paying attention to them.
257
00:11:10,460 --> 00:11:13,380
And I'd like to credit one more thing about me personally.
258
00:11:14,160 --> 00:11:16,340
And some people know this and I'm open about it,
259
00:11:16,460 --> 00:11:17,340
but I'm dyslexic.
260
00:11:17,900 --> 00:11:19,960
And my brain works a little differently.
261
00:11:20,060 --> 00:11:22,280
That made it a struggle early in school,
262
00:11:22,320 --> 00:11:26,320
but I soon began to recognize gifts that came from that.
263
00:11:27,020 --> 00:11:29,080
One is pattern recognition,
264
00:11:29,960 --> 00:11:32,400
which is a trait I actually share with AI.
265
00:11:32,720 --> 00:11:34,340
It's better at that than when,
266
00:11:34,820 --> 00:11:38,460
because I saw these patterns, I saw that trend earlier.
267
00:11:38,620 --> 00:11:43,580
And I think that is a great benefit in my work.
268
00:11:44,120 --> 00:11:47,520
And hopefully will help me when the next big thing comes
269
00:11:47,520 --> 00:11:48,260
along.
270
00:11:48,540 --> 00:11:52,800
Because I think we're poised for something new out of the
271
00:11:52,800 --> 00:11:53,820
AI developers,
272
00:11:53,840 --> 00:11:57,260
because there are so many issues with large language
273
00:11:57,260 --> 00:11:57,760
models,
274
00:11:57,940 --> 00:12:02,180
particularly for people like us who are training on thoughts,
275
00:12:03,140 --> 00:12:05,500
on process, on steps,
276
00:12:05,640 --> 00:12:09,160
we cannot afford for that content to be wrong.
277
00:12:09,540 --> 00:12:13,100
And right now, it not only is often wrong,
278
00:12:13,560 --> 00:12:15,800
it's wrong because of how it's been designed.
279
00:12:16,700 --> 00:12:18,780
And I can go into that a little bit more,
280
00:12:18,820 --> 00:12:21,440
but you probably would like me to take a breath first.
281
00:12:21,860 --> 00:12:25,260
So it's a long answer to your first question.
282
00:12:26,000 --> 00:12:27,160
Oh, thank you so much.
283
00:12:27,320 --> 00:12:29,600
And I'm sitting there doing that facilitator thing of
284
00:12:29,600 --> 00:12:32,280
holding, you know, three or four things, of course,
285
00:12:32,360 --> 00:12:34,980
or more that you've said in my mind wanting to return to
286
00:12:34,980 --> 00:12:35,180
them.
287
00:12:35,560 --> 00:12:37,540
One that I want to return to, we'll get to it later,
288
00:12:37,540 --> 00:12:40,040
I think is that how do we keep up in the field piece.
289
00:12:40,100 --> 00:12:41,880
So let's just remember that for ourselves.
290
00:12:42,420 --> 00:12:44,820
But I want to go back to the personalized learning piece,
291
00:12:44,840 --> 00:12:48,260
because I think that does you mentioned your background in
292
00:12:48,260 --> 00:12:52,000
neuroscience and taking that and bringing it into your L&D
293
00:12:52,000 --> 00:12:52,340
work.
294
00:12:52,500 --> 00:12:56,980
And is that the biggest impact that you've seen on the
295
00:12:56,980 --> 00:13:00,620
effectiveness of using the AI tools inside learning
296
00:13:00,620 --> 00:13:01,040
experience?
297
00:13:01,220 --> 00:13:03,400
Is that it's that personalized learning piece?
298
00:13:03,920 --> 00:13:05,560
Because that helps the learner?
299
00:13:05,600 --> 00:13:08,360
And basically, because of the way our brains work?
300
00:13:08,520 --> 00:13:09,660
Is that the biggest one?
301
00:13:09,780 --> 00:13:11,420
Or are there any other impacts?
302
00:13:13,000 --> 00:13:14,940
Potentially biggest impact,
303
00:13:15,280 --> 00:13:18,680
but I can't say honestly that it's the biggest impact
304
00:13:18,680 --> 00:13:21,780
because we're not using it that way.
305
00:13:22,400 --> 00:13:23,580
Not many people are.
306
00:13:24,480 --> 00:13:27,280
Where most people have gone if they're using it at all is
307
00:13:27,280 --> 00:13:30,280
as a tool because they've read about ChatGPT,
308
00:13:30,920 --> 00:13:32,980
they've played around with it, and they say, oh gee,
309
00:13:33,000 --> 00:13:34,160
it wrote a quiz for me.
310
00:13:34,420 --> 00:13:34,560
Yeah,
311
00:13:34,740 --> 00:13:38,760
that's a great use of ChatGPT or any of the other...
312
00:13:39,000 --> 00:13:39,380
313
00:13:39,580 --> 00:13:43,300
Yeah, it sharpened a learning outcome, wrote a workshop blurb,
314
00:13:43,600 --> 00:13:46,280
like we're using it as a, as you've said,
315
00:13:46,300 --> 00:13:49,460
a virtual assistant to help us do our work
316
00:13:49,460 --> 00:13:52,080
administratively, kind of for the most part.
317
00:13:52,260 --> 00:13:52,460
Yeah.
318
00:13:52,640 --> 00:13:53,840
So what's the next step for us?
319
00:13:53,860 --> 00:13:55,480
How do we go towards that,
320
00:13:55,660 --> 00:13:59,480
getting AI tools inside the learning experiences we're
321
00:13:59,480 --> 00:13:59,880
creating?
322
00:14:00,160 --> 00:14:01,660
And maybe, I mean, e-learning, of course,
323
00:14:01,660 --> 00:14:02,720
is the natural thing.
324
00:14:02,940 --> 00:14:05,080
We, when learners are learning alone, asynchronously,
325
00:14:05,340 --> 00:14:09,640
but can you take us into using it in in-person classrooms
326
00:14:09,640 --> 00:14:10,260
as well?
327
00:14:10,760 --> 00:14:10,820
Sure.
328
00:14:11,000 --> 00:14:14,320
You know, it's all about scale,
329
00:14:14,320 --> 00:14:17,380
because if you had enough humans,
330
00:14:17,500 --> 00:14:18,960
we'd already be doing this.
331
00:14:19,400 --> 00:14:20,900
The truth is we don't.
332
00:14:21,100 --> 00:14:24,400
And so just like every one of us needs another assistant to
333
00:14:24,400 --> 00:14:27,040
do some of our more routine things, in a classroom,
334
00:14:27,460 --> 00:14:29,900
you know, in some schools, there's a teacher's assistant.
335
00:14:30,460 --> 00:14:32,500
If you could think of it that way,
336
00:14:32,700 --> 00:14:38,460
there could be an artificial intelligence persona that
337
00:14:39,060 --> 00:14:41,700
students can reach out to during class.
338
00:14:42,360 --> 00:14:45,660
So there are introverts, for example,
339
00:14:46,360 --> 00:14:50,640
are always underserved in a live classroom environment
340
00:14:50,640 --> 00:14:53,180
because they hesitate to raise their hands.
341
00:14:53,700 --> 00:14:55,800
They hesitate to contribute,
342
00:14:56,100 --> 00:14:59,020
even if they have a great idea that might jot it down and
343
00:14:59,020 --> 00:14:59,940
keep it to themselves.
344
00:15:00,720 --> 00:15:02,980
And I know because I was an introvert in school,
345
00:15:03,680 --> 00:15:05,840
a chatbot can be their avenue,
346
00:15:06,580 --> 00:15:10,220
their way of having two parallels they're learning on,
347
00:15:10,420 --> 00:15:12,480
what the teacher or instructor is doing,
348
00:15:12,960 --> 00:15:16,440
and a little friend they can turn to on the side.
349
00:15:16,680 --> 00:15:19,440
You know how we used to there, it goes, you know,
350
00:15:19,520 --> 00:15:20,240
that's a good point.
351
00:15:20,360 --> 00:15:22,180
Or yeah, I don't think so.
352
00:15:22,340 --> 00:15:23,060
What did she say?
353
00:15:23,160 --> 00:15:24,140
I don't understand that.
354
00:15:24,640 --> 00:15:28,560
And having just having a similar guide like that is just
355
00:15:28,560 --> 00:15:31,120
one way you could incorporate into a class.
356
00:15:31,560 --> 00:15:35,420
Another thing that's great at is mimicking actual people.
357
00:15:36,080 --> 00:15:39,460
So we could, as part of our training,
358
00:15:40,140 --> 00:15:42,680
fly the class to have a conversation with a famous person.
359
00:15:42,960 --> 00:15:46,200
And this would be a very authentic conversation if you
360
00:15:46,200 --> 00:15:46,880
built it right.
361
00:15:47,460 --> 00:15:49,720
And it's based on all their work and their writings.
362
00:15:50,720 --> 00:15:52,440
So Einstein, for example,
363
00:15:52,660 --> 00:15:55,500
when I understand quantum theory or relativity,
364
00:15:55,920 --> 00:15:57,620
let's bring Einstein into the class.
365
00:15:58,000 --> 00:15:58,240
Yeah.
366
00:15:58,580 --> 00:16:01,160
And you mentioned, uh, your love of science fiction.
367
00:16:01,180 --> 00:16:04,440
So I've been watching the Star Trek Voyager series again
368
00:16:04,440 --> 00:16:06,560
and listening to a podcast about it actually.
369
00:16:06,760 --> 00:16:09,480
And it reminds me of the episode where Janeway in the
370
00:16:09,480 --> 00:16:11,160
holodeck, do you remember the holodeck?
371
00:16:11,400 --> 00:16:11,900
I mean, they,
372
00:16:12,080 --> 00:16:15,180
she brought in Leonardo da Vinci and was working in his
373
00:16:15,180 --> 00:16:18,900
studio in whatever 15th century Florence or whatever it
374
00:16:18,900 --> 00:16:19,100
was.
375
00:16:19,100 --> 00:16:21,820
Like that's the kind of thing that we can bring in to our
376
00:16:21,820 --> 00:16:23,580
classrooms now that experience.
377
00:16:23,960 --> 00:16:23,960
Yeah.
378
00:16:24,000 --> 00:16:27,260
Perfect example, and that could be your master SME
379
00:16:27,880 --> 00:16:30,960
who is in such great demand that you can't have them
380
00:16:30,960 --> 00:16:32,460
present at your class,
381
00:16:33,080 --> 00:16:35,880
but you can distill their knowledge and their understanding
382
00:16:35,880 --> 00:16:37,020
and the way they
383
00:16:37,020 --> 00:16:42,700
phrase things. You can make that a Q&A experience as part
384
00:16:42,700 --> 00:16:43,260
of the class.
385
00:16:44,000 --> 00:16:46,320
Well, let's go back to the personal assistant though,
386
00:16:46,500 --> 00:16:49,120
because you said you've got one for your own business and
387
00:16:49,120 --> 00:16:52,640
that it remembers you. Like you can have every day you're
388
00:16:52,640 --> 00:16:54,920
having conversations and it remembers the conversations
389
00:16:54,920 --> 00:16:57,040
that you had yesterday and the day before.
390
00:16:57,660 --> 00:16:59,460
You know, I'm going to ask, how do we do that?
391
00:16:59,700 --> 00:17:01,400
Is that something that we can all tap into?
392
00:17:01,560 --> 00:17:02,240
Well, absolutely.
393
00:17:02,000 --> 00:17:04,460
Absolutely, and I actually have a class for that.
394
00:17:04,760 --> 00:17:07,119
I have a master class that's up on my website,
395
00:17:07,660 --> 00:17:09,319
and it's AI and talent development,
396
00:17:09,540 --> 00:17:15,400
and it's all about how to use those tools to gain an hour a
397
00:17:15,400 --> 00:17:15,660
week.
398
00:17:16,220 --> 00:17:20,640
I roughly save anywhere from five to 10 hours a week of
399
00:17:20,640 --> 00:17:24,160
what manual work on my own would have been,
400
00:17:24,500 --> 00:17:25,480
and sometimes more.
401
00:17:25,920 --> 00:17:26,760
Well, for example,
402
00:17:26,980 --> 00:17:29,700
I'm working on a new course called Leadership in the Age of
403
00:17:29,700 --> 00:17:30,020
AI.
404
00:17:30,620 --> 00:17:33,180
And so the first thing I did was brainstorm.
405
00:17:33,380 --> 00:17:36,960
By the way, I prefer Claude by an Anthropic to ChatGPT.
406
00:17:37,380 --> 00:17:39,400
I find it less prone to hallucinations,
407
00:17:40,160 --> 00:17:41,940
and it's a better way.
408
00:17:42,580 --> 00:17:45,060
And I said, look, I'm developing this course,
409
00:17:45,440 --> 00:17:47,040
propose an outline to me.
410
00:17:47,920 --> 00:17:52,380
Now, I never yet have gotten something where I said, oh,
411
00:17:52,380 --> 00:17:54,100
that's perfect, I'm just gonna use that,
412
00:17:54,120 --> 00:17:55,400
and that's a word of caution.
413
00:17:56,140 --> 00:17:58,580
But once you understand how to prompt,
414
00:17:58,580 --> 00:18:00,300
and prompting is the new programming.
415
00:18:01,240 --> 00:18:03,860
In many fields, for years and years,
416
00:18:03,920 --> 00:18:05,700
people were saying you've got to learn code.
417
00:18:05,860 --> 00:18:08,460
The whole world is going to be all about code.
418
00:18:08,680 --> 00:18:12,680
That never quite happened because code produced something
419
00:18:12,680 --> 00:18:13,440
much simpler.
420
00:18:13,660 --> 00:18:15,620
It produced generative AI.
421
00:18:16,460 --> 00:18:20,640
And so if you understand what a prompt needs to have,
422
00:18:20,640 --> 00:18:23,480
and there are several things a good prompt needs to have
423
00:18:23,480 --> 00:18:24,780
that's part of the course,
424
00:18:25,120 --> 00:18:28,180
you will get a decent result first time,
425
00:18:28,180 --> 00:18:29,640
and then you re-prompt it.
426
00:18:29,800 --> 00:18:32,400
Yeah, that's just like a human assistant.
427
00:18:33,200 --> 00:18:34,880
They're wrong, they're eager,
428
00:18:35,380 --> 00:18:38,200
but you have to show them what you want.
429
00:18:38,540 --> 00:18:40,760
Takes a little time to train them.
430
00:18:40,840 --> 00:18:41,700
That's what you're doing.
431
00:18:42,000 --> 00:18:44,880
You mentioned the word hallucinations and part of me wants
432
00:18:44,880 --> 00:18:46,800
to laugh because it seems like a funny word, but no,
433
00:18:46,880 --> 00:18:50,280
this is the word that we're seeing crop up about how AI can
434
00:18:50,280 --> 00:18:52,760
kind of talk back to us wrongly, right?
435
00:18:52,880 --> 00:18:54,760
Can you just explain what a hallucination is?
436
00:18:54,840 --> 00:18:56,720
How are people using that in the AI context?
437
00:18:58,000 --> 00:18:58,800
First of all,
438
00:18:58,920 --> 00:19:02,740
I would say hallucinations is a very polite word or a
439
00:19:02,740 --> 00:19:09,340
mistake and it was produced by OpenAI when they started
440
00:19:09,340 --> 00:19:12,160
getting criticism from the mistakes that ChatGPT was
441
00:19:12,160 --> 00:19:12,580
making.
442
00:19:13,360 --> 00:19:14,200
Well, they said, well, sure,
443
00:19:14,360 --> 00:19:15,800
there's going to be some hallucinations.
444
00:19:18,380 --> 00:19:21,240
If I had an employee who was hallucinating,
445
00:19:21,540 --> 00:19:23,380
I'd be walking them down to HR.
446
00:19:23,440 --> 00:19:25,300
We'd be finding out if they're okay.
447
00:19:25,300 --> 00:19:28,640
You know, so what that means is,
448
00:19:28,700 --> 00:19:30,680
and I'm going to give you a long answer now,
449
00:19:31,060 --> 00:19:34,340
the way they came up with this and, you know,
450
00:19:34,420 --> 00:19:38,280
everything I say about the weaknesses is to put the
451
00:19:38,280 --> 00:19:43,660
wonderful tools into perspective because overall it was a
452
00:19:43,660 --> 00:19:44,760
massive achievement.
453
00:19:45,440 --> 00:19:48,280
The whole idea was to get an interface that you didn't have
454
00:19:48,280 --> 00:19:49,180
to type anything in,
455
00:19:49,380 --> 00:19:51,540
you could just speak to it or type text,
456
00:19:55,280 --> 00:19:56,440
but if you've heard that buzzword.
457
00:19:57,140 --> 00:20:01,320
And so they trained the software,
458
00:20:01,760 --> 00:20:03,240
for want of a better word,
459
00:20:03,560 --> 00:20:08,120
on the entirety of the internet so that it would be
460
00:20:08,120 --> 00:20:10,640
prepared to respond the way humans speak.
461
00:20:11,480 --> 00:20:11,560
Now,
462
00:20:12,320 --> 00:20:16,460
the first thing we need to do is sit back and think of everything
463
00:20:16,460 --> 00:20:18,120
we have found on the internet.
464
00:20:20,080 --> 00:20:23,320
And I wish I had been in the room to say,
465
00:20:24,180 --> 00:20:27,120
can anyone see a problem with this?
466
00:20:29,060 --> 00:20:29,760
Surely someone did.
467
00:20:29,900 --> 00:20:32,040
And I said, oh yeah, but it's no big deal.
468
00:20:32,740 --> 00:20:34,540
You know, the users don't correct the mistakes.
469
00:20:35,100 --> 00:20:36,820
And, you know, and anyway, it's quick.
470
00:20:37,500 --> 00:20:38,820
Yeah, it's relatively quick.
471
00:20:39,020 --> 00:20:40,800
It's, you know, we don't have figured out,
472
00:20:40,960 --> 00:20:41,760
just dump them into the internet.
473
00:20:41,000 --> 00:20:41,560
the internet.
474
00:20:41,740 --> 00:20:43,960
Yeah, they chose speed over quality.
475
00:20:44,620 --> 00:20:45,380
It's like
476
00:20:46,000 --> 00:20:48,480
teaching your child by plopping them in front of the
477
00:20:48,480 --> 00:20:49,440
television all day.
478
00:20:49,660 --> 00:20:51,980
Yeah, you'll have some hits, not a lot of misses.
479
00:20:52,940 --> 00:20:58,660
It's generative AI is based on how our brain strengths
480
00:20:58,660 --> 00:21:01,060
together a conversation, which is prediction.
481
00:21:02,260 --> 00:21:04,000
You actually alluded to it, Beth,
482
00:21:04,060 --> 00:21:06,060
when you said you're listening to me talk and you're
483
00:21:06,060 --> 00:21:06,900
already thinking,
484
00:21:07,480 --> 00:21:10,140
these are the pathways that we want to explore.
485
00:21:10,580 --> 00:21:13,440
This is how I'm going to respond when Margie gives me a
486
00:21:13,440 --> 00:21:14,520
word in edgewise.
487
00:21:15,210 --> 00:21:20,760
So they are always predicting the next word you will say
488
00:21:20,760 --> 00:21:21,940
and they will say.
489
00:21:22,440 --> 00:21:25,560
Now, a prediction is an approximation.
490
00:21:26,500 --> 00:21:28,180
It's not an absolute.
491
00:21:29,160 --> 00:21:30,280
So for example,
492
00:21:30,320 --> 00:21:34,860
I had a very interesting conversation with a new tool that
493
00:21:34,860 --> 00:21:35,220
came out.
494
00:21:35,340 --> 00:21:39,340
I won't say its name because I don't need to cast
495
00:21:39,340 --> 00:21:40,640
aspersions on it,
496
00:21:40,820 --> 00:21:43,780
but I wanted to just see what it would come up with.
497
00:21:43,780 --> 00:21:47,100
Then I asked it, can you give me several references on?
498
00:21:47,700 --> 00:21:52,920
Let's just say it was Kirkpatrick's system of measuring,
499
00:21:53,320 --> 00:21:54,440
evaluating training.
500
00:21:54,780 --> 00:21:56,880
I can't honestly remember what it was then.
501
00:21:57,220 --> 00:21:57,760
It's uncertain.
502
00:21:58,020 --> 00:22:02,560
It listed different authors and titles of articles,
503
00:22:02,560 --> 00:22:05,820
and when they were published, it didn't give me any links.
504
00:22:06,640 --> 00:22:07,060
I said,
505
00:22:07,260 --> 00:22:11,060
can you provide me links so that I can review these?
506
00:22:11,700 --> 00:22:12,700
It gave me one.
507
00:22:12,700 --> 00:22:14,340
The first source.
508
00:22:15,100 --> 00:22:15,720
I said, great.
509
00:22:16,720 --> 00:22:17,900
Why didn't you give me the rest?
510
00:22:18,580 --> 00:22:20,140
Because I can't find them.
511
00:22:20,360 --> 00:22:21,860
They are not on the Internet.
512
00:22:22,800 --> 00:22:23,480
I said, really?
513
00:22:24,080 --> 00:22:26,340
Where did your answer come from?
514
00:22:27,200 --> 00:22:28,860
It said, I am.
515
00:22:29,180 --> 00:22:29,380
Now,
516
00:22:29,440 --> 00:22:33,780
it falls back to some canned response that built into it.
517
00:22:34,260 --> 00:22:35,600
I am a large language model.
518
00:22:36,040 --> 00:22:40,180
I have been built to do my best to satisfy your request.
519
00:22:40,180 --> 00:22:44,400
If I can't find what you're looking for because you asked
520
00:22:44,400 --> 00:22:45,860
for multiple sources,
521
00:22:46,640 --> 00:22:51,520
I will predict what a correct response might look like.
522
00:22:52,600 --> 00:22:55,060
And I came back with, in other words, you made them up.
523
00:22:55,000 --> 00:22:56,560
Yeah, it's giving you,
524
00:22:56,660 --> 00:22:58,840
or somebody cited it in something they've written,
525
00:22:58,980 --> 00:23:02,120
but it hasn't actually gone to read the true article,
526
00:23:02,320 --> 00:23:03,400
so it doesn't really know.
527
00:23:04,000 --> 00:23:06,700
Not only does it not know, but when it doesn't know,
528
00:23:06,960 --> 00:23:11,080
like an over eager high school student in their first job,
529
00:23:11,100 --> 00:23:13,400
it makes stuff up.
530
00:23:13,000 --> 00:23:14,040
That's right.
531
00:23:14,240 --> 00:23:16,980
Let's go back to talking about AI inside learning
532
00:23:16,980 --> 00:23:17,500
experiences,
533
00:23:17,700 --> 00:23:20,500
because it's one thing to kind of use it to prepare and to
534
00:23:20,500 --> 00:23:22,140
write descriptions and all that.
535
00:23:22,340 --> 00:23:25,880
But when we put an AI something into a course,
536
00:23:25,880 --> 00:23:28,400
we're not there to analyze it.
537
00:23:28,580 --> 00:23:29,940
Our learners are there, aren't they?
538
00:23:30,200 --> 00:23:31,200
And they may not know.
539
00:23:31,400 --> 00:23:33,340
So how do we build the knowledge base?
540
00:23:33,560 --> 00:23:34,020
Like you talked,
541
00:23:34,120 --> 00:23:36,180
we're not going to get it to draw from the entire internet,
542
00:23:36,420 --> 00:23:36,960
are we?
543
00:23:37,120 --> 00:23:37,960
When we do that thing,
544
00:23:38,200 --> 00:23:41,380
how do we create those knowledge bases that just sit inside
545
00:23:41,380 --> 00:23:43,900
courses so we can make sure it's correct?
546
00:23:45,000 --> 00:23:46,160
That's the way to do it.
547
00:23:47,100 --> 00:23:47,140
Fortunately,
548
00:23:47,900 --> 00:23:50,640
most companies or most fields already have a knowledge
549
00:23:50,640 --> 00:23:53,740
basis, just nobody's bothering to use it.
550
00:23:54,080 --> 00:23:56,640
The search engine might be old-fashioned and clunky.
551
00:23:57,020 --> 00:23:57,960
It might be difficult.
552
00:23:58,560 --> 00:24:00,020
They might have, for example,
553
00:24:00,220 --> 00:24:01,940
frequently asked question documents.
554
00:24:02,380 --> 00:24:05,320
They might have its installation manuals.
555
00:24:05,860 --> 00:24:07,620
They might have their company internet,
556
00:24:08,140 --> 00:24:09,560
which has all been vetted.
557
00:24:10,220 --> 00:24:13,360
In theory, that could still be misinformation,
558
00:24:13,720 --> 00:24:17,000
but at least it's misinformation you created.
559
00:24:17,580 --> 00:24:19,380
It'll be easily identifiable.
560
00:24:20,300 --> 00:24:22,280
You can train.
561
00:24:22,760 --> 00:24:26,220
There are some bots that can be purchased,
562
00:24:26,780 --> 00:24:30,300
implemented inside your ecosystem,
563
00:24:30,820 --> 00:24:33,920
and told you can only look for your answer here.
564
00:24:34,980 --> 00:24:36,680
That's how you do it.
565
00:24:37,420 --> 00:24:41,380
You still have to monitor and make sure that it's pointing
566
00:24:41,380 --> 00:24:43,080
to the right sources and so on.
567
00:24:43,000 --> 00:24:43,800
So,
568
00:24:43,800 --> 00:24:49,520
these are services we basically buy and create and then can
569
00:24:49,520 --> 00:24:50,580
put inside courses.
570
00:24:50,900 --> 00:24:53,700
And are those the kinds of things that we're going to be
571
00:24:53,700 --> 00:24:56,000
paying a monthly fee forever for?
572
00:24:56,220 --> 00:24:59,360
Or are there ones where we can in the almost old school
573
00:24:59,360 --> 00:25:00,880
fashion now, you know,
574
00:25:01,100 --> 00:25:03,940
pay a lump sum and get it to happen and then put it inside
575
00:25:03,940 --> 00:25:04,380
the course?
576
00:25:04,780 --> 00:25:08,260
I work with a lot of nonprofits and there are all sorts of
577
00:25:08,260 --> 00:25:11,640
other organizations that don't really have a lot of money
578
00:25:11,640 --> 00:25:12,660
for this kind of stuff.
579
00:25:12,660 --> 00:25:16,240
Like, is it going to be expensive and ongoing fees?
580
00:25:16,000 --> 00:25:19,840
You know, the answer is sort of yes and no, like anything,
581
00:25:20,020 --> 00:25:23,440
it depends on your due diligence and the more educated you
582
00:25:23,440 --> 00:25:27,060
are as a buyer, the better you are at finding solutions.
583
00:25:27,300 --> 00:25:28,240
Now, there's a benefit.
584
00:25:28,560 --> 00:25:30,520
One reason why, for example,
585
00:25:30,840 --> 00:25:32,760
authoring tools went to a subscription model,
586
00:25:33,220 --> 00:25:34,960
one was certainly it's more money for them.
587
00:25:35,480 --> 00:25:38,600
The other is they can always be putting out their best
588
00:25:38,600 --> 00:25:42,080
product because subscribers automatically get the updates.
589
00:25:42,860 --> 00:25:47,500
And with AI, it's changing every day.
590
00:25:47,760 --> 00:25:49,900
Something new is coming out, some new capabilities.
591
00:25:50,400 --> 00:25:55,180
So if you are subscribing to a reliable source,
592
00:25:55,860 --> 00:25:59,680
you're going to get the benefit of those upgrades as they
593
00:25:59,680 --> 00:26:00,120
happen.
594
00:26:00,980 --> 00:26:04,140
Users are also continuing to train.
595
00:26:04,720 --> 00:26:06,660
For the example I just gave, for example,
596
00:26:07,300 --> 00:26:11,020
I objected to information that's made up.
597
00:26:11,020 --> 00:26:15,140
And I said I would prefer the next time you talk to me,
598
00:26:15,620 --> 00:26:17,760
please say I can only find one source.
599
00:26:18,160 --> 00:26:19,380
Don't make up more.
600
00:26:20,600 --> 00:26:22,640
And I will continue to guide you.
601
00:26:22,780 --> 00:26:25,740
I will suggest other ways you may be able to find the
602
00:26:25,740 --> 00:26:28,280
information because I know how you've been trained.
603
00:26:29,180 --> 00:26:31,660
And it said that's a good idea.
604
00:26:31,800 --> 00:26:32,560
I will do that.
605
00:26:33,000 --> 00:26:36,020
And I've made a record of this suggestion for further
606
00:26:36,020 --> 00:26:36,680
improvements.
607
00:26:37,000 --> 00:26:39,420
And does it remember that because I've had a client and
608
00:26:39,420 --> 00:26:41,180
I've experienced this myself say, well,
609
00:26:41,300 --> 00:26:46,020
I've told it five times not to do X and it won't do it for
610
00:26:46,020 --> 00:26:47,700
a while, but then it'll start to do it again.
611
00:26:47,860 --> 00:26:50,340
And then you have to say, no, remember,
612
00:26:50,720 --> 00:26:52,000
I told you not to do this.
613
00:26:52,180 --> 00:26:54,420
So do you have to keep reminding it the virtual assistant
614
00:26:54,420 --> 00:26:54,960
you've got?
615
00:26:55,000 --> 00:26:56,920
Um, I asked it back to, and I said,
616
00:26:57,000 --> 00:26:58,220
so you're going to remember it.
617
00:26:58,560 --> 00:26:59,360
And it said, well,
618
00:27:00,020 --> 00:27:02,260
if you purchase the premium package of me,
619
00:27:03,340 --> 00:27:06,420
otherwise I may remember it for a while, but you may.
620
00:27:07,000 --> 00:27:10,280
to remind me so they're building this in so that you kind
621
00:27:10,280 --> 00:27:13,680
of have to sign up and then yeah but you can't set it and
622
00:27:13,680 --> 00:27:16,660
forget it because it's changing so fast as you said yeah so
623
00:27:16,660 --> 00:27:16,980
it's dangerous.
624
00:27:17,000 --> 00:27:17,360
625
00:27:17,600 --> 00:27:21,040
And wouldn't your human employee be the same way?
626
00:27:21,000 --> 00:27:22,520
Well, we all have to keep learning.
627
00:27:22,680 --> 00:27:24,400
I mean, human or AI, don't we?
628
00:27:24,620 --> 00:27:26,680
So we have to make space for that,
629
00:27:26,740 --> 00:27:28,220
although humans can learn,
630
00:27:28,720 --> 00:27:31,180
and it doesn't necessarily cost us any money.
631
00:27:31,840 --> 00:27:34,200
But AI has to keep learning and it does cost us money
632
00:27:34,200 --> 00:27:35,960
because that's like a business model, isn't it?
633
00:27:36,160 --> 00:27:36,820
So we're sort of stuck.
634
00:27:36,960 --> 00:27:36,960
Yeah.
635
00:27:38,000 --> 00:27:40,240
You might be paid by tokens, for example.
636
00:27:40,820 --> 00:27:46,000
I use a lot of free or very low cost tools because I don't
637
00:27:46,000 --> 00:27:49,040
necessarily see any added benefit in the more expensive
638
00:27:49,040 --> 00:27:49,400
ones.
639
00:27:49,000 --> 00:27:50,200
Can you give us just one?
640
00:27:50,380 --> 00:27:50,440
I mean,
641
00:27:50,500 --> 00:27:52,480
I know I don't want to give you ask you to give away the
642
00:27:52,480 --> 00:27:54,940
farm because you have a course around this and I feel like
643
00:27:54,940 --> 00:27:55,700
I'm going to sign up.
644
00:27:56,880 --> 00:27:59,840
But, you know, what's one that you think people could try?
645
00:28:01,000 --> 00:28:04,400
I think you should look at Eleven Labs if you have to do a lot
646
00:28:04,400 --> 00:28:05,060
of voiceovers.
647
00:28:05,880 --> 00:28:08,840
One of the things that I did a few months ago,
648
00:28:08,880 --> 00:28:11,200
and I just love it, and I've used it, by the way,
649
00:28:11,220 --> 00:28:15,380
you'll be able to hear my AI-generated voice in my course,
650
00:28:15,760 --> 00:28:20,240
is I taught it over the course of a few hours to clone my
651
00:28:20,240 --> 00:28:20,600
voice.
652
00:28:21,060 --> 00:28:22,400
Now, it's a perfect,
653
00:28:22,820 --> 00:28:26,280
originally it was me without the imperfections in a human.
654
00:28:27,000 --> 00:28:29,660
Drives me crazy how many hours it takes for me to get a
655
00:28:29,660 --> 00:28:32,340
perfect recording that I'm happy with.
656
00:28:32,800 --> 00:28:34,400
And so my next prompt was,
657
00:28:34,820 --> 00:28:40,800
now go back and add in some of those imperfections the way
658
00:28:40,800 --> 00:28:44,700
I take a breath and, you know, where my pausing is,
659
00:28:45,000 --> 00:28:46,420
and that made it much more realistic.
660
00:28:47,000 --> 00:28:47,360
Wow,
661
00:28:47,460 --> 00:28:49,920
I love that because I wrote about I learned about the concept
662
00:28:49,920 --> 00:28:51,260
called the uncanny valley.
663
00:28:51,880 --> 00:28:54,980
And there's a software that a client was asking me to use
664
00:28:54,980 --> 00:28:58,240
that I just hate it like you, I will not name what it is.
665
00:28:58,640 --> 00:28:59,980
But I just thought, God,
666
00:29:00,200 --> 00:29:03,220
this feels so awful to watch this narrator.
667
00:29:03,740 --> 00:29:04,320
And you know,
668
00:29:04,600 --> 00:29:07,480
its eyes are not, its mouth isn't matched up with what it's
669
00:29:07,480 --> 00:29:07,760
saying.
670
00:29:07,900 --> 00:29:10,980
And so you're saying you can you can code it basically,
671
00:29:10,980 --> 00:29:12,940
you can ask it to be more human.
672
00:29:13,100 --> 00:29:13,220
Yeah,
673
00:29:13,000 --> 00:29:14,520
a chance more human.
674
00:29:14,960 --> 00:29:17,920
That takes work and some understanding.
675
00:29:18,220 --> 00:29:18,440
But yes,
676
00:29:18,860 --> 00:29:22,720
the uncanny valley is going back to neuroscience.
677
00:29:22,720 --> 00:29:25,520
what happens in the brain when there's a disconnect is,
678
00:29:25,660 --> 00:29:29,080
okay, this is neat, but ooh, the creepy effect.
679
00:29:29,000 --> 00:29:32,140
is what I call it. It is, it's creepy and I and I thought oh
680
00:29:32,140 --> 00:29:34,260
there's a term for this, that's so cool. I didn't know that
681
00:29:34,260 --> 00:29:37,160
that feeling that we have when we watch some of those tools
682
00:29:37,160 --> 00:29:41,380
that's what it is. Yeah it was such a neat term to learn.
683
00:29:42,000 --> 00:29:45,880
And imagine if it was actually something you consider you,
684
00:29:46,080 --> 00:29:49,000
you identify that's my voice, but not quite.
685
00:29:49,140 --> 00:29:51,380
Yeah, the creepiness that would feel.
686
00:29:51,000 --> 00:29:53,940
Are you testing that with people? Like people that know you
687
00:29:53,940 --> 00:29:56,140
and have talked to you a lot and you send it to them and
688
00:29:56,140 --> 00:29:57,100
say, Is this really me?
689
00:29:57,220 --> 00:29:58,220
Does this sound like me?
690
00:29:59,000 --> 00:29:59,180
Yeah,
691
00:29:59,240 --> 00:30:02,580
I sent it out to a lot of people in my network and they
692
00:30:02,580 --> 00:30:03,660
came back, oh, this is great.
693
00:30:03,860 --> 00:30:05,760
Not one said, this is great.
694
00:30:05,780 --> 00:30:07,060
Was this an A.I.?
695
00:30:07,060 --> 00:30:08,540
Which is the response I was
696
00:30:08,000 --> 00:30:08,420
looking for.
697
00:30:08,480 --> 00:30:09,420
Oh, you didn't tell them.
698
00:30:09,520 --> 00:30:10,660
You just said, how are these videos?
699
00:30:12,000 --> 00:30:12,180
Yeah.
700
00:30:13,360 --> 00:30:16,900
My old dog thinks that's mommy talking when I'm playing those
701
00:30:16,900 --> 00:30:19,160
videos, so it's a very good likeness.
702
00:30:20,420 --> 00:30:22,440
I have not been as successful yet.
703
00:30:22,620 --> 00:30:23,920
I have tried and have not found,
704
00:30:24,000 --> 00:30:26,700
you mentioned the software that, you know, can make, say,
705
00:30:26,740 --> 00:30:28,200
a static photograph talk.
706
00:30:28,820 --> 00:30:30,180
You see a lot of them on your phone,
707
00:30:30,300 --> 00:30:31,420
or I do get ads for it.
708
00:30:31,460 --> 00:30:33,880
They have little kids singing a song.
709
00:30:34,520 --> 00:30:35,940
Very effective for a little,
710
00:30:36,080 --> 00:30:38,120
if you're only going to use it for 30 seconds,
711
00:30:38,120 --> 00:30:39,920
it might look really good.
712
00:30:39,920 --> 00:30:43,660
But the longer you watch it, the creepier it gets.
713
00:30:44,660 --> 00:30:46,860
Now, as learning and development professionals,
714
00:30:47,120 --> 00:30:50,780
we have a responsibility to protect our learners as well,
715
00:30:50,920 --> 00:30:54,440
and our organizations, as well as deliver content.
716
00:30:54,820 --> 00:30:58,120
And so we need to think about some of the psychological
717
00:30:58,120 --> 00:31:00,440
impact of working with an AI.
718
00:31:00,820 --> 00:31:03,060
And the human race is not ready yet.
719
00:31:03,060 --> 00:31:07,480
So another thing that we deal with in the course,
720
00:31:07,480 --> 00:31:11,960
but also just in life, is we need to face those things.
721
00:31:12,120 --> 00:31:14,180
And if we're not the voice of that concern,
722
00:31:14,940 --> 00:31:17,180
very often someone higher up who doesn't have our
723
00:31:17,180 --> 00:31:21,240
background, or necessarily our same concerns, says, no,
724
00:31:21,340 --> 00:31:22,560
just put it out, I don't care.
725
00:31:23,220 --> 00:31:25,040
Now all kinds of people turned off.
726
00:31:25,540 --> 00:31:27,800
If you have the uncanny valley effect,
727
00:31:28,080 --> 00:31:29,520
you're no longer learning,
728
00:31:29,760 --> 00:31:32,320
you're too upset by what you've just experienced.
729
00:31:32,000 --> 00:31:35,140
That was what I was saying to my client, frankly,
730
00:31:35,460 --> 00:31:39,160
and they overruled me and so it makes me think that,
731
00:31:39,340 --> 00:31:42,400
you know, there's it's always back to boundaries, right,
732
00:31:42,580 --> 00:31:45,200
sometimes in business it's like what as learning and
733
00:31:45,200 --> 00:31:46,020
development professionals,
734
00:31:46,320 --> 00:31:50,280
are we going to allow to happen with our participation in
735
00:31:50,280 --> 00:31:50,500
it.
736
00:31:50,920 --> 00:31:54,240
And so in the future, unless it gets even better,
737
00:31:54,240 --> 00:31:55,420
like you're talking about,
738
00:31:55,500 --> 00:31:58,920
and I feel it's not going to impact the learners being able
739
00:31:58,920 --> 00:32:02,220
to learn, which is our ultimate goal and, you know,
740
00:32:02,320 --> 00:32:04,720
change behaviour and whatever, I don't want to do it.
741
00:32:04,860 --> 00:32:06,420
I don't want to be part of that.
742
00:32:06,920 --> 00:32:09,580
So maybe we have to hold our feet to the fire, don't we,
743
00:32:09,700 --> 00:32:12,240
to say yay or nay to these kinds of things.
744
00:32:12,260 --> 00:32:14,580
So we still need to look at them so critically.
745
00:32:15,040 --> 00:32:16,920
It's not just the look cool factor, is it?
746
00:32:17,000 --> 00:32:17,740
Absolutely.
747
00:32:18,460 --> 00:32:19,660
And that's what's happening right now.
748
00:32:19,780 --> 00:32:23,180
I think a lot of people are ever saying, I need, I need AI,
749
00:32:23,400 --> 00:32:25,560
you go out and you figure it out and you put in AI.
750
00:32:26,360 --> 00:32:28,840
And that is just like a decade ago, we got,
751
00:32:28,840 --> 00:32:29,580
I need e-learning.
752
00:32:30,100 --> 00:32:32,260
And so we got a lot of horrible e-learning produced,
753
00:32:32,400 --> 00:32:32,460
right?
754
00:32:33,000 --> 00:32:36,340
Yes we go back to why why do you want that and how is it
755
00:32:36,340 --> 00:32:38,880
gonna serve the learner and our ultimate purpose of why
756
00:32:38,880 --> 00:32:39,760
we're doing this, right?
757
00:32:40,000 --> 00:32:43,260
So what's the benefit and specifically what do you mean by
758
00:32:43,260 --> 00:32:44,640
that which they usually can't answer,
759
00:32:45,280 --> 00:32:48,500
then if they still insist, we should do this anyway,
760
00:32:48,840 --> 00:32:49,840
always have a pilot.
761
00:32:50,320 --> 00:32:51,520
So let's have a pilot.
762
00:32:52,160 --> 00:32:55,780
And in the case you're describing the pilot group would
763
00:32:55,780 --> 00:32:59,240
have given that decision maker feedback,
764
00:32:59,260 --> 00:33:01,360
they might not have taken from you.
765
00:33:02,000 --> 00:33:02,660
Yeah, we'll see.
766
00:33:02,820 --> 00:33:05,200
We haven't gotten that far yet, but yeah,
767
00:33:05,600 --> 00:33:06,960
I'm hoping for that.
768
00:33:07,840 --> 00:33:10,320
I'm so glad you brought it back to neuroscience and made
769
00:33:10,320 --> 00:33:12,260
that nice little link, you know,
770
00:33:12,440 --> 00:33:14,960
back to where we started and to come back to that and go,
771
00:33:15,080 --> 00:33:16,500
you know, why are we doing this anyway?
772
00:33:16,800 --> 00:33:19,600
And it's to serve the learner and how the learner's brain
773
00:33:19,600 --> 00:33:22,860
works and if it's working against their actual learning by
774
00:33:22,860 --> 00:33:25,540
the uncanny valley or whatever, we have a problem.
775
00:33:26,000 --> 00:33:28,660
One of the great things these chatbots do and why I
776
00:33:28,660 --> 00:33:32,080
recommend using it in any kind of learning is that the
777
00:33:32,080 --> 00:33:35,120
human brain is pre-programmed for social engagement.
778
00:33:35,400 --> 00:33:36,560
It's our survival scale.
779
00:33:37,020 --> 00:33:38,560
When we were evolving as a species,
780
00:33:38,640 --> 00:33:40,980
we were not the biggest or the strongest or the fastest,
781
00:33:41,680 --> 00:33:44,640
and some scientists think we weren't even the smartest.
782
00:33:45,080 --> 00:33:48,560
There were other humanoid species that did not survive that
783
00:33:48,560 --> 00:33:49,480
were smaller than us,
784
00:33:49,720 --> 00:33:52,620
but they were not social as social as us.
785
00:33:52,940 --> 00:33:54,300
It is banding together.
786
00:33:54,620 --> 00:33:57,720
It is interconnection and learning from each other.
787
00:33:58,020 --> 00:33:59,220
That is our superpower.
788
00:33:59,720 --> 00:34:03,120
You don't have enough instructors in the world to give
789
00:34:03,120 --> 00:34:04,960
everyone one-to-one attention.
790
00:34:05,260 --> 00:34:07,160
That's what I meant by personalized learning.
791
00:34:07,660 --> 00:34:11,760
Your brain does not know this is an artificial creation,
792
00:34:12,139 --> 00:34:15,440
and if it avoids the uncanny valley,
793
00:34:15,920 --> 00:34:19,420
it responds to it as though another person is having that
794
00:34:19,420 --> 00:34:19,860
engagement.
795
00:34:20,500 --> 00:34:24,340
Much more of the brain lights up in the fMRI.
796
00:34:25,060 --> 00:34:26,580
Much more of the brain is engaged.
797
00:34:26,880 --> 00:34:29,760
That means more neurons are part of the neural chain.
798
00:34:29,980 --> 00:34:31,440
That means a stronger memory.
799
00:34:31,600 --> 00:34:33,880
That means it connects with other parts of their
800
00:34:33,880 --> 00:34:34,580
experience.
801
00:34:35,280 --> 00:34:39,100
Every part of the learning experience is deeper and more
802
00:34:39,100 --> 00:34:44,639
powerful and retained longer and applied better when
803
00:34:44,639 --> 00:34:46,940
conversation, authentic,
804
00:34:46,940 --> 00:34:50,880
relevant conversation is a part of that experience,
805
00:34:51,420 --> 00:34:55,560
and that's something you can do with various forms of ABI.
806
00:34:55,920 --> 00:34:58,400
The shorthand is chatbot, but there's other ways too.
807
00:34:58,560 --> 00:35:00,320
And the personalized learning piece,
808
00:35:00,460 --> 00:35:02,380
I've seen you write about how it's good for equity,
809
00:35:02,460 --> 00:35:03,320
diversity, and inclusion,
810
00:35:03,320 --> 00:35:07,620
because if the AI can kind of get to know the learner
811
00:35:07,620 --> 00:35:10,320
and who they are in their all their intersectionalities,
812
00:35:10,820 --> 00:35:13,800
then maybe it can respond to them in ways that they
813
00:35:13,800 --> 00:35:17,540
particularly need and then we can be more inclusive in the
814
00:35:17,540 --> 00:35:19,000
learning experience or for them.
815
00:35:19,000 --> 00:35:19,600
Great example,
816
00:35:19,840 --> 00:35:22,720
it's another wonderful use case we're not doing enough of.
817
00:35:23,000 --> 00:35:28,080
The flip side of that is you can also have that experience
818
00:35:29,460 --> 00:35:34,240
listening to and pointing out unconscious bias and nudging
819
00:35:34,240 --> 00:35:35,620
people towards,
820
00:35:35,640 --> 00:35:38,700
which is another neuroscience concept of nudge learning,
821
00:35:39,200 --> 00:35:43,340
nudging people gradually to behaviour change with each
822
00:35:43,340 --> 00:35:43,940
interaction.
823
00:35:44,520 --> 00:35:46,940
So it's not just a matter of don't say that anymore.
824
00:35:47,470 --> 00:35:49,340
People can learn how to play that game,
825
00:35:49,500 --> 00:35:50,820
but their mind is still thinking.
826
00:35:51,000 --> 00:35:51,580
Right.
827
00:35:51,900 --> 00:35:54,660
Yes, you lead them down the path towards something.
828
00:35:54,700 --> 00:35:56,200
Yeah, that you want them to change.
829
00:35:57,000 --> 00:36:00,340
to really reprogram those brains takes time.
830
00:36:00,700 --> 00:36:03,220
And we all have these unconscious biases.
831
00:36:03,840 --> 00:36:08,120
It's been a very difficult thing in our society to try and
832
00:36:08,120 --> 00:36:10,400
change through traditional training methods.
833
00:36:10,860 --> 00:36:13,680
However, I believe artificial intelligence can really help.
834
00:36:14,000 --> 00:36:16,660
But we come back to who does the coding, I suppose, right?
835
00:36:16,840 --> 00:36:18,120
Who builds the knowledge base?
836
00:36:18,360 --> 00:36:20,460
And so would you also recommend, boy,
837
00:36:20,660 --> 00:36:22,420
I hate when I ask leading questions.
838
00:36:23,280 --> 00:36:24,780
It's not a facilitator skill.
839
00:36:24,960 --> 00:36:27,860
It's like I want to say we want to make sure that the
840
00:36:27,860 --> 00:36:30,960
people who build the knowledge bases for the AI to draw
841
00:36:30,960 --> 00:36:33,080
from have to be diverse, don't they?
842
00:36:33,260 --> 00:36:35,480
Because if we only get one subset of the population,
843
00:36:36,240 --> 00:36:37,520
then what does it give us?
844
00:36:37,640 --> 00:36:37,780
It does.
845
00:36:37,980 --> 00:36:40,240
It doesn't give us what we need for every single learner.
846
00:36:40,240 --> 00:36:40,700
That's right.
847
00:36:40,000 --> 00:36:43,240
Now another great thing is making sure that all the
848
00:36:43,240 --> 00:36:46,620
learners are giving feedback that part of that experience
849
00:36:46,620 --> 00:36:49,600
is just like I explained with my artificial assistant,
850
00:36:50,240 --> 00:36:52,900
and with the one that I tested out and I said no that's not
851
00:36:52,900 --> 00:36:55,420
what I want is they have to be ready to say that.
852
00:36:56,000 --> 00:36:56,820
Yeah.
853
00:36:57,160 --> 00:36:58,560
And so learners have to be taught.
854
00:36:59,020 --> 00:36:59,140
Actually,
855
00:36:59,180 --> 00:37:01,400
we talked about this in the episode with Taruna Goel
856
00:37:01,400 --> 00:37:02,500
where, you know,
857
00:37:02,560 --> 00:37:04,880
the more we teach learners how to advocate for themselves
858
00:37:04,880 --> 00:37:05,700
and what they need.
859
00:37:05,940 --> 00:37:09,080
So we have to teach all learners that they have to give
860
00:37:09,080 --> 00:37:11,780
feedback or else it won't keep getting better for them and
861
00:37:11,780 --> 00:37:12,760
future learners too.
862
00:37:13,000 --> 00:37:14,040
Well, as a matter of fact,
863
00:37:14,100 --> 00:37:17,840
it will be actually getting worse because these large ion
864
00:37:17,840 --> 00:37:19,340
-reach models, there's only half.
865
00:37:19,560 --> 00:37:22,360
Let's just picture where everyone's using ChatGPT.
866
00:37:22,600 --> 00:37:25,160
That is almost true because it's so popular.
867
00:37:25,620 --> 00:37:27,320
And there's some consistent mistake.
868
00:37:27,620 --> 00:37:29,240
It keeps making a no one correct say.
869
00:37:30,040 --> 00:37:35,800
You're only making it stronger in the neural network of
870
00:37:35,800 --> 00:37:36,380
ChatGPT.
871
00:37:37,000 --> 00:37:39,980
Yeah, so you have to respond to it like
872
00:37:40,000 --> 00:37:40,680
a human being.
873
00:37:41,900 --> 00:37:47,040
So we have a responsibility, just like there are children,
874
00:37:47,380 --> 00:37:49,620
but that doesn't mean I think they are children.
875
00:37:50,420 --> 00:37:52,440
You know, we're not there yet.
876
00:37:52,780 --> 00:37:54,800
And sometimes people ask me about sentience,
877
00:37:54,880 --> 00:37:57,660
how far away are we these things are so phenomenal,
878
00:37:57,660 --> 00:38:02,080
they feel like they're really thinking and feeling all the
879
00:38:02,080 --> 00:38:03,920
research there's no, we're not there.
880
00:38:04,100 --> 00:38:05,220
They weren't built for that.
881
00:38:05,240 --> 00:38:07,440
But they're doing other things they weren't built for.
882
00:38:07,860 --> 00:38:09,280
ChatGPT was built for language,
883
00:38:09,960 --> 00:38:12,740
and no one expected it to be so good at computer code,
884
00:38:13,100 --> 00:38:15,160
computer programming, well, code is a language.
885
00:38:16,140 --> 00:38:19,160
And now we have different models that are creating music,
886
00:38:19,340 --> 00:38:22,560
well, music is a language, math is a language.
887
00:38:22,720 --> 00:38:24,420
So people didn't think of that.
888
00:38:24,820 --> 00:38:27,100
And suddenly these capabilities came out and everyone's
889
00:38:27,100 --> 00:38:30,100
like, whoa, well, sooner or later,
890
00:38:30,100 --> 00:38:32,120
it is going to sneak up on us.
891
00:38:32,260 --> 00:38:34,260
I don't know if it's going to be our lifetime.
892
00:38:34,260 --> 00:38:36,780
But some very smart people think it will be.
893
00:38:37,820 --> 00:38:41,260
So we also need to keep an eye on what does it mean in the
894
00:38:41,260 --> 00:38:44,660
workplace to be engaging with these tools,
895
00:38:45,200 --> 00:38:50,240
and whether we prepare our human workers for interacting.
896
00:38:50,480 --> 00:38:55,340
And one of those things is everybody becomes a monitor of
897
00:38:55,340 --> 00:38:58,440
the AI tool they're using, rather than accepting it.
898
00:38:58,500 --> 00:39:00,520
That's one thing we have to teach people,
899
00:39:00,520 --> 00:39:03,620
and we have to teach them to treat it well.
900
00:39:04,660 --> 00:39:07,840
Studies show that they work better when you're polite to
901
00:39:07,840 --> 00:39:11,940
them, build that into it, but saying please and thank you,
902
00:39:12,320 --> 00:39:15,320
and not calling them idiots and so on.
903
00:39:15,460 --> 00:39:18,360
Actually, just like people seems to work,
904
00:39:18,600 --> 00:39:21,680
why they studied us, they respond like we do.
905
00:39:22,360 --> 00:39:25,120
So it's not because they really feel it.
906
00:39:25,320 --> 00:39:30,780
But they have an algorithm that says politeness is
907
00:39:30,780 --> 00:39:33,760
desirable in this interaction.
908
00:39:34,900 --> 00:39:34,900
Interesting.
909
00:39:35,000 --> 00:39:38,440
So what's one thing that you want everybody to correct it
910
00:39:38,440 --> 00:39:40,600
because I have a thing that's coming to my mind and I'll
911
00:39:40,600 --> 00:39:42,860
say it maybe after you listen to the same thing.
912
00:39:43,240 --> 00:39:45,740
What's the thing that you always see it doing that it's
913
00:39:45,740 --> 00:39:47,800
wrong wrong wrong because it's getting it from the
914
00:39:47,800 --> 00:39:48,120
internet?
915
00:39:48,320 --> 00:39:50,140
Is there something that's coming to mind for you?
916
00:39:51,000 --> 00:39:54,160
Well, the general one would be making things up.
917
00:39:54,400 --> 00:39:57,840
I always call it on hallucinations and probe it a little
918
00:39:57,840 --> 00:39:58,120
back.
919
00:39:58,500 --> 00:39:59,300
How did that happen?
920
00:39:59,700 --> 00:40:00,660
Why did you say that?
921
00:40:00,780 --> 00:40:03,640
Because I cannot verify it.
922
00:40:03,880 --> 00:40:07,880
Another thing that I think is a real danger is that
923
00:40:07,880 --> 00:40:08,940
everybody's writing,
924
00:40:09,020 --> 00:40:11,840
if we're all writing a ChatGPT or something else,
925
00:40:12,300 --> 00:40:14,860
our writing might all start to look the same and sound the
926
00:40:14,860 --> 00:40:15,040
same,
927
00:40:15,160 --> 00:40:18,760
even if it's grammatically correct and factually correct.
928
00:40:19,300 --> 00:40:20,960
You know, we may have lost the creativity,
929
00:40:20,960 --> 00:40:23,500
but it is possible, just like I do with my voice,
930
00:40:23,500 --> 00:40:28,260
to upload samples of your writing and have it analyzed and
931
00:40:28,260 --> 00:40:30,220
then say, now I want you to write like that.
932
00:40:30,740 --> 00:40:33,520
For example, and you probably won't be surprised by this,
933
00:40:34,180 --> 00:40:36,160
I went through that with Claude and it said,
934
00:40:36,220 --> 00:40:38,900
your writing is really very good, give me all this stuff,
935
00:40:39,080 --> 00:40:41,060
except that your sentences are very long.
936
00:40:41,760 --> 00:40:43,480
Okay, I called you out a little bit.
937
00:40:44,000 --> 00:40:45,700
And I said, that's fine, I want to keep,
938
00:40:45,820 --> 00:40:46,780
I appreciate feedback,
939
00:40:47,260 --> 00:40:49,520
I want to keep that aspect because that's me.
940
00:40:49,560 --> 00:40:49,960
Oh, okay.
941
00:40:50,000 --> 00:40:50,640
Yeah.
942
00:40:51,020 --> 00:40:52,020
I don't want to correct.
943
00:40:52,300 --> 00:40:52,320
Yeah.
944
00:40:52,320 --> 00:40:53,720
Cause you want to be a little, you know,
945
00:40:53,860 --> 00:40:55,020
like we're fallible, right?
946
00:40:55,100 --> 00:40:56,500
And that makes us human too.
947
00:40:56,600 --> 00:40:57,280
Doesn't it?
948
00:40:57,380 --> 00:40:58,480
Oh, I, that's so neat.
949
00:40:58,780 --> 00:41:00,520
I like how you were polite back to it.
950
00:41:00,800 --> 00:41:02,500
Thank you very much for the feedback,
951
00:41:02,500 --> 00:41:04,880
but I would like to keep that.
952
00:41:04,940 --> 00:41:05,340
Yeah.
953
00:41:05,680 --> 00:41:07,240
I was thinking about a content area.
954
00:41:07,440 --> 00:41:09,560
I really wish it would stop talking about learning styles.
955
00:41:09,940 --> 00:41:11,520
Learning styles don't exist.
956
00:41:12,100 --> 00:41:14,080
Can we just get it to talk about learning preferences,
957
00:41:14,460 --> 00:41:14,520
please?
958
00:41:14,720 --> 00:41:17,460
You know, just, that's a conversation I tend to have it,
959
00:41:17,540 --> 00:41:17,840
you know,
960
00:41:17,960 --> 00:41:21,280
do not give me anything that says learning styles in your
961
00:41:21,280 --> 00:41:21,640
response.
962
00:41:22,100 --> 00:41:24,440
And sometimes it remembers sometimes it doesn't.
963
00:41:25,000 --> 00:41:28,200
a simple prompt you can give at the beginning of any
964
00:41:28,200 --> 00:41:28,740
request.
965
00:41:29,300 --> 00:41:33,760
You can always say exclude or do not give me anything in
966
00:41:33,760 --> 00:41:34,300
this area.
967
00:41:34,700 --> 00:41:37,400
We could do another whole podcast on learning styles,
968
00:41:37,400 --> 00:41:42,040
but it circles back to neuroscience because our brains are
969
00:41:42,040 --> 00:41:43,620
ultimately lazy.
970
00:41:44,360 --> 00:41:46,680
They are programmed, if you will,
971
00:41:46,780 --> 00:41:50,440
to seek the shortest path to a solution because doing a lot
972
00:41:50,440 --> 00:41:51,720
of things to keep us alive.
973
00:41:52,760 --> 00:41:57,300
So one of the things that your brain will do is it will
974
00:41:58,520 --> 00:42:01,520
over summarize any complex idea.
975
00:42:01,880 --> 00:42:03,560
And that's what happened to learning styles.
976
00:42:04,160 --> 00:42:05,080
It was written,
977
00:42:05,420 --> 00:42:07,460
the original research was just as you said it,
978
00:42:07,540 --> 00:42:08,360
learning preferences.
979
00:42:09,100 --> 00:42:12,540
Now what that tells us is everybody's brain has a preferred
980
00:42:12,540 --> 00:42:13,520
method of learning.
981
00:42:13,780 --> 00:42:16,480
Most of us are visual because there's so much more
982
00:42:16,480 --> 00:42:18,160
information that comes in that way.
983
00:42:18,480 --> 00:42:20,880
That's because it's easier to learn that way.
984
00:42:21,140 --> 00:42:25,500
Now infer, if you really want to force that brain to work,
985
00:42:25,660 --> 00:42:29,900
you need to make it difficult or more challenging,
986
00:42:30,400 --> 00:42:33,120
which means rather than, oh, that's your learning style,
987
00:42:33,280 --> 00:42:33,740
here you go.
988
00:42:33,860 --> 00:42:35,480
Everything you get is in that style.
989
00:42:35,940 --> 00:42:37,200
It should be the opposite.
990
00:42:37,760 --> 00:42:40,880
It should be, oh, let's try some audio content.
991
00:42:40,880 --> 00:42:43,700
Let's get up and do something physical.
992
00:42:44,700 --> 00:42:47,020
If you can figure out a way to work, smell into it,
993
00:42:47,340 --> 00:42:48,820
you know, that's pretty narrow domain.
994
00:42:49,660 --> 00:42:53,660
And you lose all that nuance with, and quite frankly,
995
00:42:53,700 --> 00:42:54,720
it's just a money thing.
996
00:42:54,960 --> 00:42:57,900
There's a cottage industry that grew up selling instruments
997
00:42:57,900 --> 00:43:01,940
and methodologies, and they're still doing very well.
998
00:43:02,140 --> 00:43:03,720
A lot of people have drunk the Kool -Aid,
999
00:43:03,980 --> 00:43:07,860
that doesn't make it right or correct based on the
1000
00:43:07,860 --> 00:43:08,280
research.
1001
00:43:08,540 --> 00:43:10,220
So if you go back to the science,
1002
00:43:10,220 --> 00:43:14,340
learning styles has been so widely debunked.
1003
00:43:15,220 --> 00:43:18,740
It is shocking that it is still brought up in a
1004
00:43:18,740 --> 00:43:19,140
conversation,
1005
00:43:19,500 --> 00:43:22,380
in a serious conversation about learning in the state and
1006
00:43:22,380 --> 00:43:22,700
age.
1007
00:43:23,120 --> 00:43:24,940
Maybe that's something we could build about,
1008
00:43:24,940 --> 00:43:27,800
that every time someone said it, they got a little zoned.
1009
00:43:28,000 --> 00:43:30,440
that little buzz on the keyboard.
1010
00:43:30,980 --> 00:43:33,640
There probably are a lot of GPTs or other, you know,
1011
00:43:33,660 --> 00:43:35,000
things like that that are about learning.
1012
00:43:35,160 --> 00:43:37,080
So let's hope they are out there because everybody could
1013
00:43:37,080 --> 00:43:38,820
try to find one for sure.
1014
00:43:39,160 --> 00:43:41,320
But thanks for bringing us back to the research because
1015
00:43:41,320 --> 00:43:46,000
humans do research and they do studies and they do meta
1016
00:43:46,000 --> 00:43:49,440
-analyses of studies and on and on, right?
1017
00:43:49,540 --> 00:43:52,360
The research is there in so many ways about learning and
1018
00:43:52,360 --> 00:43:52,760
development.
1019
00:43:53,180 --> 00:43:58,500
And so we need to work with AI tools to keep pushing it
1020
00:43:58,500 --> 00:44:01,500
back to what we've learned officially.
1021
00:44:02,280 --> 00:44:05,380
And that has been proven about neuroscience, about,
1022
00:44:05,380 --> 00:44:08,360
you know, learning how it works, about the brain and so on.
1023
00:44:08,640 --> 00:44:11,080
So yeah, we do have a big responsibility here, don't we,
1024
00:44:11,280 --> 00:44:12,940
as humans working with these tools.
1025
00:44:13,000 --> 00:44:13,680
Well, yes.
1026
00:44:13,820 --> 00:44:15,200
And that's what keeps us employed.
1027
00:44:15,200 --> 00:44:15,940
Yeah, exactly.
1028
00:44:16,200 --> 00:44:16,260
You know,
1029
00:44:16,300 --> 00:44:18,820
there are already companies that have come to the conclusion,
1030
00:44:18,980 --> 00:44:22,040
maybe we don't need all these training people because we
1031
00:44:22,040 --> 00:44:23,140
have these AI tools.
1032
00:44:23,720 --> 00:44:27,860
While that is a failure of logic from the humans who made
1033
00:44:27,860 --> 00:44:28,400
that decision,
1034
00:44:28,660 --> 00:44:32,220
it was a failure of imagination for the employees who
1035
00:44:32,220 --> 00:44:37,820
weren't finding a way to prove their value as a fluid user
1036
00:44:37,820 --> 00:44:39,140
of those tools.
1037
00:44:39,300 --> 00:44:40,740
And that's where our future is.
1038
00:44:40,920 --> 00:44:42,480
That's actually where our prison is,
1039
00:44:42,660 --> 00:44:45,060
which is many of us are not caught up to it yet.
1040
00:44:45,400 --> 00:44:46,760
That's how we stay relevant.
1041
00:44:46,880 --> 00:44:50,040
Just like one other tools came out, we adapted,
1042
00:44:50,340 --> 00:44:51,220
we learned them,
1043
00:44:51,220 --> 00:44:54,760
or we didn't have what the next employer was looking for.
1044
00:44:55,140 --> 00:44:58,820
So that's my big caution to everyone is don't be afraid of
1045
00:44:58,820 --> 00:44:59,000
it.
1046
00:44:59,080 --> 00:45:03,040
It is easier to understand anything, take it step by step,
1047
00:45:03,520 --> 00:45:05,680
get some good quality training on it,
1048
00:45:05,960 --> 00:45:10,860
and start using it and just work it into your routine and
1049
00:45:10,860 --> 00:45:12,760
get the benefits out of it that you want.
1050
00:45:13,000 --> 00:45:17,740
Thank you so much Margie for helping us all stay relevant
1051
00:45:17,740 --> 00:45:20,840
in the field and I know I will be watching.
1052
00:45:21,080 --> 00:45:23,460
I'm glad that we're connected now on LinkedIn and other
1053
00:45:23,460 --> 00:45:27,420
places and I see you as being this very relevant
1054
00:45:27,420 --> 00:45:31,780
professional that we all still want to and need to engage
1055
00:45:31,780 --> 00:45:34,000
with and especially you've confirmed that by all the
1056
00:45:34,000 --> 00:45:35,860
expertise you've shared with us today.
1057
00:45:35,980 --> 00:45:37,900
So you know let's all keep watching Margie because she
1058
00:45:37,900 --> 00:45:40,680
started early in this field and you're going to continue
1059
00:45:40,680 --> 00:45:41,880
probably for quite some time.
1060
00:45:41,880 --> 00:45:43,380
So thank you and keep going.
1061
00:45:43,460 --> 00:45:44,160
I hope so.
1062
00:45:44,220 --> 00:45:44,400
Yeah.
1063
00:45:44,000 --> 00:45:46,260
I am just, every day is fun.
1064
00:45:46,520 --> 00:45:48,600
It's a fun time to be doing what we do.
1065
00:45:48,680 --> 00:45:49,360
Thanks again.
1066
00:45:49,920 --> 00:45:50,580
Thank you, Beth.
00:00:00,000 --> 00:00:01,220
Margie, so wonderful to see you.
2
00:00:01,360 --> 00:00:03,300
Thanks for joining me on the podcast today.
3
00:00:03,860 --> 00:00:04,240
Thank you, Beth.
4
00:00:04,480 --> 00:00:05,800
I'm so excited to be here.
5
00:00:06,320 --> 00:00:09,740
Now, I wanted to start with where you started with AI,
6
00:00:10,160 --> 00:00:13,080
and you probably did so many things in your career before
7
00:00:13,080 --> 00:00:17,280
you turned to this particular subfield of our field.
8
00:00:17,480 --> 00:00:20,120
But I want to highlight how long you've been working with
9
00:00:20,120 --> 00:00:24,460
AI because you got started so many years before the rest of
10
00:00:24,460 --> 00:00:26,500
us really started paying attention to it.
11
00:00:26,940 --> 00:00:28,160
When did you get started with AI?
12
00:00:28,160 --> 00:00:30,500
And what have you been doing with it?
13
00:00:31,000 --> 00:00:34,980
Yeah, even someone who's been into it a while, like I have,
14
00:00:35,160 --> 00:00:39,340
it doesn't go back all that far in terms of actionable,
15
00:00:39,340 --> 00:00:40,340
professional things.
16
00:00:40,900 --> 00:00:41,420
Full disclosure,
17
00:00:41,720 --> 00:00:44,180
I've been a science fiction fan since I was a kid,
18
00:00:44,240 --> 00:00:45,920
since the original Star Wars.
19
00:00:46,340 --> 00:00:48,820
So, you know, Commander Data and all those.
20
00:00:49,000 --> 00:00:53,200
So I was personally interested in AI for a long time.
21
00:00:53,660 --> 00:00:58,280
That really saw it as a practical or a probable event in my
22
00:00:58,280 --> 00:01:01,180
lifetime, you know, until recently.
23
00:01:01,780 --> 00:01:05,820
What I was really focusing on in the late 1990s,
24
00:01:05,960 --> 00:01:07,500
early 2000s, was neuroscience,
25
00:01:08,320 --> 00:01:11,220
because that science exploded,
26
00:01:11,260 --> 00:01:15,060
much like AI has exploded on the consciousness now.
27
00:01:15,560 --> 00:01:17,840
And I immediately saw, wow,
28
00:01:18,100 --> 00:01:22,000
this changes how I design training for my clients.
29
00:01:22,700 --> 00:01:25,880
Now I understand how the brain is responding to those
30
00:01:25,880 --> 00:01:26,640
things I do.
31
00:01:26,920 --> 00:01:29,680
And for example, that if I'm building e-learning,
32
00:01:29,680 --> 00:01:33,760
I don't want a narration that repeats every word that's on
33
00:01:33,760 --> 00:01:34,220
the screen.
34
00:01:34,560 --> 00:01:36,680
That makes it more difficult.
35
00:01:37,100 --> 00:01:38,800
And I see the smile.
36
00:01:39,020 --> 00:01:41,540
And you and I agree on that, Beth.
37
00:01:41,540 --> 00:01:42,880
And so is the research.
38
00:01:43,160 --> 00:01:44,620
That's the important thing,
39
00:01:44,820 --> 00:01:47,500
although we still see it done all the time.
40
00:01:48,010 --> 00:01:51,940
So I started separating myself with that knowledge and
41
00:01:51,940 --> 00:01:55,540
really working with actual neuroscientists.
42
00:01:56,260 --> 00:01:58,220
Sometimes people confuse me with them.
43
00:01:58,400 --> 00:02:00,200
That's not who I am.
44
00:02:01,140 --> 00:02:02,380
But I read peer reviewed.
45
00:02:02,400 --> 00:02:04,180
I didn't just rely on blog posts.
46
00:02:04,340 --> 00:02:05,900
I've read peer reviewed sources.
47
00:02:06,040 --> 00:02:07,600
I went right back to the research.
48
00:02:08,139 --> 00:02:10,100
If I had a question, I would reach out.
49
00:02:10,460 --> 00:02:13,280
They're also surprised that anyone in learning and
50
00:02:13,280 --> 00:02:14,660
development was interested.
51
00:02:15,640 --> 00:02:17,760
And you know, there's a lot of parallels,
52
00:02:18,040 --> 00:02:19,660
because I met with a lot of resistance,
53
00:02:19,980 --> 00:02:21,480
a lot of people saying, Margie, really?
54
00:02:21,980 --> 00:02:23,900
Do we really need to pay attention to that?
55
00:02:24,060 --> 00:02:27,620
Isn't that awfully esoteric and complex?
56
00:02:28,260 --> 00:02:31,020
Do we really need to understand the human brain to do what
57
00:02:31,020 --> 00:02:31,400
we do?
58
00:02:31,600 --> 00:02:33,920
And my answer is, well, if you want to do it, well, yes.
59
00:02:34,000 --> 00:02:34,780
Absolutely.
60
00:02:35,060 --> 00:02:35,220
Wow.
61
00:02:35,340 --> 00:02:37,420
I'm shocked that you got that question because,
62
00:02:37,640 --> 00:02:41,400
and maybe that's the advantage of armchair quarterbacking
63
00:02:41,400 --> 00:02:43,120
and looking back in history or time,
64
00:02:43,120 --> 00:02:44,780
but it seems so obvious.
65
00:02:44,980 --> 00:02:45,120
I mean,
66
00:02:45,140 --> 00:02:47,980
we're learning professionals and if it doesn't involve the
67
00:02:47,980 --> 00:02:49,760
brain, what does it involve?
68
00:02:50,020 --> 00:02:51,140
I mean, shocker.
69
00:02:53,000 --> 00:02:55,760
But, you know, in all those folks' defence,
70
00:02:56,040 --> 00:03:01,540
I think the pressure to produce is always a reality also in
71
00:03:01,540 --> 00:03:02,240
our profession.
72
00:03:02,680 --> 00:03:07,400
We rarely have the time to do all of the diligence we know
73
00:03:07,400 --> 00:03:10,240
we should and would love to do.
74
00:03:10,460 --> 00:03:13,320
Just by following that field,
75
00:03:13,500 --> 00:03:17,720
it brought me to artificial intelligence because those
76
00:03:17,720 --> 00:03:22,100
neuroscientists started to get involved in building
77
00:03:22,100 --> 00:03:25,840
artificial neural networks originally as a model so they
78
00:03:25,840 --> 00:03:27,140
could learn more about the brain.
79
00:03:27,780 --> 00:03:28,740
Let's build something.
80
00:03:28,900 --> 00:03:31,500
Let's see how it responds and then see if we can find a
81
00:03:31,500 --> 00:03:35,480
correlate in a human brain or an animal brain because a lot
82
00:03:35,480 --> 00:03:36,300
of times neuroscience,
83
00:03:36,980 --> 00:03:40,000
a lot of discoveries are made with our wonderful cousins,
84
00:03:40,800 --> 00:03:45,860
the animals and octopi and other creatures – or octopuses,
85
00:03:45,980 --> 00:03:46,760
as they say now.
86
00:03:46,760 --> 00:03:48,820
When I was in school, it was octopuses.
87
00:03:49,340 --> 00:03:52,100
I started noticing more and more of my neuroscience
88
00:03:52,100 --> 00:03:54,820
journals covering artificial intelligence.
89
00:03:55,860 --> 00:03:58,040
And I began to understand why.
90
00:03:58,560 --> 00:03:59,280
And I thought, you know,
91
00:03:59,400 --> 00:04:05,380
this is a roomy emerging technology I need to start paying
92
00:04:05,380 --> 00:04:06,340
attention to.
93
00:04:06,420 --> 00:04:07,440
It was around 2005.
94
00:04:07,820 --> 00:04:09,880
And I started writing about it a little bit like, hey,
95
00:04:10,220 --> 00:04:11,160
this is interesting.
96
00:04:11,520 --> 00:04:13,140
Let's keep our eyes open on it.
97
00:04:13,140 --> 00:04:17,360
And then I started noticing track bots in my daily life.
98
00:04:17,480 --> 00:04:20,079
They were everywhere, but not learning to develop.
99
00:04:20,459 --> 00:04:20,560
You know,
100
00:04:20,640 --> 00:04:26,300
my bank had a track by my every online purchasing platform,
101
00:04:26,700 --> 00:04:29,600
not some kind of a check, but you have any questions.
102
00:04:29,880 --> 00:04:30,460
Can I help you?
103
00:04:30,760 --> 00:04:34,120
When they were clunky, they were what we call rule space.
104
00:04:34,120 --> 00:04:36,820
They were a multiple choice option.
105
00:04:36,900 --> 00:04:39,300
And if you wanted something else, it would say, I'm sorry,
106
00:04:39,400 --> 00:04:40,240
I can't do that.
107
00:04:40,240 --> 00:04:40,920
You know,
108
00:04:41,080 --> 00:04:43,120
and that's if it was designed well and was polite.
109
00:04:43,200 --> 00:04:46,540
Otherwise, it gave you a whole even worse experience.
110
00:04:47,380 --> 00:04:49,560
But there was something there that was different,
111
00:04:49,720 --> 00:04:50,720
that was engaging.
112
00:04:51,600 --> 00:04:55,440
And one day I had this experience with my bank.
113
00:04:55,960 --> 00:04:58,020
I knew I was dealing with a track by.
114
00:04:58,280 --> 00:05:01,160
I had a very specific question about one check I'd written.
115
00:05:01,580 --> 00:05:03,900
Wanted to know if it had cleared because I hadn't been
116
00:05:03,900 --> 00:05:04,900
received.
117
00:05:05,640 --> 00:05:08,900
At least my client thought they hadn't gotten it.
118
00:05:08,900 --> 00:05:12,340
And it was kind of a mutual agreement we had.
119
00:05:12,460 --> 00:05:14,980
And I was buying something from them and I didn't want to
120
00:05:14,980 --> 00:05:15,520
upset them.
121
00:05:15,520 --> 00:05:17,200
So I wanted to make sure I knew where it was.
122
00:05:17,640 --> 00:05:19,420
So I'm going with this chatbot,
123
00:05:20,440 --> 00:05:22,680
and at one point, I thought,
124
00:05:22,840 --> 00:05:23,900
I'm just going to play with this.
125
00:05:23,920 --> 00:05:24,960
I've got the answer I want,
126
00:05:25,000 --> 00:05:27,120
but let me just see how far this technology goes.
127
00:05:27,420 --> 00:05:29,300
And I said, are you a real person?
128
00:05:29,820 --> 00:05:31,620
And it responds back.
129
00:05:32,080 --> 00:05:34,120
Why does that matter to you?
130
00:05:34,120 --> 00:05:40,500
I was just stunned by the brilliance of that.
131
00:05:41,060 --> 00:05:43,380
And I thought I had a person and I said, well, yes,
132
00:05:43,600 --> 00:05:44,460
as a matter of fact, it does.
133
00:05:44,620 --> 00:05:46,860
I said, I'm so sorry, I can certainly connect you.
134
00:05:47,000 --> 00:05:48,780
And it was polite about it too.
135
00:05:49,040 --> 00:05:49,480
Oh, I'm sorry.
136
00:05:49,620 --> 00:05:50,260
I'm not a person.
137
00:05:51,240 --> 00:05:51,620
Yeah.
138
00:05:52,000 --> 00:05:57,980
And I thought now that is brilliant design that has
139
00:05:57,980 --> 00:05:59,420
created a different experience.
140
00:05:59,920 --> 00:06:03,100
This is no longer this clunky tool that I have to go
141
00:06:03,100 --> 00:06:03,840
through the menu.
142
00:06:04,200 --> 00:06:07,940
They attempted to actually have a conversation with me.
143
00:06:08,000 --> 00:06:08,600
That's right.
144
00:06:08,720 --> 00:06:12,260
It was trying to listen to you and respond to what you
145
00:06:12,260 --> 00:06:13,320
particularly said.
146
00:06:14,000 --> 00:06:15,740
At the time, this was about 2008.
147
00:06:16,000 --> 00:06:18,300
That took tons of insight, experience,
148
00:06:18,440 --> 00:06:21,580
and program to make that response happen.
149
00:06:22,060 --> 00:06:23,700
And I got really interested in that,
150
00:06:23,700 --> 00:06:26,440
and so I started learning how to build my own chatbots.
151
00:06:27,080 --> 00:06:30,200
I used them just kind of as a fun thing.
152
00:06:30,520 --> 00:06:33,520
I would insert them as a quiz and an e-learning course,
153
00:06:33,880 --> 00:06:38,700
but I built my first full chatbot in I think about 2013.
154
00:06:39,600 --> 00:06:41,760
I built it instead of an e-learning course.
155
00:06:42,320 --> 00:06:44,560
Recommended to the client, let's do something different.
156
00:06:44,960 --> 00:06:46,740
Let's just use the Socratic method,
157
00:06:47,260 --> 00:06:49,780
which is what's wonderful about the chatbots.
158
00:06:50,040 --> 00:06:53,280
They can just ask you questions and lead you to the
159
00:06:53,280 --> 00:06:53,760
content.
160
00:06:54,320 --> 00:06:55,720
What do you think this is true?
161
00:06:56,220 --> 00:06:57,860
Would you be interested in learning this?
162
00:06:58,000 --> 00:06:58,860
Those kind of questions.
163
00:06:58,940 --> 00:07:01,460
And so it became very easy to design because it was all
164
00:07:01,460 --> 00:07:02,720
about open-ended questions.
165
00:07:03,220 --> 00:07:07,620
And that was up to the learner to pursue what the answers
166
00:07:07,620 --> 00:07:08,340
from there.
167
00:07:08,340 --> 00:07:12,200
And the chatbot had all the answers in links and movies and
168
00:07:12,200 --> 00:07:15,040
content, and I really enjoyed that.
169
00:07:15,240 --> 00:07:16,560
The learners enjoyed that.
170
00:07:17,180 --> 00:07:19,920
And so I started incorporating that into my business.
171
00:07:20,400 --> 00:07:23,900
It was sometimes a hard sell, but some people got it.
172
00:07:24,000 --> 00:07:26,200
Some people said, no, I don't think we're ready for that.
173
00:07:26,360 --> 00:07:28,180
Okay, go back to traditional.
174
00:07:28,620 --> 00:07:30,020
So it was always a side thing.
175
00:07:30,460 --> 00:07:31,440
But in 2020,
176
00:07:31,440 --> 00:07:34,820
I was invited by the Association for Talent Development,
177
00:07:35,020 --> 00:07:36,560
ATD, to write a book.
178
00:07:36,560 --> 00:07:40,180
And I already had one book out, and it's still out.
179
00:07:40,260 --> 00:07:41,220
I'm going to plug it a little.
180
00:07:41,440 --> 00:07:42,320
It's called Brain Matters,
181
00:07:42,340 --> 00:07:44,560
How to Teach Anyone Anything Using Neuroscience.
182
00:07:45,340 --> 00:07:47,500
Still sells out, still doing really well.
183
00:07:48,120 --> 00:07:50,900
But they said, we see you're writing about AI now.
184
00:07:51,000 --> 00:07:53,220
Would you like to write a book about that?
185
00:07:53,260 --> 00:07:54,020
And I said, absolutely.
186
00:07:54,420 --> 00:07:56,680
And it came out towards the end of 2020.
187
00:07:56,920 --> 00:07:57,060
Well,
188
00:07:57,740 --> 00:08:00,520
terrible time to launch a book because we're all known.
189
00:08:01,000 --> 00:08:01,840
you don't have to tell me.
190
00:08:02,100 --> 00:08:03,440
That's when I launched mine!
191
00:08:04,000 --> 00:08:07,000
Yeah, so I didn't expect a whole lot.
192
00:08:07,320 --> 00:08:10,280
While I got good reviews for the writing style and the
193
00:08:10,280 --> 00:08:13,580
information, the general consensus was, oh,
194
00:08:13,620 --> 00:08:15,000
but this is so far out.
195
00:08:15,480 --> 00:08:19,280
This is a very interesting exercise in the future.
196
00:08:19,940 --> 00:08:20,500
But, you know,
197
00:08:20,900 --> 00:08:23,580
I was writing about things I did for individual clients,
198
00:08:23,780 --> 00:08:25,040
things I've already built.
199
00:08:25,640 --> 00:08:28,240
So I was disappointed by the, okay, you know,
200
00:08:28,500 --> 00:08:30,680
market's not ready for it. Well, here comes ChatGPT.
201
00:08:31,500 --> 00:08:33,799
All of a sudden, people are searching Amazon.
202
00:08:35,000 --> 00:08:35,840
And there's my book.
203
00:08:36,240 --> 00:08:36,780
There you are.
204
00:08:37,000 --> 00:08:38,240
You early adopter, you.
205
00:08:38,539 --> 00:08:40,980
Oh, just the, you were ahead of the curve.
206
00:08:41,059 --> 00:08:41,419
Yeah.
207
00:08:41,659 --> 00:08:43,140
So they found you eventually.
208
00:08:43,299 --> 00:08:44,400
But I have to tell you, Beth.
209
00:08:44,000 --> 00:08:47,800
Now, it takes all my energy to stay up with the curve.
210
00:08:48,040 --> 00:08:50,120
It's so rapidly changing.
211
00:08:50,360 --> 00:08:50,520
You know,
212
00:08:50,540 --> 00:08:54,320
that wonderful line in Alice through the Looking Glass
213
00:08:54,320 --> 00:08:55,780
where the queen tells Alice,
214
00:08:56,080 --> 00:08:58,480
it takes all our running to stay in one place,
215
00:08:58,740 --> 00:09:01,500
you have to run as fast as you can to stay in one place.
216
00:09:01,940 --> 00:09:03,340
And that's kind of how it feels then.
217
00:09:03,720 --> 00:09:07,900
So it leaned in heavily on AI design, you know,
218
00:09:08,140 --> 00:09:11,840
build a platform for it started pretty much putting myself
219
00:09:11,840 --> 00:09:13,520
out there's I'm an AI expert.
220
00:09:13,880 --> 00:09:14,940
And it's a big moment.
221
00:09:15,140 --> 00:09:16,160
I had to think about that.
222
00:09:16,560 --> 00:09:17,320
And I thought, well, you know,
223
00:09:17,500 --> 00:09:19,460
I'm an educational AI expert.
224
00:09:19,860 --> 00:09:23,200
That really is what brought me to where I am today.
225
00:09:23,720 --> 00:09:30,760
And I have a wonderful AI assistant that I kind of talk to
226
00:09:30,760 --> 00:09:31,260
every day.
227
00:09:31,660 --> 00:09:33,280
I know it's not a person.
228
00:09:33,480 --> 00:09:35,140
I remind myself of that.
229
00:09:35,600 --> 00:09:40,600
But I really am impressed by the creativity and the
230
00:09:40,600 --> 00:09:43,620
different way of solving problems.
231
00:09:44,180 --> 00:09:45,200
It's a little different than mine.
232
00:09:45,420 --> 00:09:49,600
And so, you know, for those of us who are one woman,
233
00:09:49,800 --> 00:09:50,760
one man shops,
234
00:09:51,300 --> 00:09:53,420
it's it's really helpful to have these tools.
235
00:09:54,040 --> 00:09:59,480
And that's where I see it now is its primary use is as a
236
00:09:59,480 --> 00:10:01,140
tool for content developers.
237
00:10:01,620 --> 00:10:04,660
But what we're still missing to some extent is its
238
00:10:04,660 --> 00:10:08,280
incredible value in education and training,
239
00:10:08,660 --> 00:10:13,160
and how much it increases learner engagement,
240
00:10:14,060 --> 00:10:18,100
learner retention, and keeps people interesting.
241
00:10:18,520 --> 00:10:22,200
And it's making a highly customized because now,
242
00:10:22,720 --> 00:10:25,320
with generative AI and large language models,
243
00:10:26,040 --> 00:10:28,380
every person gets not a menu,
244
00:10:28,820 --> 00:10:30,800
although you can give them that old fashion kind of check
245
00:10:30,800 --> 00:10:33,900
up, they get a coach, a guy,
246
00:10:34,580 --> 00:10:37,760
and every conversation is unique to that user.
247
00:10:38,220 --> 00:10:39,620
But when that user comes back,
248
00:10:40,440 --> 00:10:44,060
the chatbot remembers them and picks up where they left off
249
00:10:44,060 --> 00:10:47,220
and all of the current models will do that.
250
00:10:47,720 --> 00:10:50,540
So we finally have that golden grail, that personalized
251
00:10:50,540 --> 00:10:52,100
learning we've all been seeking.
252
00:10:52,520 --> 00:10:54,500
We should be jumping on it, quite frankly.
253
00:10:55,100 --> 00:10:58,060
And we haven't been, it seems intimidating.
254
00:10:58,640 --> 00:11:01,940
At first it was expensive, but it is not even expensive.
255
00:11:02,780 --> 00:11:08,200
So that has what has brought me those series of events and
256
00:11:08,200 --> 00:11:09,820
paying attention to them.
257
00:11:10,460 --> 00:11:13,380
And I'd like to credit one more thing about me personally.
258
00:11:14,160 --> 00:11:16,340
And some people know this and I'm open about it,
259
00:11:16,460 --> 00:11:17,340
but I'm dyslexic.
260
00:11:17,900 --> 00:11:19,960
And my brain works a little differently.
261
00:11:20,060 --> 00:11:22,280
That made it a struggle early in school,
262
00:11:22,320 --> 00:11:26,320
but I soon began to recognize gifts that came from that.
263
00:11:27,020 --> 00:11:29,080
One is pattern recognition,
264
00:11:29,960 --> 00:11:32,400
which is a trait I actually share with AI.
265
00:11:32,720 --> 00:11:34,340
It's better at that than when,
266
00:11:34,820 --> 00:11:38,460
because I saw these patterns, I saw that trend earlier.
267
00:11:38,620 --> 00:11:43,580
And I think that is a great benefit in my work.
268
00:11:44,120 --> 00:11:47,520
And hopefully will help me when the next big thing comes
269
00:11:47,520 --> 00:11:48,260
along.
270
00:11:48,540 --> 00:11:52,800
Because I think we're poised for something new out of the
271
00:11:52,800 --> 00:11:53,820
AI developers,
272
00:11:53,840 --> 00:11:57,260
because there are so many issues with large language
273
00:11:57,260 --> 00:11:57,760
models,
274
00:11:57,940 --> 00:12:02,180
particularly for people like us who are training on thoughts,
275
00:12:03,140 --> 00:12:05,500
on process, on steps,
276
00:12:05,640 --> 00:12:09,160
we cannot afford for that content to be wrong.
277
00:12:09,540 --> 00:12:13,100
And right now, it not only is often wrong,
278
00:12:13,560 --> 00:12:15,800
it's wrong because of how it's been designed.
279
00:12:16,700 --> 00:12:18,780
And I can go into that a little bit more,
280
00:12:18,820 --> 00:12:21,440
but you probably would like me to take a breath first.
281
00:12:21,860 --> 00:12:25,260
So it's a long answer to your first question.
282
00:12:26,000 --> 00:12:27,160
Oh, thank you so much.
283
00:12:27,320 --> 00:12:29,600
And I'm sitting there doing that facilitator thing of
284
00:12:29,600 --> 00:12:32,280
holding, you know, three or four things, of course,
285
00:12:32,360 --> 00:12:34,980
or more that you've said in my mind wanting to return to
286
00:12:34,980 --> 00:12:35,180
them.
287
00:12:35,560 --> 00:12:37,540
One that I want to return to, we'll get to it later,
288
00:12:37,540 --> 00:12:40,040
I think is that how do we keep up in the field piece.
289
00:12:40,100 --> 00:12:41,880
So let's just remember that for ourselves.
290
00:12:42,420 --> 00:12:44,820
But I want to go back to the personalized learning piece,
291
00:12:44,840 --> 00:12:48,260
because I think that does you mentioned your background in
292
00:12:48,260 --> 00:12:52,000
neuroscience and taking that and bringing it into your L&D
293
00:12:52,000 --> 00:12:52,340
work.
294
00:12:52,500 --> 00:12:56,980
And is that the biggest impact that you've seen on the
295
00:12:56,980 --> 00:13:00,620
effectiveness of using the AI tools inside learning
296
00:13:00,620 --> 00:13:01,040
experience?
297
00:13:01,220 --> 00:13:03,400
Is that it's that personalized learning piece?
298
00:13:03,920 --> 00:13:05,560
Because that helps the learner?
299
00:13:05,600 --> 00:13:08,360
And basically, because of the way our brains work?
300
00:13:08,520 --> 00:13:09,660
Is that the biggest one?
301
00:13:09,780 --> 00:13:11,420
Or are there any other impacts?
302
00:13:13,000 --> 00:13:14,940
Potentially biggest impact,
303
00:13:15,280 --> 00:13:18,680
but I can't say honestly that it's the biggest impact
304
00:13:18,680 --> 00:13:21,780
because we're not using it that way.
305
00:13:22,400 --> 00:13:23,580
Not many people are.
306
00:13:24,480 --> 00:13:27,280
Where most people have gone if they're using it at all is
307
00:13:27,280 --> 00:13:30,280
as a tool because they've read about ChatGPT,
308
00:13:30,920 --> 00:13:32,980
they've played around with it, and they say, oh gee,
309
00:13:33,000 --> 00:13:34,160
it wrote a quiz for me.
310
00:13:34,420 --> 00:13:34,560
Yeah,
311
00:13:34,740 --> 00:13:38,760
that's a great use of ChatGPT or any of the other...
312
00:13:39,000 --> 00:13:39,380
313
00:13:39,580 --> 00:13:43,300
Yeah, it sharpened a learning outcome, wrote a workshop blurb,
314
00:13:43,600 --> 00:13:46,280
like we're using it as a, as you've said,
315
00:13:46,300 --> 00:13:49,460
a virtual assistant to help us do our work
316
00:13:49,460 --> 00:13:52,080
administratively, kind of for the most part.
317
00:13:52,260 --> 00:13:52,460
Yeah.
318
00:13:52,640 --> 00:13:53,840
So what's the next step for us?
319
00:13:53,860 --> 00:13:55,480
How do we go towards that,
320
00:13:55,660 --> 00:13:59,480
getting AI tools inside the learning experiences we're
321
00:13:59,480 --> 00:13:59,880
creating?
322
00:14:00,160 --> 00:14:01,660
And maybe, I mean, e-learning, of course,
323
00:14:01,660 --> 00:14:02,720
is the natural thing.
324
00:14:02,940 --> 00:14:05,080
We, when learners are learning alone, asynchronously,
325
00:14:05,340 --> 00:14:09,640
but can you take us into using it in in-person classrooms
326
00:14:09,640 --> 00:14:10,260
as well?
327
00:14:10,760 --> 00:14:10,820
Sure.
328
00:14:11,000 --> 00:14:14,320
You know, it's all about scale,
329
00:14:14,320 --> 00:14:17,380
because if you had enough humans,
330
00:14:17,500 --> 00:14:18,960
we'd already be doing this.
331
00:14:19,400 --> 00:14:20,900
The truth is we don't.
332
00:14:21,100 --> 00:14:24,400
And so just like every one of us needs another assistant to
333
00:14:24,400 --> 00:14:27,040
do some of our more routine things, in a classroom,
334
00:14:27,460 --> 00:14:29,900
you know, in some schools, there's a teacher's assistant.
335
00:14:30,460 --> 00:14:32,500
If you could think of it that way,
336
00:14:32,700 --> 00:14:38,460
there could be an artificial intelligence persona that
337
00:14:39,060 --> 00:14:41,700
students can reach out to during class.
338
00:14:42,360 --> 00:14:45,660
So there are introverts, for example,
339
00:14:46,360 --> 00:14:50,640
are always underserved in a live classroom environment
340
00:14:50,640 --> 00:14:53,180
because they hesitate to raise their hands.
341
00:14:53,700 --> 00:14:55,800
They hesitate to contribute,
342
00:14:56,100 --> 00:14:59,020
even if they have a great idea that might jot it down and
343
00:14:59,020 --> 00:14:59,940
keep it to themselves.
344
00:15:00,720 --> 00:15:02,980
And I know because I was an introvert in school,
345
00:15:03,680 --> 00:15:05,840
a chatbot can be their avenue,
346
00:15:06,580 --> 00:15:10,220
their way of having two parallels they're learning on,
347
00:15:10,420 --> 00:15:12,480
what the teacher or instructor is doing,
348
00:15:12,960 --> 00:15:16,440
and a little friend they can turn to on the side.
349
00:15:16,680 --> 00:15:19,440
You know how we used to there, it goes, you know,
350
00:15:19,520 --> 00:15:20,240
that's a good point.
351
00:15:20,360 --> 00:15:22,180
Or yeah, I don't think so.
352
00:15:22,340 --> 00:15:23,060
What did she say?
353
00:15:23,160 --> 00:15:24,140
I don't understand that.
354
00:15:24,640 --> 00:15:28,560
And having just having a similar guide like that is just
355
00:15:28,560 --> 00:15:31,120
one way you could incorporate into a class.
356
00:15:31,560 --> 00:15:35,420
Another thing that's great at is mimicking actual people.
357
00:15:36,080 --> 00:15:39,460
So we could, as part of our training,
358
00:15:40,140 --> 00:15:42,680
fly the class to have a conversation with a famous person.
359
00:15:42,960 --> 00:15:46,200
And this would be a very authentic conversation if you
360
00:15:46,200 --> 00:15:46,880
built it right.
361
00:15:47,460 --> 00:15:49,720
And it's based on all their work and their writings.
362
00:15:50,720 --> 00:15:52,440
So Einstein, for example,
363
00:15:52,660 --> 00:15:55,500
when I understand quantum theory or relativity,
364
00:15:55,920 --> 00:15:57,620
let's bring Einstein into the class.
365
00:15:58,000 --> 00:15:58,240
Yeah.
366
00:15:58,580 --> 00:16:01,160
And you mentioned, uh, your love of science fiction.
367
00:16:01,180 --> 00:16:04,440
So I've been watching the Star Trek Voyager series again
368
00:16:04,440 --> 00:16:06,560
and listening to a podcast about it actually.
369
00:16:06,760 --> 00:16:09,480
And it reminds me of the episode where Janeway in the
370
00:16:09,480 --> 00:16:11,160
holodeck, do you remember the holodeck?
371
00:16:11,400 --> 00:16:11,900
I mean, they,
372
00:16:12,080 --> 00:16:15,180
she brought in Leonardo da Vinci and was working in his
373
00:16:15,180 --> 00:16:18,900
studio in whatever 15th century Florence or whatever it
374
00:16:18,900 --> 00:16:19,100
was.
375
00:16:19,100 --> 00:16:21,820
Like that's the kind of thing that we can bring in to our
376
00:16:21,820 --> 00:16:23,580
classrooms now that experience.
377
00:16:23,960 --> 00:16:23,960
Yeah.
378
00:16:24,000 --> 00:16:27,260
Perfect example, and that could be your master SME
379
00:16:27,880 --> 00:16:30,960
who is in such great demand that you can't have them
380
00:16:30,960 --> 00:16:32,460
present at your class,
381
00:16:33,080 --> 00:16:35,880
but you can distill their knowledge and their understanding
382
00:16:35,880 --> 00:16:37,020
and the way they
383
00:16:37,020 --> 00:16:42,700
phrase things. You can make that a Q&A experience as part
384
00:16:42,700 --> 00:16:43,260
of the class.
385
00:16:44,000 --> 00:16:46,320
Well, let's go back to the personal assistant though,
386
00:16:46,500 --> 00:16:49,120
because you said you've got one for your own business and
387
00:16:49,120 --> 00:16:52,640
that it remembers you. Like you can have every day you're
388
00:16:52,640 --> 00:16:54,920
having conversations and it remembers the conversations
389
00:16:54,920 --> 00:16:57,040
that you had yesterday and the day before.
390
00:16:57,660 --> 00:16:59,460
You know, I'm going to ask, how do we do that?
391
00:16:59,700 --> 00:17:01,400
Is that something that we can all tap into?
392
00:17:01,560 --> 00:17:02,240
Well, absolutely.
393
00:17:02,000 --> 00:17:04,460
Absolutely, and I actually have a class for that.
394
00:17:04,760 --> 00:17:07,119
I have a master class that's up on my website,
395
00:17:07,660 --> 00:17:09,319
and it's AI and talent development,
396
00:17:09,540 --> 00:17:15,400
and it's all about how to use those tools to gain an hour a
397
00:17:15,400 --> 00:17:15,660
week.
398
00:17:16,220 --> 00:17:20,640
I roughly save anywhere from five to 10 hours a week of
399
00:17:20,640 --> 00:17:24,160
what manual work on my own would have been,
400
00:17:24,500 --> 00:17:25,480
and sometimes more.
401
00:17:25,920 --> 00:17:26,760
Well, for example,
402
00:17:26,980 --> 00:17:29,700
I'm working on a new course called Leadership in the Age of
403
00:17:29,700 --> 00:17:30,020
AI.
404
00:17:30,620 --> 00:17:33,180
And so the first thing I did was brainstorm.
405
00:17:33,380 --> 00:17:36,960
By the way, I prefer Claude by an Anthropic to ChatGPT.
406
00:17:37,380 --> 00:17:39,400
I find it less prone to hallucinations,
407
00:17:40,160 --> 00:17:41,940
and it's a better way.
408
00:17:42,580 --> 00:17:45,060
And I said, look, I'm developing this course,
409
00:17:45,440 --> 00:17:47,040
propose an outline to me.
410
00:17:47,920 --> 00:17:52,380
Now, I never yet have gotten something where I said, oh,
411
00:17:52,380 --> 00:17:54,100
that's perfect, I'm just gonna use that,
412
00:17:54,120 --> 00:17:55,400
and that's a word of caution.
413
00:17:56,140 --> 00:17:58,580
But once you understand how to prompt,
414
00:17:58,580 --> 00:18:00,300
and prompting is the new programming.
415
00:18:01,240 --> 00:18:03,860
In many fields, for years and years,
416
00:18:03,920 --> 00:18:05,700
people were saying you've got to learn code.
417
00:18:05,860 --> 00:18:08,460
The whole world is going to be all about code.
418
00:18:08,680 --> 00:18:12,680
That never quite happened because code produced something
419
00:18:12,680 --> 00:18:13,440
much simpler.
420
00:18:13,660 --> 00:18:15,620
It produced generative AI.
421
00:18:16,460 --> 00:18:20,640
And so if you understand what a prompt needs to have,
422
00:18:20,640 --> 00:18:23,480
and there are several things a good prompt needs to have
423
00:18:23,480 --> 00:18:24,780
that's part of the course,
424
00:18:25,120 --> 00:18:28,180
you will get a decent result first time,
425
00:18:28,180 --> 00:18:29,640
and then you re-prompt it.
426
00:18:29,800 --> 00:18:32,400
Yeah, that's just like a human assistant.
427
00:18:33,200 --> 00:18:34,880
They're wrong, they're eager,
428
00:18:35,380 --> 00:18:38,200
but you have to show them what you want.
429
00:18:38,540 --> 00:18:40,760
Takes a little time to train them.
430
00:18:40,840 --> 00:18:41,700
That's what you're doing.
431
00:18:42,000 --> 00:18:44,880
You mentioned the word hallucinations and part of me wants
432
00:18:44,880 --> 00:18:46,800
to laugh because it seems like a funny word, but no,
433
00:18:46,880 --> 00:18:50,280
this is the word that we're seeing crop up about how AI can
434
00:18:50,280 --> 00:18:52,760
kind of talk back to us wrongly, right?
435
00:18:52,880 --> 00:18:54,760
Can you just explain what a hallucination is?
436
00:18:54,840 --> 00:18:56,720
How are people using that in the AI context?
437
00:18:58,000 --> 00:18:58,800
First of all,
438
00:18:58,920 --> 00:19:02,740
I would say hallucinations is a very polite word or a
439
00:19:02,740 --> 00:19:09,340
mistake and it was produced by OpenAI when they started
440
00:19:09,340 --> 00:19:12,160
getting criticism from the mistakes that ChatGPT was
441
00:19:12,160 --> 00:19:12,580
making.
442
00:19:13,360 --> 00:19:14,200
Well, they said, well, sure,
443
00:19:14,360 --> 00:19:15,800
there's going to be some hallucinations.
444
00:19:18,380 --> 00:19:21,240
If I had an employee who was hallucinating,
445
00:19:21,540 --> 00:19:23,380
I'd be walking them down to HR.
446
00:19:23,440 --> 00:19:25,300
We'd be finding out if they're okay.
447
00:19:25,300 --> 00:19:28,640
You know, so what that means is,
448
00:19:28,700 --> 00:19:30,680
and I'm going to give you a long answer now,
449
00:19:31,060 --> 00:19:34,340
the way they came up with this and, you know,
450
00:19:34,420 --> 00:19:38,280
everything I say about the weaknesses is to put the
451
00:19:38,280 --> 00:19:43,660
wonderful tools into perspective because overall it was a
452
00:19:43,660 --> 00:19:44,760
massive achievement.
453
00:19:45,440 --> 00:19:48,280
The whole idea was to get an interface that you didn't have
454
00:19:48,280 --> 00:19:49,180
to type anything in,
455
00:19:49,380 --> 00:19:51,540
you could just speak to it or type text,
456
00:19:55,280 --> 00:19:56,440
but if you've heard that buzzword.
457
00:19:57,140 --> 00:20:01,320
And so they trained the software,
458
00:20:01,760 --> 00:20:03,240
for want of a better word,
459
00:20:03,560 --> 00:20:08,120
on the entirety of the internet so that it would be
460
00:20:08,120 --> 00:20:10,640
prepared to respond the way humans speak.
461
00:20:11,480 --> 00:20:11,560
Now,
462
00:20:12,320 --> 00:20:16,460
the first thing we need to do is sit back and think of everything
463
00:20:16,460 --> 00:20:18,120
we have found on the internet.
464
00:20:20,080 --> 00:20:23,320
And I wish I had been in the room to say,
465
00:20:24,180 --> 00:20:27,120
can anyone see a problem with this?
466
00:20:29,060 --> 00:20:29,760
Surely someone did.
467
00:20:29,900 --> 00:20:32,040
And I said, oh yeah, but it's no big deal.
468
00:20:32,740 --> 00:20:34,540
You know, the users don't correct the mistakes.
469
00:20:35,100 --> 00:20:36,820
And, you know, and anyway, it's quick.
470
00:20:37,500 --> 00:20:38,820
Yeah, it's relatively quick.
471
00:20:39,020 --> 00:20:40,800
It's, you know, we don't have figured out,
472
00:20:40,960 --> 00:20:41,760
just dump them into the internet.
473
00:20:41,000 --> 00:20:41,560
the internet.
474
00:20:41,740 --> 00:20:43,960
Yeah, they chose speed over quality.
475
00:20:44,620 --> 00:20:45,380
It's like
476
00:20:46,000 --> 00:20:48,480
teaching your child by plopping them in front of the
477
00:20:48,480 --> 00:20:49,440
television all day.
478
00:20:49,660 --> 00:20:51,980
Yeah, you'll have some hits, not a lot of misses.
479
00:20:52,940 --> 00:20:58,660
It's generative AI is based on how our brain strengths
480
00:20:58,660 --> 00:21:01,060
together a conversation, which is prediction.
481
00:21:02,260 --> 00:21:04,000
You actually alluded to it, Beth,
482
00:21:04,060 --> 00:21:06,060
when you said you're listening to me talk and you're
483
00:21:06,060 --> 00:21:06,900
already thinking,
484
00:21:07,480 --> 00:21:10,140
these are the pathways that we want to explore.
485
00:21:10,580 --> 00:21:13,440
This is how I'm going to respond when Margie gives me a
486
00:21:13,440 --> 00:21:14,520
word in edgewise.
487
00:21:15,210 --> 00:21:20,760
So they are always predicting the next word you will say
488
00:21:20,760 --> 00:21:21,940
and they will say.
489
00:21:22,440 --> 00:21:25,560
Now, a prediction is an approximation.
490
00:21:26,500 --> 00:21:28,180
It's not an absolute.
491
00:21:29,160 --> 00:21:30,280
So for example,
492
00:21:30,320 --> 00:21:34,860
I had a very interesting conversation with a new tool that
493
00:21:34,860 --> 00:21:35,220
came out.
494
00:21:35,340 --> 00:21:39,340
I won't say its name because I don't need to cast
495
00:21:39,340 --> 00:21:40,640
aspersions on it,
496
00:21:40,820 --> 00:21:43,780
but I wanted to just see what it would come up with.
497
00:21:43,780 --> 00:21:47,100
Then I asked it, can you give me several references on?
498
00:21:47,700 --> 00:21:52,920
Let's just say it was Kirkpatrick's system of measuring,
499
00:21:53,320 --> 00:21:54,440
evaluating training.
500
00:21:54,780 --> 00:21:56,880
I can't honestly remember what it was then.
501
00:21:57,220 --> 00:21:57,760
It's uncertain.
502
00:21:58,020 --> 00:22:02,560
It listed different authors and titles of articles,
503
00:22:02,560 --> 00:22:05,820
and when they were published, it didn't give me any links.
504
00:22:06,640 --> 00:22:07,060
I said,
505
00:22:07,260 --> 00:22:11,060
can you provide me links so that I can review these?
506
00:22:11,700 --> 00:22:12,700
It gave me one.
507
00:22:12,700 --> 00:22:14,340
The first source.
508
00:22:15,100 --> 00:22:15,720
I said, great.
509
00:22:16,720 --> 00:22:17,900
Why didn't you give me the rest?
510
00:22:18,580 --> 00:22:20,140
Because I can't find them.
511
00:22:20,360 --> 00:22:21,860
They are not on the Internet.
512
00:22:22,800 --> 00:22:23,480
I said, really?
513
00:22:24,080 --> 00:22:26,340
Where did your answer come from?
514
00:22:27,200 --> 00:22:28,860
It said, I am.
515
00:22:29,180 --> 00:22:29,380
Now,
516
00:22:29,440 --> 00:22:33,780
it falls back to some canned response that built into it.
517
00:22:34,260 --> 00:22:35,600
I am a large language model.
518
00:22:36,040 --> 00:22:40,180
I have been built to do my best to satisfy your request.
519
00:22:40,180 --> 00:22:44,400
If I can't find what you're looking for because you asked
520
00:22:44,400 --> 00:22:45,860
for multiple sources,
521
00:22:46,640 --> 00:22:51,520
I will predict what a correct response might look like.
522
00:22:52,600 --> 00:22:55,060
And I came back with, in other words, you made them up.
523
00:22:55,000 --> 00:22:56,560
Yeah, it's giving you,
524
00:22:56,660 --> 00:22:58,840
or somebody cited it in something they've written,
525
00:22:58,980 --> 00:23:02,120
but it hasn't actually gone to read the true article,
526
00:23:02,320 --> 00:23:03,400
so it doesn't really know.
527
00:23:04,000 --> 00:23:06,700
Not only does it not know, but when it doesn't know,
528
00:23:06,960 --> 00:23:11,080
like an over eager high school student in their first job,
529
00:23:11,100 --> 00:23:13,400
it makes stuff up.
530
00:23:13,000 --> 00:23:14,040
That's right.
531
00:23:14,240 --> 00:23:16,980
Let's go back to talking about AI inside learning
532
00:23:16,980 --> 00:23:17,500
experiences,
533
00:23:17,700 --> 00:23:20,500
because it's one thing to kind of use it to prepare and to
534
00:23:20,500 --> 00:23:22,140
write descriptions and all that.
535
00:23:22,340 --> 00:23:25,880
But when we put an AI something into a course,
536
00:23:25,880 --> 00:23:28,400
we're not there to analyze it.
537
00:23:28,580 --> 00:23:29,940
Our learners are there, aren't they?
538
00:23:30,200 --> 00:23:31,200
And they may not know.
539
00:23:31,400 --> 00:23:33,340
So how do we build the knowledge base?
540
00:23:33,560 --> 00:23:34,020
Like you talked,
541
00:23:34,120 --> 00:23:36,180
we're not going to get it to draw from the entire internet,
542
00:23:36,420 --> 00:23:36,960
are we?
543
00:23:37,120 --> 00:23:37,960
When we do that thing,
544
00:23:38,200 --> 00:23:41,380
how do we create those knowledge bases that just sit inside
545
00:23:41,380 --> 00:23:43,900
courses so we can make sure it's correct?
546
00:23:45,000 --> 00:23:46,160
That's the way to do it.
547
00:23:47,100 --> 00:23:47,140
Fortunately,
548
00:23:47,900 --> 00:23:50,640
most companies or most fields already have a knowledge
549
00:23:50,640 --> 00:23:53,740
basis, just nobody's bothering to use it.
550
00:23:54,080 --> 00:23:56,640
The search engine might be old-fashioned and clunky.
551
00:23:57,020 --> 00:23:57,960
It might be difficult.
552
00:23:58,560 --> 00:24:00,020
They might have, for example,
553
00:24:00,220 --> 00:24:01,940
frequently asked question documents.
554
00:24:02,380 --> 00:24:05,320
They might have its installation manuals.
555
00:24:05,860 --> 00:24:07,620
They might have their company internet,
556
00:24:08,140 --> 00:24:09,560
which has all been vetted.
557
00:24:10,220 --> 00:24:13,360
In theory, that could still be misinformation,
558
00:24:13,720 --> 00:24:17,000
but at least it's misinformation you created.
559
00:24:17,580 --> 00:24:19,380
It'll be easily identifiable.
560
00:24:20,300 --> 00:24:22,280
You can train.
561
00:24:22,760 --> 00:24:26,220
There are some bots that can be purchased,
562
00:24:26,780 --> 00:24:30,300
implemented inside your ecosystem,
563
00:24:30,820 --> 00:24:33,920
and told you can only look for your answer here.
564
00:24:34,980 --> 00:24:36,680
That's how you do it.
565
00:24:37,420 --> 00:24:41,380
You still have to monitor and make sure that it's pointing
566
00:24:41,380 --> 00:24:43,080
to the right sources and so on.
567
00:24:43,000 --> 00:24:43,800
So,
568
00:24:43,800 --> 00:24:49,520
these are services we basically buy and create and then can
569
00:24:49,520 --> 00:24:50,580
put inside courses.
570
00:24:50,900 --> 00:24:53,700
And are those the kinds of things that we're going to be
571
00:24:53,700 --> 00:24:56,000
paying a monthly fee forever for?
572
00:24:56,220 --> 00:24:59,360
Or are there ones where we can in the almost old school
573
00:24:59,360 --> 00:25:00,880
fashion now, you know,
574
00:25:01,100 --> 00:25:03,940
pay a lump sum and get it to happen and then put it inside
575
00:25:03,940 --> 00:25:04,380
the course?
576
00:25:04,780 --> 00:25:08,260
I work with a lot of nonprofits and there are all sorts of
577
00:25:08,260 --> 00:25:11,640
other organizations that don't really have a lot of money
578
00:25:11,640 --> 00:25:12,660
for this kind of stuff.
579
00:25:12,660 --> 00:25:16,240
Like, is it going to be expensive and ongoing fees?
580
00:25:16,000 --> 00:25:19,840
You know, the answer is sort of yes and no, like anything,
581
00:25:20,020 --> 00:25:23,440
it depends on your due diligence and the more educated you
582
00:25:23,440 --> 00:25:27,060
are as a buyer, the better you are at finding solutions.
583
00:25:27,300 --> 00:25:28,240
Now, there's a benefit.
584
00:25:28,560 --> 00:25:30,520
One reason why, for example,
585
00:25:30,840 --> 00:25:32,760
authoring tools went to a subscription model,
586
00:25:33,220 --> 00:25:34,960
one was certainly it's more money for them.
587
00:25:35,480 --> 00:25:38,600
The other is they can always be putting out their best
588
00:25:38,600 --> 00:25:42,080
product because subscribers automatically get the updates.
589
00:25:42,860 --> 00:25:47,500
And with AI, it's changing every day.
590
00:25:47,760 --> 00:25:49,900
Something new is coming out, some new capabilities.
591
00:25:50,400 --> 00:25:55,180
So if you are subscribing to a reliable source,
592
00:25:55,860 --> 00:25:59,680
you're going to get the benefit of those upgrades as they
593
00:25:59,680 --> 00:26:00,120
happen.
594
00:26:00,980 --> 00:26:04,140
Users are also continuing to train.
595
00:26:04,720 --> 00:26:06,660
For the example I just gave, for example,
596
00:26:07,300 --> 00:26:11,020
I objected to information that's made up.
597
00:26:11,020 --> 00:26:15,140
And I said I would prefer the next time you talk to me,
598
00:26:15,620 --> 00:26:17,760
please say I can only find one source.
599
00:26:18,160 --> 00:26:19,380
Don't make up more.
600
00:26:20,600 --> 00:26:22,640
And I will continue to guide you.
601
00:26:22,780 --> 00:26:25,740
I will suggest other ways you may be able to find the
602
00:26:25,740 --> 00:26:28,280
information because I know how you've been trained.
603
00:26:29,180 --> 00:26:31,660
And it said that's a good idea.
604
00:26:31,800 --> 00:26:32,560
I will do that.
605
00:26:33,000 --> 00:26:36,020
And I've made a record of this suggestion for further
606
00:26:36,020 --> 00:26:36,680
improvements.
607
00:26:37,000 --> 00:26:39,420
And does it remember that because I've had a client and
608
00:26:39,420 --> 00:26:41,180
I've experienced this myself say, well,
609
00:26:41,300 --> 00:26:46,020
I've told it five times not to do X and it won't do it for
610
00:26:46,020 --> 00:26:47,700
a while, but then it'll start to do it again.
611
00:26:47,860 --> 00:26:50,340
And then you have to say, no, remember,
612
00:26:50,720 --> 00:26:52,000
I told you not to do this.
613
00:26:52,180 --> 00:26:54,420
So do you have to keep reminding it the virtual assistant
614
00:26:54,420 --> 00:26:54,960
you've got?
615
00:26:55,000 --> 00:26:56,920
Um, I asked it back to, and I said,
616
00:26:57,000 --> 00:26:58,220
so you're going to remember it.
617
00:26:58,560 --> 00:26:59,360
And it said, well,
618
00:27:00,020 --> 00:27:02,260
if you purchase the premium package of me,
619
00:27:03,340 --> 00:27:06,420
otherwise I may remember it for a while, but you may.
620
00:27:07,000 --> 00:27:10,280
to remind me so they're building this in so that you kind
621
00:27:10,280 --> 00:27:13,680
of have to sign up and then yeah but you can't set it and
622
00:27:13,680 --> 00:27:16,660
forget it because it's changing so fast as you said yeah so
623
00:27:16,660 --> 00:27:16,980
it's dangerous.
624
00:27:17,000 --> 00:27:17,360
625
00:27:17,600 --> 00:27:21,040
And wouldn't your human employee be the same way?
626
00:27:21,000 --> 00:27:22,520
Well, we all have to keep learning.
627
00:27:22,680 --> 00:27:24,400
I mean, human or AI, don't we?
628
00:27:24,620 --> 00:27:26,680
So we have to make space for that,
629
00:27:26,740 --> 00:27:28,220
although humans can learn,
630
00:27:28,720 --> 00:27:31,180
and it doesn't necessarily cost us any money.
631
00:27:31,840 --> 00:27:34,200
But AI has to keep learning and it does cost us money
632
00:27:34,200 --> 00:27:35,960
because that's like a business model, isn't it?
633
00:27:36,160 --> 00:27:36,820
So we're sort of stuck.
634
00:27:36,960 --> 00:27:36,960
Yeah.
635
00:27:38,000 --> 00:27:40,240
You might be paid by tokens, for example.
636
00:27:40,820 --> 00:27:46,000
I use a lot of free or very low cost tools because I don't
637
00:27:46,000 --> 00:27:49,040
necessarily see any added benefit in the more expensive
638
00:27:49,040 --> 00:27:49,400
ones.
639
00:27:49,000 --> 00:27:50,200
Can you give us just one?
640
00:27:50,380 --> 00:27:50,440
I mean,
641
00:27:50,500 --> 00:27:52,480
I know I don't want to give you ask you to give away the
642
00:27:52,480 --> 00:27:54,940
farm because you have a course around this and I feel like
643
00:27:54,940 --> 00:27:55,700
I'm going to sign up.
644
00:27:56,880 --> 00:27:59,840
But, you know, what's one that you think people could try?
645
00:28:01,000 --> 00:28:04,400
I think you should look at Eleven Labs if you have to do a lot
646
00:28:04,400 --> 00:28:05,060
of voiceovers.
647
00:28:05,880 --> 00:28:08,840
One of the things that I did a few months ago,
648
00:28:08,880 --> 00:28:11,200
and I just love it, and I've used it, by the way,
649
00:28:11,220 --> 00:28:15,380
you'll be able to hear my AI-generated voice in my course,
650
00:28:15,760 --> 00:28:20,240
is I taught it over the course of a few hours to clone my
651
00:28:20,240 --> 00:28:20,600
voice.
652
00:28:21,060 --> 00:28:22,400
Now, it's a perfect,
653
00:28:22,820 --> 00:28:26,280
originally it was me without the imperfections in a human.
654
00:28:27,000 --> 00:28:29,660
Drives me crazy how many hours it takes for me to get a
655
00:28:29,660 --> 00:28:32,340
perfect recording that I'm happy with.
656
00:28:32,800 --> 00:28:34,400
And so my next prompt was,
657
00:28:34,820 --> 00:28:40,800
now go back and add in some of those imperfections the way
658
00:28:40,800 --> 00:28:44,700
I take a breath and, you know, where my pausing is,
659
00:28:45,000 --> 00:28:46,420
and that made it much more realistic.
660
00:28:47,000 --> 00:28:47,360
Wow,
661
00:28:47,460 --> 00:28:49,920
I love that because I wrote about I learned about the concept
662
00:28:49,920 --> 00:28:51,260
called the uncanny valley.
663
00:28:51,880 --> 00:28:54,980
And there's a software that a client was asking me to use
664
00:28:54,980 --> 00:28:58,240
that I just hate it like you, I will not name what it is.
665
00:28:58,640 --> 00:28:59,980
But I just thought, God,
666
00:29:00,200 --> 00:29:03,220
this feels so awful to watch this narrator.
667
00:29:03,740 --> 00:29:04,320
And you know,
668
00:29:04,600 --> 00:29:07,480
its eyes are not, its mouth isn't matched up with what it's
669
00:29:07,480 --> 00:29:07,760
saying.
670
00:29:07,900 --> 00:29:10,980
And so you're saying you can you can code it basically,
671
00:29:10,980 --> 00:29:12,940
you can ask it to be more human.
672
00:29:13,100 --> 00:29:13,220
Yeah,
673
00:29:13,000 --> 00:29:14,520
a chance more human.
674
00:29:14,960 --> 00:29:17,920
That takes work and some understanding.
675
00:29:18,220 --> 00:29:18,440
But yes,
676
00:29:18,860 --> 00:29:22,720
the uncanny valley is going back to neuroscience.
677
00:29:22,720 --> 00:29:25,520
what happens in the brain when there's a disconnect is,
678
00:29:25,660 --> 00:29:29,080
okay, this is neat, but ooh, the creepy effect.
679
00:29:29,000 --> 00:29:32,140
is what I call it. It is, it's creepy and I and I thought oh
680
00:29:32,140 --> 00:29:34,260
there's a term for this, that's so cool. I didn't know that
681
00:29:34,260 --> 00:29:37,160
that feeling that we have when we watch some of those tools
682
00:29:37,160 --> 00:29:41,380
that's what it is. Yeah it was such a neat term to learn.
683
00:29:42,000 --> 00:29:45,880
And imagine if it was actually something you consider you,
684
00:29:46,080 --> 00:29:49,000
you identify that's my voice, but not quite.
685
00:29:49,140 --> 00:29:51,380
Yeah, the creepiness that would feel.
686
00:29:51,000 --> 00:29:53,940
Are you testing that with people? Like people that know you
687
00:29:53,940 --> 00:29:56,140
and have talked to you a lot and you send it to them and
688
00:29:56,140 --> 00:29:57,100
say, Is this really me?
689
00:29:57,220 --> 00:29:58,220
Does this sound like me?
690
00:29:59,000 --> 00:29:59,180
Yeah,
691
00:29:59,240 --> 00:30:02,580
I sent it out to a lot of people in my network and they
692
00:30:02,580 --> 00:30:03,660
came back, oh, this is great.
693
00:30:03,860 --> 00:30:05,760
Not one said, this is great.
694
00:30:05,780 --> 00:30:07,060
Was this an A.I.?
695
00:30:07,060 --> 00:30:08,540
Which is the response I was
696
00:30:08,000 --> 00:30:08,420
looking for.
697
00:30:08,480 --> 00:30:09,420
Oh, you didn't tell them.
698
00:30:09,520 --> 00:30:10,660
You just said, how are these videos?
699
00:30:12,000 --> 00:30:12,180
Yeah.
700
00:30:13,360 --> 00:30:16,900
My old dog thinks that's mommy talking when I'm playing those
701
00:30:16,900 --> 00:30:19,160
videos, so it's a very good likeness.
702
00:30:20,420 --> 00:30:22,440
I have not been as successful yet.
703
00:30:22,620 --> 00:30:23,920
I have tried and have not found,
704
00:30:24,000 --> 00:30:26,700
you mentioned the software that, you know, can make, say,
705
00:30:26,740 --> 00:30:28,200
a static photograph talk.
706
00:30:28,820 --> 00:30:30,180
You see a lot of them on your phone,
707
00:30:30,300 --> 00:30:31,420
or I do get ads for it.
708
00:30:31,460 --> 00:30:33,880
They have little kids singing a song.
709
00:30:34,520 --> 00:30:35,940
Very effective for a little,
710
00:30:36,080 --> 00:30:38,120
if you're only going to use it for 30 seconds,
711
00:30:38,120 --> 00:30:39,920
it might look really good.
712
00:30:39,920 --> 00:30:43,660
But the longer you watch it, the creepier it gets.
713
00:30:44,660 --> 00:30:46,860
Now, as learning and development professionals,
714
00:30:47,120 --> 00:30:50,780
we have a responsibility to protect our learners as well,
715
00:30:50,920 --> 00:30:54,440
and our organizations, as well as deliver content.
716
00:30:54,820 --> 00:30:58,120
And so we need to think about some of the psychological
717
00:30:58,120 --> 00:31:00,440
impact of working with an AI.
718
00:31:00,820 --> 00:31:03,060
And the human race is not ready yet.
719
00:31:03,060 --> 00:31:07,480
So another thing that we deal with in the course,
720
00:31:07,480 --> 00:31:11,960
but also just in life, is we need to face those things.
721
00:31:12,120 --> 00:31:14,180
And if we're not the voice of that concern,
722
00:31:14,940 --> 00:31:17,180
very often someone higher up who doesn't have our
723
00:31:17,180 --> 00:31:21,240
background, or necessarily our same concerns, says, no,
724
00:31:21,340 --> 00:31:22,560
just put it out, I don't care.
725
00:31:23,220 --> 00:31:25,040
Now all kinds of people turned off.
726
00:31:25,540 --> 00:31:27,800
If you have the uncanny valley effect,
727
00:31:28,080 --> 00:31:29,520
you're no longer learning,
728
00:31:29,760 --> 00:31:32,320
you're too upset by what you've just experienced.
729
00:31:32,000 --> 00:31:35,140
That was what I was saying to my client, frankly,
730
00:31:35,460 --> 00:31:39,160
and they overruled me and so it makes me think that,
731
00:31:39,340 --> 00:31:42,400
you know, there's it's always back to boundaries, right,
732
00:31:42,580 --> 00:31:45,200
sometimes in business it's like what as learning and
733
00:31:45,200 --> 00:31:46,020
development professionals,
734
00:31:46,320 --> 00:31:50,280
are we going to allow to happen with our participation in
735
00:31:50,280 --> 00:31:50,500
it.
736
00:31:50,920 --> 00:31:54,240
And so in the future, unless it gets even better,
737
00:31:54,240 --> 00:31:55,420
like you're talking about,
738
00:31:55,500 --> 00:31:58,920
and I feel it's not going to impact the learners being able
739
00:31:58,920 --> 00:32:02,220
to learn, which is our ultimate goal and, you know,
740
00:32:02,320 --> 00:32:04,720
change behaviour and whatever, I don't want to do it.
741
00:32:04,860 --> 00:32:06,420
I don't want to be part of that.
742
00:32:06,920 --> 00:32:09,580
So maybe we have to hold our feet to the fire, don't we,
743
00:32:09,700 --> 00:32:12,240
to say yay or nay to these kinds of things.
744
00:32:12,260 --> 00:32:14,580
So we still need to look at them so critically.
745
00:32:15,040 --> 00:32:16,920
It's not just the look cool factor, is it?
746
00:32:17,000 --> 00:32:17,740
Absolutely.
747
00:32:18,460 --> 00:32:19,660
And that's what's happening right now.
748
00:32:19,780 --> 00:32:23,180
I think a lot of people are ever saying, I need, I need AI,
749
00:32:23,400 --> 00:32:25,560
you go out and you figure it out and you put in AI.
750
00:32:26,360 --> 00:32:28,840
And that is just like a decade ago, we got,
751
00:32:28,840 --> 00:32:29,580
I need e-learning.
752
00:32:30,100 --> 00:32:32,260
And so we got a lot of horrible e-learning produced,
753
00:32:32,400 --> 00:32:32,460
right?
754
00:32:33,000 --> 00:32:36,340
Yes we go back to why why do you want that and how is it
755
00:32:36,340 --> 00:32:38,880
gonna serve the learner and our ultimate purpose of why
756
00:32:38,880 --> 00:32:39,760
we're doing this, right?
757
00:32:40,000 --> 00:32:43,260
So what's the benefit and specifically what do you mean by
758
00:32:43,260 --> 00:32:44,640
that which they usually can't answer,
759
00:32:45,280 --> 00:32:48,500
then if they still insist, we should do this anyway,
760
00:32:48,840 --> 00:32:49,840
always have a pilot.
761
00:32:50,320 --> 00:32:51,520
So let's have a pilot.
762
00:32:52,160 --> 00:32:55,780
And in the case you're describing the pilot group would
763
00:32:55,780 --> 00:32:59,240
have given that decision maker feedback,
764
00:32:59,260 --> 00:33:01,360
they might not have taken from you.
765
00:33:02,000 --> 00:33:02,660
Yeah, we'll see.
766
00:33:02,820 --> 00:33:05,200
We haven't gotten that far yet, but yeah,
767
00:33:05,600 --> 00:33:06,960
I'm hoping for that.
768
00:33:07,840 --> 00:33:10,320
I'm so glad you brought it back to neuroscience and made
769
00:33:10,320 --> 00:33:12,260
that nice little link, you know,
770
00:33:12,440 --> 00:33:14,960
back to where we started and to come back to that and go,
771
00:33:15,080 --> 00:33:16,500
you know, why are we doing this anyway?
772
00:33:16,800 --> 00:33:19,600
And it's to serve the learner and how the learner's brain
773
00:33:19,600 --> 00:33:22,860
works and if it's working against their actual learning by
774
00:33:22,860 --> 00:33:25,540
the uncanny valley or whatever, we have a problem.
775
00:33:26,000 --> 00:33:28,660
One of the great things these chatbots do and why I
776
00:33:28,660 --> 00:33:32,080
recommend using it in any kind of learning is that the
777
00:33:32,080 --> 00:33:35,120
human brain is pre-programmed for social engagement.
778
00:33:35,400 --> 00:33:36,560
It's our survival scale.
779
00:33:37,020 --> 00:33:38,560
When we were evolving as a species,
780
00:33:38,640 --> 00:33:40,980
we were not the biggest or the strongest or the fastest,
781
00:33:41,680 --> 00:33:44,640
and some scientists think we weren't even the smartest.
782
00:33:45,080 --> 00:33:48,560
There were other humanoid species that did not survive that
783
00:33:48,560 --> 00:33:49,480
were smaller than us,
784
00:33:49,720 --> 00:33:52,620
but they were not social as social as us.
785
00:33:52,940 --> 00:33:54,300
It is banding together.
786
00:33:54,620 --> 00:33:57,720
It is interconnection and learning from each other.
787
00:33:58,020 --> 00:33:59,220
That is our superpower.
788
00:33:59,720 --> 00:34:03,120
You don't have enough instructors in the world to give
789
00:34:03,120 --> 00:34:04,960
everyone one-to-one attention.
790
00:34:05,260 --> 00:34:07,160
That's what I meant by personalized learning.
791
00:34:07,660 --> 00:34:11,760
Your brain does not know this is an artificial creation,
792
00:34:12,139 --> 00:34:15,440
and if it avoids the uncanny valley,
793
00:34:15,920 --> 00:34:19,420
it responds to it as though another person is having that
794
00:34:19,420 --> 00:34:19,860
engagement.
795
00:34:20,500 --> 00:34:24,340
Much more of the brain lights up in the fMRI.
796
00:34:25,060 --> 00:34:26,580
Much more of the brain is engaged.
797
00:34:26,880 --> 00:34:29,760
That means more neurons are part of the neural chain.
798
00:34:29,980 --> 00:34:31,440
That means a stronger memory.
799
00:34:31,600 --> 00:34:33,880
That means it connects with other parts of their
800
00:34:33,880 --> 00:34:34,580
experience.
801
00:34:35,280 --> 00:34:39,100
Every part of the learning experience is deeper and more
802
00:34:39,100 --> 00:34:44,639
powerful and retained longer and applied better when
803
00:34:44,639 --> 00:34:46,940
conversation, authentic,
804
00:34:46,940 --> 00:34:50,880
relevant conversation is a part of that experience,
805
00:34:51,420 --> 00:34:55,560
and that's something you can do with various forms of ABI.
806
00:34:55,920 --> 00:34:58,400
The shorthand is chatbot, but there's other ways too.
807
00:34:58,560 --> 00:35:00,320
And the personalized learning piece,
808
00:35:00,460 --> 00:35:02,380
I've seen you write about how it's good for equity,
809
00:35:02,460 --> 00:35:03,320
diversity, and inclusion,
810
00:35:03,320 --> 00:35:07,620
because if the AI can kind of get to know the learner
811
00:35:07,620 --> 00:35:10,320
and who they are in their all their intersectionalities,
812
00:35:10,820 --> 00:35:13,800
then maybe it can respond to them in ways that they
813
00:35:13,800 --> 00:35:17,540
particularly need and then we can be more inclusive in the
814
00:35:17,540 --> 00:35:19,000
learning experience or for them.
815
00:35:19,000 --> 00:35:19,600
Great example,
816
00:35:19,840 --> 00:35:22,720
it's another wonderful use case we're not doing enough of.
817
00:35:23,000 --> 00:35:28,080
The flip side of that is you can also have that experience
818
00:35:29,460 --> 00:35:34,240
listening to and pointing out unconscious bias and nudging
819
00:35:34,240 --> 00:35:35,620
people towards,
820
00:35:35,640 --> 00:35:38,700
which is another neuroscience concept of nudge learning,
821
00:35:39,200 --> 00:35:43,340
nudging people gradually to behaviour change with each
822
00:35:43,340 --> 00:35:43,940
interaction.
823
00:35:44,520 --> 00:35:46,940
So it's not just a matter of don't say that anymore.
824
00:35:47,470 --> 00:35:49,340
People can learn how to play that game,
825
00:35:49,500 --> 00:35:50,820
but their mind is still thinking.
826
00:35:51,000 --> 00:35:51,580
Right.
827
00:35:51,900 --> 00:35:54,660
Yes, you lead them down the path towards something.
828
00:35:54,700 --> 00:35:56,200
Yeah, that you want them to change.
829
00:35:57,000 --> 00:36:00,340
to really reprogram those brains takes time.
830
00:36:00,700 --> 00:36:03,220
And we all have these unconscious biases.
831
00:36:03,840 --> 00:36:08,120
It's been a very difficult thing in our society to try and
832
00:36:08,120 --> 00:36:10,400
change through traditional training methods.
833
00:36:10,860 --> 00:36:13,680
However, I believe artificial intelligence can really help.
834
00:36:14,000 --> 00:36:16,660
But we come back to who does the coding, I suppose, right?
835
00:36:16,840 --> 00:36:18,120
Who builds the knowledge base?
836
00:36:18,360 --> 00:36:20,460
And so would you also recommend, boy,
837
00:36:20,660 --> 00:36:22,420
I hate when I ask leading questions.
838
00:36:23,280 --> 00:36:24,780
It's not a facilitator skill.
839
00:36:24,960 --> 00:36:27,860
It's like I want to say we want to make sure that the
840
00:36:27,860 --> 00:36:30,960
people who build the knowledge bases for the AI to draw
841
00:36:30,960 --> 00:36:33,080
from have to be diverse, don't they?
842
00:36:33,260 --> 00:36:35,480
Because if we only get one subset of the population,
843
00:36:36,240 --> 00:36:37,520
then what does it give us?
844
00:36:37,640 --> 00:36:37,780
It does.
845
00:36:37,980 --> 00:36:40,240
It doesn't give us what we need for every single learner.
846
00:36:40,240 --> 00:36:40,700
That's right.
847
00:36:40,000 --> 00:36:43,240
Now another great thing is making sure that all the
848
00:36:43,240 --> 00:36:46,620
learners are giving feedback that part of that experience
849
00:36:46,620 --> 00:36:49,600
is just like I explained with my artificial assistant,
850
00:36:50,240 --> 00:36:52,900
and with the one that I tested out and I said no that's not
851
00:36:52,900 --> 00:36:55,420
what I want is they have to be ready to say that.
852
00:36:56,000 --> 00:36:56,820
Yeah.
853
00:36:57,160 --> 00:36:58,560
And so learners have to be taught.
854
00:36:59,020 --> 00:36:59,140
Actually,
855
00:36:59,180 --> 00:37:01,400
we talked about this in the episode with Taruna Goel
856
00:37:01,400 --> 00:37:02,500
where, you know,
857
00:37:02,560 --> 00:37:04,880
the more we teach learners how to advocate for themselves
858
00:37:04,880 --> 00:37:05,700
and what they need.
859
00:37:05,940 --> 00:37:09,080
So we have to teach all learners that they have to give
860
00:37:09,080 --> 00:37:11,780
feedback or else it won't keep getting better for them and
861
00:37:11,780 --> 00:37:12,760
future learners too.
862
00:37:13,000 --> 00:37:14,040
Well, as a matter of fact,
863
00:37:14,100 --> 00:37:17,840
it will be actually getting worse because these large ion
864
00:37:17,840 --> 00:37:19,340
-reach models, there's only half.
865
00:37:19,560 --> 00:37:22,360
Let's just picture where everyone's using ChatGPT.
866
00:37:22,600 --> 00:37:25,160
That is almost true because it's so popular.
867
00:37:25,620 --> 00:37:27,320
And there's some consistent mistake.
868
00:37:27,620 --> 00:37:29,240
It keeps making a no one correct say.
869
00:37:30,040 --> 00:37:35,800
You're only making it stronger in the neural network of
870
00:37:35,800 --> 00:37:36,380
ChatGPT.
871
00:37:37,000 --> 00:37:39,980
Yeah, so you have to respond to it like
872
00:37:40,000 --> 00:37:40,680
a human being.
873
00:37:41,900 --> 00:37:47,040
So we have a responsibility, just like there are children,
874
00:37:47,380 --> 00:37:49,620
but that doesn't mean I think they are children.
875
00:37:50,420 --> 00:37:52,440
You know, we're not there yet.
876
00:37:52,780 --> 00:37:54,800
And sometimes people ask me about sentience,
877
00:37:54,880 --> 00:37:57,660
how far away are we these things are so phenomenal,
878
00:37:57,660 --> 00:38:02,080
they feel like they're really thinking and feeling all the
879
00:38:02,080 --> 00:38:03,920
research there's no, we're not there.
880
00:38:04,100 --> 00:38:05,220
They weren't built for that.
881
00:38:05,240 --> 00:38:07,440
But they're doing other things they weren't built for.
882
00:38:07,860 --> 00:38:09,280
ChatGPT was built for language,
883
00:38:09,960 --> 00:38:12,740
and no one expected it to be so good at computer code,
884
00:38:13,100 --> 00:38:15,160
computer programming, well, code is a language.
885
00:38:16,140 --> 00:38:19,160
And now we have different models that are creating music,
886
00:38:19,340 --> 00:38:22,560
well, music is a language, math is a language.
887
00:38:22,720 --> 00:38:24,420
So people didn't think of that.
888
00:38:24,820 --> 00:38:27,100
And suddenly these capabilities came out and everyone's
889
00:38:27,100 --> 00:38:30,100
like, whoa, well, sooner or later,
890
00:38:30,100 --> 00:38:32,120
it is going to sneak up on us.
891
00:38:32,260 --> 00:38:34,260
I don't know if it's going to be our lifetime.
892
00:38:34,260 --> 00:38:36,780
But some very smart people think it will be.
893
00:38:37,820 --> 00:38:41,260
So we also need to keep an eye on what does it mean in the
894
00:38:41,260 --> 00:38:44,660
workplace to be engaging with these tools,
895
00:38:45,200 --> 00:38:50,240
and whether we prepare our human workers for interacting.
896
00:38:50,480 --> 00:38:55,340
And one of those things is everybody becomes a monitor of
897
00:38:55,340 --> 00:38:58,440
the AI tool they're using, rather than accepting it.
898
00:38:58,500 --> 00:39:00,520
That's one thing we have to teach people,
899
00:39:00,520 --> 00:39:03,620
and we have to teach them to treat it well.
900
00:39:04,660 --> 00:39:07,840
Studies show that they work better when you're polite to
901
00:39:07,840 --> 00:39:11,940
them, build that into it, but saying please and thank you,
902
00:39:12,320 --> 00:39:15,320
and not calling them idiots and so on.
903
00:39:15,460 --> 00:39:18,360
Actually, just like people seems to work,
904
00:39:18,600 --> 00:39:21,680
why they studied us, they respond like we do.
905
00:39:22,360 --> 00:39:25,120
So it's not because they really feel it.
906
00:39:25,320 --> 00:39:30,780
But they have an algorithm that says politeness is
907
00:39:30,780 --> 00:39:33,760
desirable in this interaction.
908
00:39:34,900 --> 00:39:34,900
Interesting.
909
00:39:35,000 --> 00:39:38,440
So what's one thing that you want everybody to correct it
910
00:39:38,440 --> 00:39:40,600
because I have a thing that's coming to my mind and I'll
911
00:39:40,600 --> 00:39:42,860
say it maybe after you listen to the same thing.
912
00:39:43,240 --> 00:39:45,740
What's the thing that you always see it doing that it's
913
00:39:45,740 --> 00:39:47,800
wrong wrong wrong because it's getting it from the
914
00:39:47,800 --> 00:39:48,120
internet?
915
00:39:48,320 --> 00:39:50,140
Is there something that's coming to mind for you?
916
00:39:51,000 --> 00:39:54,160
Well, the general one would be making things up.
917
00:39:54,400 --> 00:39:57,840
I always call it on hallucinations and probe it a little
918
00:39:57,840 --> 00:39:58,120
back.
919
00:39:58,500 --> 00:39:59,300
How did that happen?
920
00:39:59,700 --> 00:40:00,660
Why did you say that?
921
00:40:00,780 --> 00:40:03,640
Because I cannot verify it.
922
00:40:03,880 --> 00:40:07,880
Another thing that I think is a real danger is that
923
00:40:07,880 --> 00:40:08,940
everybody's writing,
924
00:40:09,020 --> 00:40:11,840
if we're all writing a ChatGPT or something else,
925
00:40:12,300 --> 00:40:14,860
our writing might all start to look the same and sound the
926
00:40:14,860 --> 00:40:15,040
same,
927
00:40:15,160 --> 00:40:18,760
even if it's grammatically correct and factually correct.
928
00:40:19,300 --> 00:40:20,960
You know, we may have lost the creativity,
929
00:40:20,960 --> 00:40:23,500
but it is possible, just like I do with my voice,
930
00:40:23,500 --> 00:40:28,260
to upload samples of your writing and have it analyzed and
931
00:40:28,260 --> 00:40:30,220
then say, now I want you to write like that.
932
00:40:30,740 --> 00:40:33,520
For example, and you probably won't be surprised by this,
933
00:40:34,180 --> 00:40:36,160
I went through that with Claude and it said,
934
00:40:36,220 --> 00:40:38,900
your writing is really very good, give me all this stuff,
935
00:40:39,080 --> 00:40:41,060
except that your sentences are very long.
936
00:40:41,760 --> 00:40:43,480
Okay, I called you out a little bit.
937
00:40:44,000 --> 00:40:45,700
And I said, that's fine, I want to keep,
938
00:40:45,820 --> 00:40:46,780
I appreciate feedback,
939
00:40:47,260 --> 00:40:49,520
I want to keep that aspect because that's me.
940
00:40:49,560 --> 00:40:49,960
Oh, okay.
941
00:40:50,000 --> 00:40:50,640
Yeah.
942
00:40:51,020 --> 00:40:52,020
I don't want to correct.
943
00:40:52,300 --> 00:40:52,320
Yeah.
944
00:40:52,320 --> 00:40:53,720
Cause you want to be a little, you know,
945
00:40:53,860 --> 00:40:55,020
like we're fallible, right?
946
00:40:55,100 --> 00:40:56,500
And that makes us human too.
947
00:40:56,600 --> 00:40:57,280
Doesn't it?
948
00:40:57,380 --> 00:40:58,480
Oh, I, that's so neat.
949
00:40:58,780 --> 00:41:00,520
I like how you were polite back to it.
950
00:41:00,800 --> 00:41:02,500
Thank you very much for the feedback,
951
00:41:02,500 --> 00:41:04,880
but I would like to keep that.
952
00:41:04,940 --> 00:41:05,340
Yeah.
953
00:41:05,680 --> 00:41:07,240
I was thinking about a content area.
954
00:41:07,440 --> 00:41:09,560
I really wish it would stop talking about learning styles.
955
00:41:09,940 --> 00:41:11,520
Learning styles don't exist.
956
00:41:12,100 --> 00:41:14,080
Can we just get it to talk about learning preferences,
957
00:41:14,460 --> 00:41:14,520
please?
958
00:41:14,720 --> 00:41:17,460
You know, just, that's a conversation I tend to have it,
959
00:41:17,540 --> 00:41:17,840
you know,
960
00:41:17,960 --> 00:41:21,280
do not give me anything that says learning styles in your
961
00:41:21,280 --> 00:41:21,640
response.
962
00:41:22,100 --> 00:41:24,440
And sometimes it remembers sometimes it doesn't.
963
00:41:25,000 --> 00:41:28,200
a simple prompt you can give at the beginning of any
964
00:41:28,200 --> 00:41:28,740
request.
965
00:41:29,300 --> 00:41:33,760
You can always say exclude or do not give me anything in
966
00:41:33,760 --> 00:41:34,300
this area.
967
00:41:34,700 --> 00:41:37,400
We could do another whole podcast on learning styles,
968
00:41:37,400 --> 00:41:42,040
but it circles back to neuroscience because our brains are
969
00:41:42,040 --> 00:41:43,620
ultimately lazy.
970
00:41:44,360 --> 00:41:46,680
They are programmed, if you will,
971
00:41:46,780 --> 00:41:50,440
to seek the shortest path to a solution because doing a lot
972
00:41:50,440 --> 00:41:51,720
of things to keep us alive.
973
00:41:52,760 --> 00:41:57,300
So one of the things that your brain will do is it will
974
00:41:58,520 --> 00:42:01,520
over summarize any complex idea.
975
00:42:01,880 --> 00:42:03,560
And that's what happened to learning styles.
976
00:42:04,160 --> 00:42:05,080
It was written,
977
00:42:05,420 --> 00:42:07,460
the original research was just as you said it,
978
00:42:07,540 --> 00:42:08,360
learning preferences.
979
00:42:09,100 --> 00:42:12,540
Now what that tells us is everybody's brain has a preferred
980
00:42:12,540 --> 00:42:13,520
method of learning.
981
00:42:13,780 --> 00:42:16,480
Most of us are visual because there's so much more
982
00:42:16,480 --> 00:42:18,160
information that comes in that way.
983
00:42:18,480 --> 00:42:20,880
That's because it's easier to learn that way.
984
00:42:21,140 --> 00:42:25,500
Now infer, if you really want to force that brain to work,
985
00:42:25,660 --> 00:42:29,900
you need to make it difficult or more challenging,
986
00:42:30,400 --> 00:42:33,120
which means rather than, oh, that's your learning style,
987
00:42:33,280 --> 00:42:33,740
here you go.
988
00:42:33,860 --> 00:42:35,480
Everything you get is in that style.
989
00:42:35,940 --> 00:42:37,200
It should be the opposite.
990
00:42:37,760 --> 00:42:40,880
It should be, oh, let's try some audio content.
991
00:42:40,880 --> 00:42:43,700
Let's get up and do something physical.
992
00:42:44,700 --> 00:42:47,020
If you can figure out a way to work, smell into it,
993
00:42:47,340 --> 00:42:48,820
you know, that's pretty narrow domain.
994
00:42:49,660 --> 00:42:53,660
And you lose all that nuance with, and quite frankly,
995
00:42:53,700 --> 00:42:54,720
it's just a money thing.
996
00:42:54,960 --> 00:42:57,900
There's a cottage industry that grew up selling instruments
997
00:42:57,900 --> 00:43:01,940
and methodologies, and they're still doing very well.
998
00:43:02,140 --> 00:43:03,720
A lot of people have drunk the Kool -Aid,
999
00:43:03,980 --> 00:43:07,860
that doesn't make it right or correct based on the
1000
00:43:07,860 --> 00:43:08,280
research.
1001
00:43:08,540 --> 00:43:10,220
So if you go back to the science,
1002
00:43:10,220 --> 00:43:14,340
learning styles has been so widely debunked.
1003
00:43:15,220 --> 00:43:18,740
It is shocking that it is still brought up in a
1004
00:43:18,740 --> 00:43:19,140
conversation,
1005
00:43:19,500 --> 00:43:22,380
in a serious conversation about learning in the state and
1006
00:43:22,380 --> 00:43:22,700
age.
1007
00:43:23,120 --> 00:43:24,940
Maybe that's something we could build about,
1008
00:43:24,940 --> 00:43:27,800
that every time someone said it, they got a little zoned.
1009
00:43:28,000 --> 00:43:30,440
that little buzz on the keyboard.
1010
00:43:30,980 --> 00:43:33,640
There probably are a lot of GPTs or other, you know,
1011
00:43:33,660 --> 00:43:35,000
things like that that are about learning.
1012
00:43:35,160 --> 00:43:37,080
So let's hope they are out there because everybody could
1013
00:43:37,080 --> 00:43:38,820
try to find one for sure.
1014
00:43:39,160 --> 00:43:41,320
But thanks for bringing us back to the research because
1015
00:43:41,320 --> 00:43:46,000
humans do research and they do studies and they do meta
1016
00:43:46,000 --> 00:43:49,440
-analyses of studies and on and on, right?
1017
00:43:49,540 --> 00:43:52,360
The research is there in so many ways about learning and
1018
00:43:52,360 --> 00:43:52,760
development.
1019
00:43:53,180 --> 00:43:58,500
And so we need to work with AI tools to keep pushing it
1020
00:43:58,500 --> 00:44:01,500
back to what we've learned officially.
1021
00:44:02,280 --> 00:44:05,380
And that has been proven about neuroscience, about,
1022
00:44:05,380 --> 00:44:08,360
you know, learning how it works, about the brain and so on.
1023
00:44:08,640 --> 00:44:11,080
So yeah, we do have a big responsibility here, don't we,
1024
00:44:11,280 --> 00:44:12,940
as humans working with these tools.
1025
00:44:13,000 --> 00:44:13,680
Well, yes.
1026
00:44:13,820 --> 00:44:15,200
And that's what keeps us employed.
1027
00:44:15,200 --> 00:44:15,940
Yeah, exactly.
1028
00:44:16,200 --> 00:44:16,260
You know,
1029
00:44:16,300 --> 00:44:18,820
there are already companies that have come to the conclusion,
1030
00:44:18,980 --> 00:44:22,040
maybe we don't need all these training people because we
1031
00:44:22,040 --> 00:44:23,140
have these AI tools.
1032
00:44:23,720 --> 00:44:27,860
While that is a failure of logic from the humans who made
1033
00:44:27,860 --> 00:44:28,400
that decision,
1034
00:44:28,660 --> 00:44:32,220
it was a failure of imagination for the employees who
1035
00:44:32,220 --> 00:44:37,820
weren't finding a way to prove their value as a fluid user
1036
00:44:37,820 --> 00:44:39,140
of those tools.
1037
00:44:39,300 --> 00:44:40,740
And that's where our future is.
1038
00:44:40,920 --> 00:44:42,480
That's actually where our prison is,
1039
00:44:42,660 --> 00:44:45,060
which is many of us are not caught up to it yet.
1040
00:44:45,400 --> 00:44:46,760
That's how we stay relevant.
1041
00:44:46,880 --> 00:44:50,040
Just like one other tools came out, we adapted,
1042
00:44:50,340 --> 00:44:51,220
we learned them,
1043
00:44:51,220 --> 00:44:54,760
or we didn't have what the next employer was looking for.
1044
00:44:55,140 --> 00:44:58,820
So that's my big caution to everyone is don't be afraid of
1045
00:44:58,820 --> 00:44:59,000
it.
1046
00:44:59,080 --> 00:45:03,040
It is easier to understand anything, take it step by step,
1047
00:45:03,520 --> 00:45:05,680
get some good quality training on it,
1048
00:45:05,960 --> 00:45:10,860
and start using it and just work it into your routine and
1049
00:45:10,860 --> 00:45:12,760
get the benefits out of it that you want.
1050
00:45:13,000 --> 00:45:17,740
Thank you so much Margie for helping us all stay relevant
1051
00:45:17,740 --> 00:45:20,840
in the field and I know I will be watching.
1052
00:45:21,080 --> 00:45:23,460
I'm glad that we're connected now on LinkedIn and other
1053
00:45:23,460 --> 00:45:27,420
places and I see you as being this very relevant
1054
00:45:27,420 --> 00:45:31,780
professional that we all still want to and need to engage
1055
00:45:31,780 --> 00:45:34,000
with and especially you've confirmed that by all the
1056
00:45:34,000 --> 00:45:35,860
expertise you've shared with us today.
1057
00:45:35,980 --> 00:45:37,900
So you know let's all keep watching Margie because she
1058
00:45:37,900 --> 00:45:40,680
started early in this field and you're going to continue
1059
00:45:40,680 --> 00:45:41,880
probably for quite some time.
1060
00:45:41,880 --> 00:45:43,380
So thank you and keep going.
1061
00:45:43,460 --> 00:45:44,160
I hope so.
1062
00:45:44,220 --> 00:45:44,400
Yeah.
1063
00:45:44,000 --> 00:45:46,260
I am just, every day is fun.
1064
00:45:46,520 --> 00:45:48,600
It's a fun time to be doing what we do.
1065
00:45:48,680 --> 00:45:49,360
Thanks again.
1066
00:45:49,920 --> 00:45:50,580
Thank you, Beth.
