TRANSCRIPT 🔗
WEBVTT
00:00:07.599 --> 00:00:09.839
Welcome to the human side of results.
00:00:09.839 --> 00:00:10.560
I'm Dr.
00:00:10.560 --> 00:00:11.359
Pillay.
00:00:11.359 --> 00:00:15.839
On this podcast, we explore a simple but difficult question.
00:00:15.839 --> 00:00:24.960
What does it really take to turn what people know into what they do and what they do into meaningful results?
00:00:24.960 --> 00:00:29.600
And today's guest has spent years working on exactly that problem.
00:00:29.600 --> 00:00:38.320
Melanie Martinelli is CEO of the Institute for Transfer Effectiveness and founder of Going Beyond Training.
00:00:38.320 --> 00:00:42.960
Her work focuses on one of the most persistent challenges in learning and development.
00:00:42.960 --> 00:00:46.320
What happens after the training ends?
00:00:46.320 --> 00:00:49.759
How do we get people to actually apply what they've learned?
00:00:49.759 --> 00:00:56.719
And perhaps even more importantly, how do we create the conditions around them that make application possible?
00:00:56.719 --> 00:01:03.280
Melanie, I've been looking forward to this conversation because you and I care deeply about the same problem.
00:01:03.280 --> 00:01:06.000
Welcome to the human side of results.
00:01:06.319 --> 00:01:07.200
Thank you so much.
00:01:07.200 --> 00:01:08.719
I'm really excited to be here.
00:01:08.719 --> 00:01:10.239
Finally, this is happening.
00:01:10.239 --> 00:01:13.439
And uh thank you so much for that wonderful introduction.
00:01:13.760 --> 00:01:16.400
Well, first thing I want to know is where in the world are you?
00:01:16.400 --> 00:01:17.920
Because you are a globetrotter.
00:01:18.000 --> 00:01:19.120
I don't know where you are right now.
00:01:19.120 --> 00:01:20.319
I am actually right home.
00:01:20.319 --> 00:01:23.359
You actually, what you see, I mean, I know people can't see me, but you do.
00:01:23.359 --> 00:01:25.439
You actually see me right in Switzerland.
00:01:25.439 --> 00:01:28.239
I am, I just got back from London yesterday night.
00:01:28.239 --> 00:01:36.640
I'm off on a holiday with my family tomorrow in the Swiss mountains, but right now I am in my hometown, Basel, Switzerland.
00:01:36.959 --> 00:01:37.760
That's awesome.
00:01:37.760 --> 00:01:44.000
Well, just uh so you know where I am in the world, I'm in Austin, Texas, and uh it's exciting to be speaking with you way.
00:01:44.640 --> 00:01:46.480
Can I tell you that I have family in Texas?
00:01:46.480 --> 00:01:48.560
I have family in San Antonio.
00:01:48.560 --> 00:01:50.159
Yeah, I have family in San Antonio.
00:01:50.159 --> 00:01:51.840
I know.
00:01:51.840 --> 00:01:53.840
I've been to the Austin Airport.
00:01:55.599 --> 00:01:56.400
There you go.
00:01:57.280 --> 00:01:59.840
Oh, this is this is so it's so exciting already.
00:01:59.840 --> 00:02:04.400
I just want to make sure you know, Melanie, before I start, that I am a fan of your work.
00:02:04.400 --> 00:02:06.879
And I've been following you and uh Dr.
00:02:06.879 --> 00:02:09.039
Aina uh for for years.
00:02:09.039 --> 00:02:12.240
You guys are top, top, top of the tier.
00:02:12.240 --> 00:02:14.319
And uh I'm excited for what you're gonna share.
00:02:14.319 --> 00:02:16.159
So let's just go ahead and jump in.
00:02:16.159 --> 00:02:19.280
But let's start with let's start with a little bit of history.
00:02:19.280 --> 00:02:21.039
Let's what is your story?
00:02:21.039 --> 00:02:22.800
You know, you've been in this space.
00:02:22.800 --> 00:02:26.159
No, seriously, you you've been in this space for a while.
00:02:26.159 --> 00:02:27.599
You've done great things.
00:02:27.599 --> 00:02:31.280
You're sort of the integrator of this industry for us, really.
00:02:31.280 --> 00:02:32.960
You bring everyone together.
00:02:32.960 --> 00:02:39.199
Tell me a little bit about what got you started in learning transfer and in this space in general.
00:02:39.599 --> 00:02:43.280
Happy to do that because it's not the typical, I think, route.
00:02:43.280 --> 00:02:45.840
I never thought I'm gonna end up in there learning and development.
00:02:45.840 --> 00:02:49.280
I didn't even know learning and development existed when I studied.
00:02:49.280 --> 00:02:59.280
I studied international business management with a focus on finance, yes, uh, which kind of links nicely back to your podcast, you know, results, finance and stuff.
00:02:59.280 --> 00:03:01.599
Um and I also study culture.
00:03:01.599 --> 00:03:05.759
So I was very much focusing on cross-cultural collaborations as a student.
00:03:05.759 --> 00:03:14.240
And at the end of my studies, I had the opportunity to go and do research on how cultural differences impact outsourcing projects.
00:03:14.240 --> 00:03:20.000
This was 2004, uh, outsourcing, especially from Europe to India, started to boom.
00:03:20.000 --> 00:03:26.800
And we wanted to find out are you really saving money when you outsource work or are the cultural differences too big?
00:03:26.800 --> 00:03:31.520
So, long story short, I was sent to India for four months of research and I never came back.
00:03:31.520 --> 00:03:35.120
Well, now I did, but for 18 years, I stayed in India.
00:03:35.120 --> 00:03:44.400
I did research and uh I got a job offer to work in an Indian IT company as the head of sales and marketing.
00:03:44.400 --> 00:03:49.680
Uh, and as part of that job, I was buying training as well for my team.
00:03:49.680 --> 00:03:57.599
And kind of with my finance hat, when I started to buy training, I of course always wondered so how will I know that training worked?
00:03:57.599 --> 00:04:02.159
These vendors are charging me a bomb, you know, they're charging me a lot of money for training.
00:04:02.159 --> 00:04:04.319
How will I really know it worked?
00:04:04.319 --> 00:04:10.719
And I was a little bit disappointed to get happy sheets because as a finance person, I mean, happy sheets didn't really cut it for me.
00:04:10.719 --> 00:04:15.120
But I also observed
Load full transcript
that very often people didn't apply what they've learned.
00:04:15.120 --> 00:04:18.000
You know, I spent money on training, but I didn't see the change.
00:04:18.000 --> 00:04:20.319
So I got very intrigued.
00:04:20.319 --> 00:04:25.360
And uh as I was also doing my research, I started teaching at the university.
00:04:25.360 --> 00:04:29.199
So there was a little bit of that teaching, you know, element coming slowly in.
00:04:29.199 --> 00:04:39.199
Um, and then I stumbled across research of Prinkorf, the work of um the Kirkpatricks, and kind of started to get very curious.
00:04:39.199 --> 00:04:43.920
Somehow, Jim Kirkpatrick was in Dubai, which is not really far from India.
00:04:43.920 --> 00:04:48.240
Uh, and I flew to uh Dubai to meet Jim, yeah.
00:04:48.240 --> 00:04:50.319
Um, and I think that tells you a lot about me.
00:04:50.319 --> 00:04:52.079
I'm just really a human connector.
00:04:52.079 --> 00:04:53.199
I like to kind of meet people.
00:04:53.199 --> 00:05:01.920
So I got into a plane, flew to um Dubai, learned about evaluation, and I started to do work for the Kirkpatricks in Asia.
00:05:01.920 --> 00:05:12.399
Um, and I quit my corporate job and actually, together with other colleagues, started a training business on cross-cultural training, uh, but really trying to make sure those trainings work.
00:05:12.399 --> 00:05:20.000
And so I used a lot of the Kirkpatrick methodology to evaluate whether training worked, did a lot of impact studies as well.
00:05:20.000 --> 00:05:20.800
And guess what?
00:05:20.800 --> 00:05:24.399
Most of the time I found out that training didn't work.
00:05:24.399 --> 00:05:29.600
And it wasn't, you know, the impact we wanted didn't happen.
00:05:29.600 --> 00:05:35.839
And so then I started digging deeper into what is required to make training work.
00:05:35.839 --> 00:05:38.560
And that's when I stumbled across the work of Dr.
00:05:38.560 --> 00:05:44.720
Ina Weinbauer-Heidel, the founder of our institute that I then took over two years ago.
00:05:44.720 --> 00:05:48.240
And uh yeah, in a nutshell, that's it.
00:05:48.240 --> 00:06:03.199
So kind of finance business, buying training, and wondering whether the training I buy actually delivers results, to then realizing, hey, there's a real problem our industry has, let me try and be part of the solution.
00:06:03.519 --> 00:06:05.279
Well, you're not just part of the solution.
00:06:05.279 --> 00:06:07.839
You know, you mentioned you mentioned that you're a connector.
00:06:07.839 --> 00:06:09.120
I'm gonna tell you what you are.
00:06:09.120 --> 00:06:15.360
You are also extremely passionate about this work, and it shows and it's it's very contagious.
00:06:15.360 --> 00:06:24.720
And I have to ask you though, as you also said just now, in a world where a lot of training just doesn't work, okay, let's just be honest.
00:06:24.720 --> 00:06:26.639
You know, there's a lot of data that shows that.
00:06:26.639 --> 00:06:33.040
But we know that training can work, and you're one of those who really is powering this new movement.
00:06:33.040 --> 00:06:42.879
My question for you is why is it that it's we are still so much better in learning and development at measuring training, but not at measuring results?
00:06:42.879 --> 00:06:46.720
Why is it that we just don't measure what happens after training?
00:06:47.279 --> 00:06:48.399
That's a good question.
00:06:48.399 --> 00:06:51.199
I mean, I I there's multiple things going in there.
00:06:51.199 --> 00:06:58.240
First of all, I think part of it is as human beings, and you know, I wouldn't consider myself per se lazy.
00:06:58.240 --> 00:07:06.240
I'm a very active person, but I think the human race, we are a little bit lazy, you know, we go over the stuff that's easy, uh, which requires less effort.
00:07:06.240 --> 00:07:14.399
And for example, measuring things like uh completion or satisfaction or attendance numbers, it's easy.
00:07:14.399 --> 00:07:19.920
Whereas when you kind of start getting into the field of application, it starts to get messy.
00:07:19.920 --> 00:07:30.560
Learning and development is like in the traditional way, even though I would like to more maybe call it learning and performance, because that's when we already start kind of segueing maybe and expand our focus.
00:07:30.560 --> 00:07:43.519
But if you traditionally look at learning and development, um, it's very contained, you know, be it an e-learning, be it a face-to-face, whatever course you design, it's very easy to control that, what happens in there.
00:07:43.519 --> 00:07:52.240
But when people then leave the e-learning, they go back to work, or they leave the Zoom meeting or the training room, we are not really part of that.
00:07:52.240 --> 00:07:54.240
You know, we are not with them every day.
00:07:54.240 --> 00:08:00.959
And so things start to get messy, and messy is scary, messy is complicated, it requires effort.
00:08:00.959 --> 00:08:03.600
And so I think that is one reason.
00:08:03.600 --> 00:08:10.560
And I think the other reason, though, also is, and I saw that myself, and I try to find solutions.
00:08:10.560 --> 00:08:23.759
I think the transfer researchers who've been, you know, doing work for hundreds of years on this topic have actually done a bad job in translating some of this work for us as practitioners.
00:08:23.759 --> 00:08:35.360
Because I think if you go deeper, I think I also felt very overwhelmed, and I know Ina did as well when she went in her research, but she then turned it around, and now we can benefit from it, but I'm sure we'll get to that.
00:08:35.360 --> 00:08:41.200
But if you look at hundreds of years of transfer research, there are so many factors impacting transfer.
00:08:41.200 --> 00:08:44.080
And researchers talk about all of them.
00:08:44.080 --> 00:08:49.279
And but how are you and I as a practitioner supposed to make sense of hundreds of factors?
00:08:49.279 --> 00:08:50.399
Not possible.
00:08:50.399
--> 00:08:57.200
And so, to be fair enough as well, I think it's also not really been easy.
00:08:57.200 --> 00:09:02.639
Uh, like the information wasn't really available in a digestible practical way.
00:09:02.639 --> 00:09:14.799
Part of it is it was just easier to focus on other stuff, and also one last point though, I really want to stress a lot of people didn't have the urgency, the what's in it for me.
00:09:14.799 --> 00:09:22.720
Like, I think when I look at the situation right now, and I'm in the privileged position that my business is doing really well right now.
00:09:22.720 --> 00:09:30.000
Uh, sometimes I feel kind of bad, sometimes I feel relieved because I fought so hard for many years to try and get attention on this topic.
00:09:30.000 --> 00:09:38.639
But when economies were doing well, and we've been in, you know, multiple years now where it's a little bit harder, people didn't care.
00:09:38.639 --> 00:09:40.799
Enough money to spend.
00:09:40.799 --> 00:09:47.840
And the thing is, to be honest, I get every single L and D professional who tells me, Why should I start measuring these results?
00:09:47.840 --> 00:09:49.120
No one is asking me for it.
00:09:49.120 --> 00:09:52.799
All that I know are completion numbers, you know.
00:09:52.799 --> 00:09:59.759
So I don't only want to blame L and D for this, I think the business had its share to play in this as well.
00:10:00.240 --> 00:10:00.960
Absolutely.
00:10:00.960 --> 00:10:12.960
And you know, I have to tell you, um, first of all, I have a view that is similar to what you're seeing, but I have a little analogy that I think points to this being the most exciting time in this industry that we're talking about.
00:10:12.960 --> 00:10:14.159
I'll tell you what that is.
00:10:14.159 --> 00:10:18.320
You're aware of uh Tesla, of course, yeah, electric uh vehicles.
00:10:18.320 --> 00:10:28.159
Now, one thing that a lot of people may not uh realize is that the very first uh vehicle that was invented a hundred years ago or whatever was electric.
00:10:28.159 --> 00:10:32.639
It was the the technology has been available since the very beginning.
00:10:32.639 --> 00:10:44.320
And then you wonder wait, why do wait a second, why did it take uh eighty to a hundred years uh for everybody to be on uh internal combustion engines, ice, yeah, and nobody was doing electric.
00:10:44.320 --> 00:10:44.960
Why is that?
00:10:44.960 --> 00:10:49.440
Well, it's not because it was impossible, it was because no one had made it easy.
00:10:49.440 --> 00:10:50.399
Right?
00:10:50.399 --> 00:10:54.559
Yeah, and so all of a sudden Tesla makes it easy, shows you it can happen.
00:10:54.559 --> 00:10:55.519
Yeah, guess what?
00:10:55.519 --> 00:10:57.840
Everybody's making electric vehicles right now.
00:10:57.840 --> 00:11:14.240
So, you know, I actually think that some of the big players, uh Melanie, uh in the L in the uh in the you know, you know, in the in the training world, software and development
are probably gonna see you and and the work that I'm doing and others are doing.
00:11:14.240 --> 00:11:21.039
And when they realize that it's a it's a new focus because it is changing, people are beginning to want what results provide.
00:11:21.039 --> 00:11:22.960
I think there's gonna be great opportunities.
00:11:22.960 --> 00:11:24.159
So I agree with you.
00:11:24.480 --> 00:11:35.360
And I also want to bring in just one other aspect, you know, like I mean, kind of building because I've seen this, having done work globally, I've seen certain industries being a lot more focused on impact for a long time.
00:11:35.360 --> 00:11:44.399
For example, NGOs I worked with, because they had to operate with limited budgets, proved very often to their donors that what they were doing was working.
00:11:44.399 --> 00:11:46.639
So they had that pressure.
00:11:46.639 --> 00:11:51.679
Uh, also government agencies, health care, healthcare, exactly.
00:11:51.679 --> 00:11:52.720
Because guess what?
00:11:52.720 --> 00:11:53.919
The stakes are very high.
00:11:54.000 --> 00:11:57.120
If uh you're training nurses, people can live or die.
00:11:57.440 --> 00:11:57.919
Exactly.
00:11:57.919 --> 00:12:02.480
Army, you know, you know, defense forces, yeah, yeah, and that as well.
00:12:02.480 --> 00:12:09.440
I mean, if not, you know, if at the Navy, you know, the armed forces, if the training doesn't work, well, there's a risk people come back in a body bag.
00:12:09.440 --> 00:12:11.360
And so the stakes were a lot higher.
00:12:11.360 --> 00:12:14.879
Because I think the corporate world has been a little bit spoiled.
00:12:14.879 --> 00:12:38.399
And then though, there's also, and this is a very provocative statement, uh, and I'm gonna say it, um, is I think also for a long time, if you really think about it, for a lot of LD providers, they actually make it keep them busy to retrain people, it keeps them in, you know, and especially when you think about some of the large providers.
00:12:38.399 --> 00:12:47.600
I've had discussions with people, they actually told me, well, actually, I mean, if we are implementing what you're teaching us, aren't we actually gonna put ourselves out of business?
00:12:47.600 --> 00:12:55.519
Because if we are teaching, training people in such a way that they actually do apply what we learned, then we don't retrain them anymore.
00:12:55.519 --> 00:13:00.159
So I'm gonna scratch my head and say, there's some truth in that, sad truth.
00:13:00.159 --> 00:13:06.399
Uh, but I would invite, and I think it's such exciting times now, especially with AI as well.
00:13:06.399 --> 00:13:11.759
You know, there's a real opportunity for LD and instructional designers to expand our work.
00:13:11.759 --> 00:13:22.159
To say, okay, we are not just training people, we are not just developing courses, we are building ecosystems for change to take place, for transfer to occur.
00:13:22.159 --> 00:13:26.399
We are building solutions to get supervisors on board because they are an important factor.
00:13:26.399 --> 00:13:29.440
We are the ones removing obstacles to transfer.
00:13:29.440 --> 00:13:36.159
And so I think, yes, short term, you might be doing less training, but it doesn't mean that you're doing less work.
00:13:36.159 --> 00:13:41.279
It's just that your work is shifting, in my opinion, to something that is adding more value.
00:13:41.679 --> 00:13:42.480
Absolutely.
00:13:42.480 --> 00:13:54.080
You know, let's talk a little bit about the science because you are a practitioner, you are not a theorist, you get things done based on the research, the the you know, evidence, and all that stuff.
00:13:54.080 --> 00:13:57.039
Talk about the 12 levers of transfer effectiveness.
00:13:57.039 --> 00:14:02.879
Give us a quick, you know, obviously we don't, we we can't do the whole thing or else we have to come with Switzerland, right?
00:14:03.679 --> 00:14:06.240
And I love it that you're not yeah, yeah.
00:14:06.639 --> 00:14:07.919
Give us a quick overview.
00:14:07.919 --> 00:14:11.519
What exactly is the 12 levers of transfer effectiveness?
00:14:11.759 --> 00:14:12.720
And you're probably gonna hear me.
00:14:12.720 --> 00:14:17.919
I always it always makes me laugh because yeah, in the US you guys say levers, while we in Europe say levers.
00:14:17.919 --> 00:14:25.279
So even though I keep trying, but this actually shows you straight away one uh lever is opportunities for application.
00:14:25.279 --> 00:14:34.320
And since I do a lot more work in Asia and here in Europe, where we speak more British English, I say lever a lot more than levers.
00:14:34.320 --> 00:14:36.960
Because I don't have an opportunity that often.
00:14:36.960 --> 00:14:39.840
But yeah, in a nutshell, the 12 levers.
00:14:39.840 --> 00:14:41.840
See, I'm trying, I'm practicing.
00:14:41.840 --> 00:14:45.440
Immediate opportunities are important.
00:14:45.440 --> 00:14:49.200
The 12 levers are basically uh the work uh of Dr.
00:14:49.200 --> 00:15:03.279
Ina Weinbauer-Heidel, who reviewed hundreds of years of transfer research and realized there's a lot of factors mentioned in research that are nice to know, but not really of value to you and me as practitioners, because we can't do anything about them.
00:15:03.279 --> 00:15:06.240
And also some of them just have a very minimal impact.
00:15:06.240 --> 00:15:12.639
So the 12 levers she identified are significant contributors to transfer.
00:15:12.639 --> 00:15:15.759
And so it's worth that we pay attention to them.
00:15:15.759 --> 00:15:21.919
Plus, you and I, with specific tools and interventions, can either control or influence them.
00:15:21.919 --> 00:15:24.720
They fall in three categories.
00:15:24.720 --> 00:15:32.960
Category one are the trainees themselves, things like their motivation, their confidence, but also their energy to apply.
00:15:32.960 --> 00:15:36.320
I want, I can, I will, in a nutshell.
00:15:36.320 --> 00:15:44.879
Then
there are four levers in the design, the so-called training design aspect, which are things like clarity of expectations.
00:15:44.879 --> 00:15:48.320
Do people actually know what's expected of them?
00:15:48.320 --> 00:15:51.360
Not what's expected for them to know, to do.
00:15:51.360 --> 00:15:55.840
So it's moving away from learning objectives to transfer goals, as we call it.
00:15:55.840 --> 00:16:00.240
It's content relevance, active practice, and transfer planning.
00:16:00.240 --> 00:16:02.159
Those are things in the design.
00:16:02.159 --> 00:16:06.480
And then there are five levers connected to the organization.
00:16:06.480 --> 00:16:10.240
Things like do people have opportunities to apply?
00:16:10.240 --> 00:16:16.639
Do people have the systems, the time, the processes to apply it?
00:16:16.639 --> 00:16:18.240
That's all about capacity.
00:16:18.240 --> 00:16:22.080
Then it's support of supervisors, support of peers.
00:16:22.080 --> 00:16:27.679
And then, very important, the last one is about the culture in an organization.
00:16:27.679 --> 00:16:32.159
Does the culture of an organization actually support transfer?
00:16:32.159 --> 00:16:34.480
Measure it, pay attention to it.
00:16:34.480 --> 00:16:35.919
Simple example.
00:16:35.919 --> 00:16:40.480
If we hand out certificates of attendance, what are we signaling?
00:16:40.480 --> 00:16:45.200
Our objective is for you to attend, versus the certificate of implementation.
00:16:45.200 --> 00:16:50.960
Or going back to when I started, if we are measuring satisfaction, what signal are we sending?
00:16:50.960 --> 00:16:52.720
We are here to make you happy.
00:16:52.720 --> 00:17:01.200
Whereas if we are measuring relevance, confidence to apply, uh, commitment to apply, we are sending a very different signal.
00:17:01.200 --> 00:17:07.599
Yeah, so in a nutshell, three levers around the trainees, five in design, five in the organization.
00:17:07.599 --> 00:17:09.839
Of course, they're all interconnected.
00:17:09.839 --> 00:17:18.480
Uh, for example, if we don't have a lot of support of supervisor, it very often negatively impacts people's motivation to apply.
00:17:18.480 --> 00:17:21.839
So, but that's basically it.
00:17:22.160 --> 00:17:22.960
That is powerful.
00:17:22.960 --> 00:17:24.960
And you got that in in a nutshell.
00:17:24.960 --> 00:17:28.960
I can't believe you did that because I've read your work and it's very detailed.
00:17:28.960 --> 00:17:30.559
You know, you got all that in.
00:17:30.559 --> 00:17:33.200
Um, but on that note, I want to give you a perspective.
00:17:33.200 --> 00:17:43.279
Um, you know, what I've uh written in my book, uh Result Pal, and what I talk about a lot is this idea that it takes a village to create results.
00:17:43.279 --> 00:17:51.759
And I think your methodology is showing that it is not only the participant that is responsible for results, and that's a big thing I try to preach.
00:17:51.759 --> 00:17:52.319
Yeah.
00:17:52.319 -->
00:17:56.079
How can you finish a training or a program of some kind and just say goodbye?
00:17:56.079 --> 00:18:05.359
And everybody just, you know, leaves the poor participant to go try to create new habits and behaviors all by themselves when the environment is not supporting, yeah, and so on.
00:18:05.359 --> 00:18:08.000
But but there's one thing I've learned that I want to ask you about.
00:18:08.000 --> 00:18:12.880
Okay, so we all know that managers are very important, and that's kind of the first given.
00:18:12.880 --> 00:18:14.000
It's it's well known.
00:18:14.000 --> 00:18:17.759
But what I've come across is the power of peer support.
00:18:17.759 --> 00:18:18.000
Yeah.
00:18:18.000 --> 00:18:26.400
And in fact, in in the technology that you know we have, which is an AI first approach to all of these things, peer support is like top of the list.
00:18:26.400 --> 00:18:32.400
What's your what's your view on peer support after someone is finished with training?
00:18:32.400 --> 00:18:34.000
How can peers be part of that?
00:18:34.000 --> 00:18:35.119
I mean, like, how does that work?
00:18:35.119 --> 00:18:37.359
I want to know in your in your view how that works.
00:18:37.680 --> 00:18:38.880
And it's very interesting, I love it.
00:18:38.880 --> 00:18:41.599
Like, because a peer support is it is really crucial.
00:18:41.599 --> 00:18:46.880
And I think before we get into how you can do it, there's two, there's one very important distinction to make.
00:18:46.880 --> 00:18:52.319
When we talk about peer support, we actually have to think about two types of peers.
00:18:52.319 --> 00:18:55.599
There are the um the peers in the cohort.
00:18:55.599 --> 00:19:04.559
So let's say you're going through a cohort-based learning, uh, they are your learning peers, and they are very important during the learning experience.
00:19:04.559 --> 00:19:08.880
And very often those are also the ones we then tend to focus on.
00:19:08.880 --> 00:19:14.240
However, what's actually even more important is to consider the peers back at work.
00:19:14.720 --> 00:19:15.200
Exactly.
00:19:15.440 --> 00:19:24.640
Because just like the supervisor, you might have a wonderful cohort with people really supporting you, but then you don't see them anymore.
00:19:24.640 --> 00:19:30.640
You go back to your work environment, you come back to your actual work colleagues, and they look at you and say, Oh my god, Dr.
00:19:30.640 --> 00:19:33.920
Pele, what kind of training are you coming back from at just stop all of this?
00:19:33.920 --> 00:19:34.799
No, no, no, no, no.
00:19:34.799 --> 00:19:35.519
Just stop.
00:19:35.519 --> 00:19:40.079
Like, don't just because you went on a training, don't try and now preach this to us, you know.
00:19:40.079 --> 00:19:42.880
And so the thing is that creates a lot more resistance.
00:19:42.880 --> 00:19:54.559
And so we gotta think about not only how to strengthen the cohort peer support, but most important is actually how can we get the peers at work involved?
00:19:54.559 --> 00:20:04.640
And so part of it might be, you know, that we one of my favorite tools is that I get participants sometimes to assign a so-called uh change observer.
00:20:04.640 --> 00:20:10.319
So they choose someone in their work environment that they say, hey, you're gonna be kind of my change observer.
00:20:10.319 --> 00:20:24.880
And so let's assume I'm coming back from a difficult conversation training and I want to pay more attention on getting less uh uh defensive, yeah, when someone is kind of giving me feedback.
00:20:24.880 --> 00:20:31.440
And so I'm sharing this with my change observer and ask them to pay attention to where I'm actually doing this.
00:20:31.440 --> 00:20:38.000
And so I'm giving them an invitation to be part of my journey, and that can be very, very powerful.
00:20:38.640 --> 00:20:41.359
That is very powerful, and you're a very simple example.
00:20:41.359 --> 00:20:46.559
So you should have been are you sure you weren't like uh on the on the wall when we were designing this?
00:20:46.559 --> 00:20:48.640
Because that's exactly how it works.
00:20:48.640 --> 00:20:51.039
You know, maybe just to follow a book, no?
00:20:51.039 --> 00:20:52.160
Yeah, yeah.
00:20:52.160 --> 00:20:54.960
There's something that you uh say that I really love.
00:20:54.960 --> 00:21:00.640
You say um that people have to move from aha moments to apply moments.
00:21:00.640 --> 00:21:01.359
I love that.
00:21:01.359 --> 00:21:05.599
That's that's more fun than you gotta go from knowing to doing gap or something like that.
00:21:05.599 --> 00:21:06.880
I think I like that.
00:21:06.880 --> 00:21:10.799
But the question I have for you is after the doing, right?
00:21:10.799 --> 00:21:14.000
What can we do to actually connect to results?
00:21:14.000 --> 00:21:15.759
So I'll give you an example of what I mean.
00:21:15.759 --> 00:21:25.599
A lot of people actually don't want to know if their program uh has influenced a specific result or not, because what if the influence was not positive, you know?
00:21:25.599 --> 00:21:36.640
Um what I like to say to counter that is when you can see behavior, when you have visibility, you also have results intelligence, which allows you to influence that behavior and increase the results.
00:21:36.640 --> 00:21:42.880
But how can we correlate a program's uh content to a specific result?
00:21:42.880 --> 00:21:46.960
Because if if I I think that's the final holy grail.
00:21:46.960 --> 00:21:57.279
And although although I have my little secret and it's it's been done, it's created, it's ready, I want to know what the expert in the field thinks about linking a program to results.
00:21:57.599 --> 00:21:57.839
Yeah.
00:21:57.839 --> 00:22:04.720
And this is it's I'm really grateful for the Question because I think it is really the missing element a lot.
00:22:04.720 --> 00:22:08.079
Um, and but what you said is, and then I'll come to how I would do it.
00:22:08.079 --> 00:22:11.279
I love what you said because it's so true.
00:22:11.279 --> 00:22:16.079
The biggest barrier with evaluation very often is people are afraid of what they find out.
00:22:16.079 --> 00:22:26.880
I cannot tell you the number of projects that I was asked to kind of start making the data disappear because the data all of a sudden did not show things were working.
00:22:26.880 --> 00:22:29.920
And I think there's a big thing we need to change in our industry.
00:22:29.920 --> 00:22:34.720
It's this thing about that evaluation is about a pat on the shoulder, it's about celebrating.
00:22:34.720 --> 00:22:36.480
Yes, great if it works.
00:22:36.480 --> 00:22:40.160
But for me, evaluation is all about monitoring and adjusting.
00:22:40.160 --> 00:22:42.880
It's continuous, yeah, adjusting.
00:22:42.880 --> 00:22:55.759
It's a continuous improvement process and collecting data at multiple intervals right at the end of a program, maybe 30 days in, 60 days in, 90, 120, just like when I run a marathon.
00:22:55.759 --> 00:23:02.160
I'm not only gonna see when I run over the finish line whether I achieve my objective of running it in less than three hours.
00:23:02.160 --> 00:23:04.799
No, because then either it was great or not.
00:23:04.880 --> 00:23:11.200
It was a hit or something or even like a child, a child learns to walk by sometimes falling down.
00:23:11.599 --> 00:23:12.000
Exactly.
00:23:12.000 --> 00:23:14.000
And so we need to change that.
00:23:14.000 --> 00:23:23.039
Yeah, we need to also change that mindset that we are L and D only successful if the business result did occur.
00:23:23.039 --> 00:23:30.720
You know, like I think we are doing our job if we very early on can start pointing out, hey, there might be an issue with this program.
00:23:30.720 --> 00:23:35.039
A lot of people at the end of the session said they're not really confident in applying.
00:23:35.039 --> 00:23:38.880
Let's go find out why they're not confident so that we can course correct.
00:23:38.880 --> 00:23:47.920
But to your earlier question, I think the biggest challenge is what I still see is um, and it's so simple, no, it's the starting with the end in mind.
00:23:47.920 --> 00:23:56.880
We've all heard it, uh, whether it was Kirby who said it or others, but it's one of these phrases in our industry we hear, but still people forget about it.
00:23:56.880 --> 00:24:01.759
They think evaluation starts after the program, just like transfer starts after.
00:24:01.759 --> 00:24:05.519
No, it starts way before, before you even design.
00:24:05.519 --> 00:24:08.319
You gotta be clear on why we are doing a program.
00:24:08.319 --> 00:24:10.160
So let's take a simple example.
00:24:10.160 --> 00:24:14.799
Let's take a call center that kind of comes to me and says, you know, we need training.
00:24:14.799 --> 00:24:16.240
Okay, why do we need training?
00:24:16.240 --> 00:24:25.039
Well, we're seeing a lot of agents um making mistakes uh when they are advising uh, you know, customers that are calling.
00:24:25.039 --> 00:24:26.559
Okay, making mistakes.
00:24:26.559 --> 00:24:28.799
Okay, how is that impacting the business?
00:24:28.799 --> 00:24:31.920
Well, our customers are raising massive amounts of complaints.
00:24:31.920 --> 00:24:33.920
We have so many complaints, you know.
00:24:33.920 --> 00:24:36.319
Oh, okay, and what's happening because of that?
00:24:36.319 --> 00:24:39.519
Well, that means you know, we have a lot of escalation issues.
00:24:39.519 --> 00:24:44.960
Managers are really busy solving these things and not really operating at the level they should.
00:24:44.960 --> 00:24:47.680
It's impacting customer satisfaction.
00:24:47.680 --> 00:24:50.799
And to be honest, we are starting to lose clients, you know.
00:24:50.799 --> 00:24:53.359
So I guess it's impacting our revenue.
00:24:53.359 --> 00:24:57.279
I mean, accelerated version, of course, you're kind of doing a lot of digging.
00:24:57.279 --> 00:25:02.079
But now I've basically heard why we need a program.
00:25:02.079 --> 00:25:10.240
Of course, I need to validate that we now really have a skill and the knowledge issue, because if not, training is not required, but let's assume we do.
00:25:10.240 --> 00:25:17.839
Now it becomes a very easy thing to start monitoring whether the desired results are going to happen.
00:25:17.839 --> 00:25:30.799
Because if the people in the training are learning the skills and knowledge we agreed upon, and they are transferring it onto the job, what should we see that changes fairly quickly?
00:25:30.799 --> 00:25:38.079
Well, we should probably see fairly quickly that they're making less mistakes when answering queries from customers.
00:25:38.079 --> 00:25:41.839
Because they're making less mistakes, what should we see next?
00:25:41.839 --> 00:25:45.200
Oh, less complaints, a reduction in complaints.
00:25:45.200 --> 00:25:50.160
Ah, because we have a reduction of complaints, probably less mistakes.
00:25:50.160 --> 00:25:52.400
We also should see less escalations.
00:25:52.400 --> 00:26:00.720
Managers again, you know, not having to step in that much, uh, higher customer satisfaction, business, you know.
00:26:00.720 --> 00:26:13.920
And the thing is, so what I'm trying to tell is it's very difficult to correlate the training to the make like the final end results, you know, that's let's say the customer satisfaction or the increase in revenue.
00:26:13.920 --> 00:26:24.160
And we gotta stop only focusing on that, we gotta focus on those so-called leading indicators, smaller milestones that are directly connected to the trading.
00:26:24.160 --> 00:26:27.359
Because that's where we can prove the trading had an impact.
00:26:27.359 --> 00:26:30.480
And that's also again coming back to what I shared earlier.
00:26:30.480 --> 00:26:36.799
If all of a sudden I see that people are still making mistakes, the mistake rate is not going down.
00:26:36.799 --> 00:26:43.759
Well,
I'm not gonna wait for one year to see whether the customer satisfaction is gonna improve, I'm gonna intervene right there.
00:26:44.079 --> 00:26:44.720
Exactly.
00:26:44.720 --> 00:26:47.039
No, you you make some powerful points.
00:26:47.039 --> 00:26:49.440
And remember, I told you that I had a little bit of a secret.
00:26:49.440 --> 00:26:50.559
You shared it.
00:26:50.559 --> 00:26:56.079
Let me tell you, let me tell you a perspective that I think you're gonna completely uh resonate with.
00:26:56.079 --> 00:27:01.039
So we all know Kirkpatrick's four levels, and the the level we're talking about right now is level four.
00:27:01.039 --> 00:27:05.119
In fact, that's why I called my company Result Pal, because that's our focus.
00:27:05.119 --> 00:27:06.079
Yeah, absolutely.
00:27:06.079 --> 00:27:17.279
So the question is Jack Phillips, another gentleman with another uh approach, yeah, tried to solve the return on investment ROI problem through a specific calculation.
00:27:17.279 --> 00:27:21.519
But he did not do that work on Kirkpatrick's fourth level.
00:27:21.519 --> 00:27:22.960
It's like a fifth level, okay?
00:27:22.960 --> 00:27:23.200
Yeah.
00:27:23.200 --> 00:27:30.480
So basically, what we've done is we've taken uh Jack Phillips' methodology and we've created it inside Kirkpatrick's level four.
00:27:30.720 --> 00:27:31.119
Yeah, yeah.
00:27:31.440 --> 00:27:32.319
And that works.
00:27:32.319 --> 00:27:36.880
We are able to bring causality and correlation much closer than it's ever been done.
00:27:36.880 --> 00:27:39.200
And and the main thing is AI.
00:27:39.200 --> 00:27:41.440
AI makes the big difference here.
00:27:41.839 --> 00:27:42.319
Absolutely.
00:27:42.799 --> 00:27:50.160
But but I do have one final question, and it's a hopeful one because I don't want people to think we're just not happy about you know results being linked.
00:27:50.160 --> 00:27:52.400
This is a hopeful question, okay?
00:27:52.400 --> 00:27:59.759
It's really for you, as someone at the very top of the industry, I'd love to know what the future looks like.
00:27:59.759 --> 00:28:07.599
Now that we have AI, we have a lot more excitement, we have Melanie and your passion pushing it out into the world.
00:28:07.599 --> 00:28:16.400
What do you think the next five to ten years looks like in this field in terms of people adopting uh the need for measurement outside of training?
00:28:16.720 --> 00:28:18.559
I'm I'm actually I'm super excited.
00:28:18.559 --> 00:28:25.759
I mean, I know I'm a I'm a you know kind of the glass is full person, not even half full, like I'm a you know, the glass is full person.
00:28:25.759 --> 00:28:29.759
But I actually really for the first time think we can do this.
00:28:29.759 --> 00:28:34.160
We can actually make transfer and evaluation happen because of the tech.
00:28:34.160 --> 00:28:45.759
Uh, because the tech can support us to make a lot of the things that consume so much time before quicker, freeing up our time to really focus on really what creates value.
00:28:45.759 --> 00:28:50.160
You know, so like, I mean, again, don't give me away, I'm just gonna take a very simple example.
00:28:50.160 --> 00:28:57.039
You know, like earlier we were spending so many hours on creating beautiful slide decks and kind of researching content.
00:28:57.039 --> 00:29:02.160
I mean, I remember when I started my PhD, you know, I had to go to the library researching books.
00:29:02.160 --> 00:29:06.000
All of that is available, you know, so quickly to us.
00:29:06.000 --> 00:29:07.599
So we can save so much time.
00:29:07.599 --> 00:29:16.160
And that time that gets freed up, we can use for those so important alignment conversations with stakeholders.
00:29:16.160 --> 00:29:22.319
We can finally go and really have those business meetings that we need to have to speak the business um language.
00:29:22.319 --> 00:29:27.680
Uh, with evaluation, when I started off in evaluation, I was crunching Excel data.
00:29:27.680 --> 00:29:35.759
It was powerful, it was painful, you know, collecting handwritten surveys that we were scanning and then translating into Excel sheets.
00:29:35.759 --> 00:29:44.720
Now I can have tools with the help of AI that allows me to very quickly analyze, create those correlations, create those charts.
00:29:44.720 --> 00:29:49.519
I can then go back with it and very quickly say, hey, look, we have 10 cohorts.
00:29:49.519 --> 00:29:55.200
In one cohort, clearly something is going on because people are reporting very different success data.
00:29:55.200 --> 00:30:04.559
So I'm really excited because I think we are finally, or even when you look at AI coaching, we know that an important lever is active practice.
00:30:04.559 --> 00:30:08.880
But so often we don't have the budget to do more active practice.
00:30:08.880 --> 00:30:11.680
Because I mean, coaching is really expensive.
00:30:11.680 --> 00:30:13.759
Now, is AI coaching perfect?
00:30:13.759 --> 00:30:17.759
No, but perfectionism is the biggest enemy of transfer, as I say all the time.
00:30:17.759 --> 00:30:30.240
But I'm just coming out of you know a two-day seminar as well that we used AI coaching, and it was great to see how scalable it is, and people were just able to practice having those high-stake conversations.
00:30:30.240 --> 00:30:32.240
And so I'm really excited.
00:30:32.240 --> 00:30:48.480
I mean, I don't know what's going to be in five, ten years, but I actually really think this is an opportunity for our industry to finally do the work we were supposed to do because we have the resources to actually do it.
00:30:48.480 --> 00:30:49.920
So I'm really excited.
00:30:50.240 --> 00:30:50.720
Wow.
00:30:50.720 --> 00:30:55.119
Well, I have just one more question for you, and that is, whoops, second.
00:30:55.119 --> 00:30:56.160
Yeah.
00:30:58.160 --> 00:30:59.680
I'm gonna have to edit that out, huh?
00:31:00.400 --> 00:31:02.000
No, perfectionism is the enemy.
00:31:02.000
--> 00:31:11.200
Like, this is this is, you know, ever since you know, Penelle, ever since I started to understand what makes transfer work, I became so relaxed about these little hiccups.
00:31:11.200 --> 00:31:17.920
Uh if I see a typo on my PowerPoint or something doesn't work before I was like, oh, what are people gonna think?
00:31:17.920 --> 00:31:20.400
They're probably gonna rate me back in my satisfaction.
00:31:20.400 --> 00:31:21.119
I don't care.
00:31:21.119 --> 00:31:26.079
I mean, you know, because I know it's not gonna have an impact on the middle, they're gonna apply afterwards, you know.
00:31:26.079 --> 00:31:26.720
Exactly.
00:31:27.200 --> 00:31:28.079
Exactly.
00:31:28.079 --> 00:31:34.160
So, so so Melanie, we have really gone all over the map in terms of this topic.
00:31:34.160 --> 00:31:42.799
And I have just one final question for you, and that is we've talked about technology, but let's come back to learning and development and the training training world.
00:31:42.799 --> 00:32:03.440
If one person is listening to us today, listening to you today, rather, um, were to take one thing away from everything you've shared that would let them change just one thing about their programs that would dramatically improve the likelihood of results, actual transfer of learning into action and then action to results.
00:32:03.440 --> 00:32:07.759
What would that one little change be that they can make in their programs for you?
00:32:08.480 --> 00:32:09.440
Yeah, that's a good one.
00:32:09.440 --> 00:32:10.559
That's a really tough one.
00:32:10.559 --> 00:32:12.319
That's one thing.
00:32:12.319 --> 00:32:19.039
But I mean, one thing I don't want to repeat because we kind of talked about it, is we got we gotta make sure that we start with that end in mind.
00:32:19.039 --> 00:32:20.559
I think that's so crucial.
00:32:20.559 --> 00:32:29.519
But if I really kind of come back, um, I think it is, you know, it is that monitor and adjust.
00:32:29.519 --> 00:32:36.079
I think that I would I would really recommend, especially if I just kind of combine transfer and evaluation.
00:32:36.079 --> 00:32:44.799
I would recommend take a look at your evaluation surveys because all of you are using them and so much stuff is out there to say, oh, forget about happy sheets.
00:32:44.799 --> 00:32:47.680
No, I'm don't forget about them, just revamp them.
00:32:47.680 --> 00:33:06.240
Take a look at those sheets and cross out all these questions that talk about satisfaction, replace them with questions around relevance, utility, include a question about how confident people are to apply, and ask them what potential barriers do they see in applying.
00:33:06.240 --> 00:33:22.240
I think with that, because the application barriers might be different in any in for each program, but by proactively asking people at the end of a program what possible challenges they see, it gives you the data you need to best support them.
00:33:22.240 --> 00:33:30.640
So I think that would be a very, no matter what kind of program you run, I think that's a really powerful uh first step.
00:33:31.920 --> 00:33:32.400
Wow.
00:33:32.400 --> 00:33:36.960
Well, Melanie, you know, we could be talking for a long time because I learned so much from you.
00:33:36.960 --> 00:33:38.400
But I'll tell I'll tell you this.
00:33:38.400 --> 00:33:52.240
I just want to say, I want to say thank you for the work you're doing to help all of us, right, uh, industry-wide move forward, move beyond learning uh for learning's sake, and move in the direction of creating results.
00:33:52.240 --> 00:33:54.960
And with that, I want to thank you for being here.
00:33:54.960 --> 00:33:57.039
This has been the human side of results.
00:33:57.039 --> 00:33:57.599
I'm Dr.
00:33:57.599 --> 00:33:59.279
Pillet, and this is the way.
00:33:59.279 --> 00:34:00.400
Thank you, Melanie.
00:34:00.799 --> 00:34:02.079
Thank you so much.