TRASCRIZIONE 🔗
WEBVTT
00:00:09.519 --> 00:00:13.519
Hi, so welcome back to the Precision Educator.
00:00:13.519 --> 00:00:24.079
I'm Viji Kurup, and I'm joined as always by my colleague and co-chair of the Society for Education and Anesthesia's Precision Education Task Force, Dr.
00:00:24.079 --> 00:00:24.879
Larry Chu.
00:00:25.280 --> 00:00:25.920
Thanks, Viji.
00:00:25.920 --> 00:00:27.359
It's great to be back.
00:00:27.359 --> 00:00:36.640
In our first two episodes, we talked about why precision education matters and how coaching helps personalize learning.
00:00:36.640 --> 00:00:46.240
Today we're going to turn to something that underlies nearly every promise we've made so far about precision education, and that's data.
00:00:46.560 --> 00:00:47.600
Yeah, that's so right.
00:00:47.600 --> 00:00:50.960
So data already shapes how we practice medicine.
00:00:50.960 --> 00:00:56.159
It's been driving quality improvement, it's been guiding how institutions adapt.
00:00:56.159 --> 00:01:01.119
And more and more it's uh influencing how we teach, right, Larry?
00:01:01.119 --> 00:01:05.920
And also how we learn, uh, even when we don't say this aloud.
00:01:06.319 --> 00:01:06.959
Absolutely.
00:01:06.959 --> 00:01:12.799
You know, but using data in education raises some interesting and some tough questions.
00:01:12.799 --> 00:01:16.319
For instance, what data actually matters?
00:01:16.319 --> 00:01:18.560
Who gets to see it?
00:01:18.560 --> 00:01:26.879
And how do we make sure it supports learning rather than turning education into surveillance or judgment or checklist?
00:01:27.439 --> 00:01:39.359
Yeah, and actually to help us explore that today, we are joined by the perfect person, someone who works right at that intersection of clinical data, analytics, and education.
00:01:39.359 --> 00:01:39.840
Dr.
00:01:39.840 --> 00:01:52.000
Matthew Caldwell is an anesthesiologist and researcher at the University of Michigan who's leading efforts to use perioperative data to individualize anesthesiology training.
00:01:52.000 --> 00:01:52.879
So welcome.
00:01:53.359 --> 00:02:05.519
Matt recently led a multi-center grant submission on educational innovation using data from MPOG, which stands for the Multi-Center Perioperative Outcomes Group.
00:02:05.840 --> 00:02:16.159
Yeah, and for listeners who may not know, MPOG is a national collaborative that pulls electronic health records from anesthesia departments across multiple institutions.
00:02:16.159 --> 00:02:24.479
And many of us know MPOG through these monthly reports that we get, which compare our practices to those of our national peers.
00:02:25.039 --> 00:02:29.199
And Matt's work asks the next logical question.
00:02:29.199 --> 00:02:38.240
If we have really good, maybe even you could say high-resolution data like this, can we improve patient outcomes?
00:02:38.240 --> 00:02:43.360
It also helps us understand and support how clinicians learn.
00:02:43.680 --> 00:02:44.879
So, Matt, welcome.
00:02:44.879 -->
Load full transcript
00:02:46.560
And we're thrilled to have you with us.
00:02:46.960 --> 00:02:48.719
It is so good to be here with you today.
00:02:48.719 --> 00:02:50.879
And uh, I'm really excited to talk about this topic.
00:02:50.879 --> 00:02:57.039
I I know the two of you are just uh engrossed in the education community and forward thinking about what could be and what will be.
00:02:57.039 --> 00:02:59.280
And so I'm I'm happy to join the conversation this morning.
00:02:59.280 --> 00:03:00.719
So thank you so much for the invitation.
00:03:01.039 --> 00:03:01.439
Amazing.
00:03:01.439 --> 00:03:11.039
Well, we're glad to have you, and uh, we really look forward to hearing about um how you're thinking about these possibilities with data and what you're doing.
00:03:11.039 --> 00:03:24.080
Um but before we actually go deep into a little bit of the technical side of things, I thought it might be helpful to ground this conversation in something familiar for our listeners.
00:03:24.080 --> 00:03:30.319
And in anesthesiology, data didn't revolutionize care or our practice overnight.
00:03:30.319 --> 00:03:32.319
It crept in slowly.
00:03:32.319 --> 00:03:36.960
Maybe some people could even say a little bit quietly through everyday practice.
00:03:37.360 --> 00:03:43.280
Yeah, and it showed up in really like practical ways in how we've changed our care and our teaching.
00:03:43.759 --> 00:03:55.919
And I would say, you know, once we had some registries or large data sets became available, such as MPOG, we started to see variation we didn't even maybe know existed.
00:03:55.919 --> 00:04:07.039
What we felt might be standard in our department at Stanford, for instance, looked totally different when compared across institutions like your department at Yale, BG.
00:04:07.439 --> 00:04:14.000
Yeah, and that's actually when our idea of an average patient uh fell apart, right?
00:04:14.400 --> 00:04:14.719
Right.
00:04:14.719 --> 00:04:32.160
And we began asking not just like what usually happens, but maybe what's most appropriate for this patient with these risks, and data moved from reacting to problems to anticipating like post-operative nausea and vomiting, for instance, and trying to prevent them.
00:04:32.720 --> 00:04:39.279
Yeah, and so this is actually in education, we are seeing a similar shift, right?
00:04:39.279 --> 00:04:45.920
Uh so we are kind of uh in for a long time we've assessed learners in a uniform way.
00:04:45.920 --> 00:04:55.040
We've done snapshots of um assessment uh without really considering the trajectory of the learner, right?
00:04:55.519 --> 00:05:12.639
And so I guess, you know, Matt, what we're wondering is what led you to believe that clinical data like MPOGs, um, and you you certainly have maybe a lot of it, uh, could actually help you understand education better.
00:05:12.959 --> 00:05:13.600
Yeah, thank you.
00:05:13.600 --> 00:05:22.480
It's funny listening to you to you both summarize kind of the state of things because it kind of speedruns the process that I experienced over the course of several years.
00:05:22.480 --> 00:05:27.040
And so there was a couple of different activities that kind of pointed me in this direction.
00:05:27.040 --> 00:05:33.759
One was the first half of my career was really around clinical care and uh leading teams.
00:05:33.759 --> 00:05:55.279
You know, so I oversaw uh one of our hospital operating room systems with you know, 30 operating rooms and a bunch of patients and all that kind of stuff, and was just impressed on the business analytic side of stuff, how well you could take data to make really informed, timely decisions about what has happened, what's coming down the pike, and try to adjust resources and plan proactively.
00:05:55.279 --> 00:05:59.199
And so kind of learning through that process was was really educational for me.
00:05:59.199 --> 00:06:01.680
So that was that was one foundational thing.
00:06:01.680 --> 00:06:09.040
Uh and the other foundational thing was just, you know, our department is the uh collaborating or the coordinating center for IMPOG.
00:06:09.040 --> 00:06:24.879
And so uh I'm fortunate to work with and around, you know, several really generous people that have um showed how well that data can be used to improve patient care and can be used to, as you mentioned, VG, just understand the variability that exists.
00:06:24.879 --> 00:06:29.279
And until you shine a light on it, it you just kind of make these assumptions.
00:06:29.279 --> 00:06:35.120
But as soon as you start to dig under the covers a little bit, you're like, oh, wait, there's something interesting going on here.
00:06:35.120 --> 00:06:49.600
And then experiencing COVID and the disruptions that went with that, you know, on the on the OR management side of stuff, I felt much better equipped to make timely decisions about, okay, here are the uh resources and the uh obstructions that we have right now.
00:06:49.600 --> 00:06:53.199
What can we do with those with those um limitations?
00:06:53.199 --> 00:06:58.319
But on the education side of stuff, we just did not have the tools to be able to adapt to that uh very rapidly.
00:06:58.319 --> 00:07:14.319
And so, you know, trying to lead a group that was heading towards graduation and were having you know less predictable experiences on rotations just just led me to start wondering, hey, you know what, you know, there we've had so much success on the on the business side of things and on the research side of things.
00:07:14.319 --> 00:07:20.800
You know, is there an opportunity to use this this data uh to help support our educational process as well?
00:07:20.800 --> 00:07:23.439
And so that's kind of what got me thinking about these areas.
00:07:23.680 --> 00:07:29.279
Yeah, that's that's really you know very eye-opening, especially those monthly MPAG reports, right?
00:07:29.279 --> 00:07:31.759
Like you open your email and there it is.
00:07:31.759 --> 00:07:34.639
It'll tell you, you know,
how have you managed blood pressure?
00:07:34.639 --> 00:07:36.639
How have you managed your fresh gas loss?
00:07:36.639 --> 00:07:41.120
And how are you doing compared to, you know, to national benchmarks?
00:07:41.120 --> 00:07:50.240
And uh earlier we just assumed that what we were doing was the right thing, and this was, you know, the state of the art of practice.
00:07:50.240 --> 00:07:58.480
And now I'm walking into my workroom and I'm asking everyone else, hey guys, why am I so low on my PO and V uh, you know, metrics?
00:07:58.480 --> 00:08:05.120
What are you doing differently that I'm not doing, that my patients seem to have a higher, you know, PO and V risk.
00:08:05.120 --> 00:08:14.480
And I think that that's why where you know you see learning and action, even if we don't always say it uh loud.
00:08:14.480 --> 00:08:17.519
But uh, Larry, we've also seen this in other areas.
00:08:18.000 --> 00:08:21.279
Well, I think that's you know where the reflection part kicks in, right?
00:08:21.279 --> 00:08:26.399
Because you, as the person reading that MPOG report, you you pause for a moment.
00:08:26.399 --> 00:08:27.680
It's personal.
00:08:27.680 --> 00:08:30.879
And you got to ask, why does my data look different?
00:08:30.879 --> 00:08:35.120
Uh and maybe you adjust, or maybe you talk about it with a colleague.
00:08:35.120 --> 00:08:37.840
But you know, that is learning.
00:08:37.840 --> 00:08:39.519
And that's learning in action.
00:08:39.840 --> 00:08:43.279
Yeah, and that dissonance is such a strong stimulus for education, right?
00:08:43.279 --> 00:08:47.360
Where you say, okay, here's who I think I am, but here's the data in front of me.
00:08:47.360 --> 00:08:49.120
Okay, now I'm confronted with this.
00:08:49.120 --> 00:08:50.960
Now, how do I want to respond to that?
00:08:50.960 --> 00:09:06.080
Um, and that's really one of the founding principles of this work that we're going forward with, is just to say, you know what, this is an opportunity for learners to, you know, they're already doing it, but to really embed that in their training and hopefully extend that into their careers the long term to say, okay, here are the challenges that I have.
00:09:06.080 --> 00:09:09.120
What do I need to equip myself to continuously improve my practice?
00:09:09.440 --> 00:09:15.759
And it's interesting, you use the word improvement because I think we've seen this before in NISQIP.
00:09:15.759 --> 00:09:20.159
I don't know if I'm pronouncing it correctly, but the National Surgical Quality Improvement Program.
00:09:20.159 --> 00:09:32.399
And in that initiative, they they showed that when institutions could see their outcomes side by side with other institutions, it was kind of like the MPOG report for surgical outcomes.
00:09:32.399 --> 00:09:37.279
And that really did lead to real improvements, just sharing that information.
00:09:37.600 --> 00:09:37.759
Yeah.
00:09:37.759 --> 00:09:48.720
And I want to take this a little uh further
and ask you to elaborate a little bit more on how you see uh EMPOG playing that kind of role in education.
00:09:49.039 --> 00:09:49.200
Yeah.
00:09:49.200 --> 00:09:50.799
So thanks for that question.
00:09:50.799 --> 00:09:57.519
You know, I think I think we're just at the foundational level, in my perspective, the foundational level of what could be done.
00:09:57.519 --> 00:10:02.080
And so, you know, I currently chair our clinical competency committee within our department.
00:10:02.080 --> 00:10:13.120
And it would be so nice to have really high quality, frequent uh assessments in uh various different settings about uh residents' performance, but it's hard to get to that, right?
00:10:13.120 --> 00:10:17.919
I mean, it's a it's a big resource, it's a big lift, um, there are constraints around there.
00:10:17.919 --> 00:10:34.639
Um, and so the data that IMPOG can provide initially is not around the observed performance from residents and faculty, but sort of the experiences that they're having in the operating room and then uh metrics that are associated with the quality of care.
00:10:34.639 --> 00:10:39.200
So process of care measures or outcome measures and things like that as well.
00:10:39.200 --> 00:11:01.200
And so we're really taking a view that this is a very long-term project and that we are trying to build a foundational system where people have trust in the data and how it's used and how it can be effective for people, and sort of together learn where the high-quality signals are uh within that that data as well.
00:11:01.200 --> 00:11:01.759
Right.
00:11:01.919 --> 00:11:11.200
And uh, you know, a single data point really doesn't tell you the full story, uh, but what are the patterns that you're seeing over time?
00:11:11.200 --> 00:11:15.279
That's probably where the learning lives, right?
00:11:15.600 --> 00:11:16.240
Yeah, I think so.
00:11:16.240 --> 00:11:24.639
Yeah, and and one of the gaps that I see is that, you know, our current recommendations for training for residents include things like time on rotations.
00:11:24.639 --> 00:11:32.799
And so, you know, uh residents have to have a certain number of months of X, Y, and Z, or a certain number of procedures of various types.
00:11:32.799 --> 00:11:39.440
But, you know, if if we talk about a specific rotation, what is it in that rotation that we're hoping that they will experience, right?
00:11:39.440 --> 00:11:42.799
Is it different types of patients with different core morbidities?
00:11:42.799 --> 00:11:45.919
Is it the anesthetic challenges within the in the case?
00:11:45.919 --> 00:11:50.399
Is it uh you know, management of specialized monitors or equipment or techniques?
00:11:50.399 --> 00:11:52.960
Is it the specific perioperative risks of those patients?
00:11:52.960 --> 00:12:01.600
And so there's a lot to unpack there in terms of under the broad category of a rotation, what are the actual experiences that residents have?
00:12:01.600 --> 00:12:11.759
And I think there's an opportunity to actually quantify those and then understand, you know, how those experiences are different within a residency for different residents and across residents in different programs as well.
00:12:12.080 --> 00:12:17.840
And I think that's a good time to pause a moment and to make a key distinction here.
00:12:17.840 --> 00:12:27.120
Because you just, I think, uh brought up a good point, which is you know, quality data helps us understand what happened and maybe what happened in a rotation.
00:12:27.120 --> 00:12:30.639
Education asks how did somebody maybe get there?
00:12:30.639 --> 00:12:36.720
And I love how you're kind of unpacking that for our listeners and bringing us through your thought process.
00:12:36.720 --> 00:12:38.799
VG, any comments there?
00:12:39.279 --> 00:12:42.879
So, yeah, so you know, uh, so what are your thoughts?
00:12:42.879 --> 00:12:49.360
You know, we've been using this data usually to just assess our residents, right?
00:12:49.360 --> 00:12:52.720
Like so we're looking at the outcomes and we're saying, how does how's our residents doing?
00:12:52.720 --> 00:13:02.159
But how are we going to use that to guide their growth for, you know, uh that data can also be used for coaching the residents.
00:13:02.159 --> 00:13:04.320
So what are your thoughts on those?
00:13:04.639 --> 00:13:08.159
Yeah, um, I I think it's a really big, broad topic.
00:13:08.159 --> 00:13:12.559
So you know, I won't, I won't pretend to cover or be an expert in all aspects of that.
00:13:12.559 --> 00:13:15.919
But I would say, you know, I think there's a couple of opportunities there.
00:13:15.919 --> 00:13:29.200
You know, there are existing quality metrics that have already been agreed upon by the community that, you know, certainly our residents are a part of the healthcare team and need to understand their the care that they deliver to the patients.
00:13:29.200 --> 00:13:35.200
But there's also um, you know, other opportunities around uh experience data as well.
00:13:35.200 --> 00:14:01.600
And the experience data, obviously, having an experience does not ensure that a resident has learned or is competent, but it is a foundational aspect of clinical training with this apprenticeship model of training that we have, where you know there's there are didactic components, uh, certainly, uh, but a lot of what our anesthesia residents um encounter is apprenticeship mode of training where they have progressive responsibilities.
00:14:01.600 --> 00:14:12.000
And we have kind of limited insight on a day-to-day basis as a supervising faculty as to what's the proper level of supervision that might be appropriate for this patient.
00:14:12.000 --> 00:14:30.240
And so, you know, I'm sure both of you and and others have had this experience where, you know, you're you're taking these cues that you get from the resident to understand where they might be in their learning curve, to understand what the appropriate amount of
autonomy is for a particular procedure or a particular aspect of care.
00:14:30.240 --> 00:14:35.840
You know, you could frame that into kind of an entrustable professional activity type of framework.
00:14:35.840 --> 00:14:44.080
Um, but wouldn't it be great if uh if there was a vetted source of information to at least understand what a resident has experienced before?
00:14:44.080 --> 00:14:57.360
And so if you knew that a resident had done 20 or 30 central line placements before, your expectations of what that resident might be capable of and what the appropriate supervision might be able to be fine-tuned a little bit better.
00:14:57.360 --> 00:15:00.000
Um, and the same thing with uh other different types of cases.
00:15:00.000 --> 00:15:12.720
And so I think it, you know, that just I'm just giving one granular example, but you know, those are one of the types of things that I think could be built upon a foundation of high-quality data that's specific to the educational process.
00:15:12.720 --> 00:15:22.960
That comes down to how data is used, is that within our program, it stands for PACE, PACE, which is precision analytics for coaching effectiveness.
00:15:22.960 --> 00:15:25.919
And the key part there is the coaching part.
00:15:25.919 --> 00:15:50.159
And so I would just like to highlight that what we're envisioning is that we have a program of data-enhanced coaching where a resident just doesn't receive an email or uh receive a dashboard, but it really is engaging in a conversation with a coach that's not telling them what to do, but sort of guiding them through a conversation of, okay, well, here's the data that is in front of you.
00:15:50.159 --> 00:15:51.440
How does that make you feel?
00:15:51.440 --> 00:15:52.639
How do you react to it?
00:15:52.639 --> 00:15:56.240
What goals and opportunities do you see related to that data?
00:15:56.240 --> 00:16:00.720
So it's really a conversational piece that we're hoping to accomplish with this as well.
00:16:01.120 --> 00:16:13.120
So it's almost like data-informed coaching, where you're you're helping to surface more in the conversation with reflection when they can see more concretely some of their own data.
00:16:13.120 --> 00:16:17.600
Well, I think that's important, though, because learning doesn't happen in snapshots.
00:16:17.600 --> 00:16:22.399
It builds over time, right, through repeated uh exposures and experience.
00:16:22.399 --> 00:16:32.320
And when as a faculty member, when we're thinking of entrustment, unfortunately, right, sometimes it is, but it shouldn't be necessarily based on a working with somebody once, right?
00:16:32.320 --> 00:16:37.440
Because entrustment is a prediction about how entrustable this person is, right?
00:16:37.440 --> 00:16:39.440
For tomorrow, the next day.
00:16:39.440 --> 00:17:00.240
And that longitudinal data, I have to imagine, is something that uh is an important piece of the puzzle that could drive helping us understand that idea of how
does someone get there, but also maybe from VG's point, uh helping us as educators with assessment.
00:17:00.240 --> 00:17:14.000
So how do you or maybe give us some of your thoughts on how clinical data might be used to trace learning over time, this trajectories concept, um, the longitudinal aspect?
00:17:14.000 --> 00:17:16.079
You mentioned EPAs before.
00:17:16.079 --> 00:17:19.759
Is there any other thought you've been you've given to this?
00:17:20.240 --> 00:17:21.200
Yeah, yeah, thanks.
00:17:21.200 --> 00:17:28.079
And I would just highlight that uh I think this, the specialty of anesthesiology for residency training, is particularly challenged, right?
00:17:28.079 --> 00:17:39.200
And so, you know, if you are a medicine or pediatrics or surgical resident, you might be paired with an individual faculty over the course of a week or a month or a rotation or something like that.
00:17:39.200 --> 00:17:42.880
You know, within anesthesiology, we have these episodic interactions.
00:17:42.880 --> 00:17:48.960
And so, you know, you might work with a resident one day and not work with them again for several months, depending on the size of your program.
00:17:48.960 --> 00:18:02.640
And so then how do you string that together in a way that you could the that both the learner, the supervising resident, and the the program as a whole can have a more complete picture of what the trajectory of that resident is.
00:18:02.640 --> 00:18:09.119
And so uh again, the the foundational part that we're uh trying to build is based on experiences.
00:18:09.119 --> 00:18:13.759
And so I'm imagining something like a growth curve that you might have for a newborn infant, right?
00:18:13.759 --> 00:18:16.720
And so uh your baby is born, it's uh a certain weight.
00:18:16.720 --> 00:18:24.400
The weights can differ depending on all sorts of factors, but there is a trajectory that is uh typical of a growing infant.
00:18:24.400 --> 00:18:34.559
Uh when something varies uh outside of that normal range, it just makes you wonder, okay, what's going on, or uh, you know, you know, may prompt for uh a deeper inquiry there.
00:18:34.559 --> 00:18:51.359
And I and I think that we might have an opportunity to do something similar there for experiences and performances of residents over time as well, to kind of create a shared understanding between those three parties as to you know how a resident is progressing in their training goals.
00:18:51.759 --> 00:18:53.039
Yeah, that's very interesting.
00:18:53.039 --> 00:19:02.160
And I also like to uh like the concept you brought of how we record experiences and equate that with learning.
00:19:02.160 --> 00:19:10.559
And it's and they're two different things because two people could have the same experience and come out with different learnings from each of them.
00:19:10.559 --> 00:19:13.680
That kind of uh, you know, like structured me.
00:19:13.680 --> 00:19:18.960
But we can also move from this with like just like can we just zoom out?
00:19:18.960 --> 00:19:29.519
Because we've talked so far and looked at precision education and looking at data and seeing how we can help our trainees with this.
00:19:29.519 --> 00:19:33.920
But really, precision education doesn't stop with residency, right?
00:19:33.920 --> 00:19:38.160
It does, it continues uh into our lives as uh faculty.
00:19:38.640 --> 00:19:39.359
Well, absolutely.
00:19:39.359 --> 00:19:42.799
And it matters a lot for practicing anesthesiologists.
00:19:42.799 --> 00:19:56.240
And certainly, you know, one aspect of that we've mentioned the monthly MPOG reports that that we get, that obviously it's impactful because we we we both mentioned it as something that, you know, it's has an impact on us.
00:19:56.240 --> 00:20:04.880
So imagining that CME reflects not just your interests, but I'm thinking, and I'm sure Matt, you've probably thought of this.
00:20:04.880 --> 00:20:08.079
What if MPOG could help education do more than that?
00:20:08.079 --> 00:20:14.480
If CME could not just reflect uh practicing anesthesiologists' interests, but maybe their actual practice?
00:20:14.480 --> 00:20:15.680
What would that look like?
00:20:16.000 --> 00:20:28.079
Yeah, I mean, I think I mean that that is sort of the holy grail, right, of of trying to understand how educational processes affect real world outcomes of patients and healthcare providers and things like that as well.
00:20:28.079 --> 00:20:42.880
And so that's not the scope of our current work right now in terms of uh practicing physicians, but I think that there is an opportunity to be able to link what residents have done and experienced and performed during residency program with the actual care that's delivered.
00:20:42.880 --> 00:20:56.400
And you know what, you know, there are certainly studies that indicate that kind of the structure of the healthcare system that you're in influences very highly the type of practice that you have after you leave that training environment.
00:20:56.400 --> 00:21:01.200
And so you get imprinted on by the uh the healthcare system that you're in as well.
00:21:01.200 --> 00:21:20.880
And so I think you know, this concept of a learning healthcare system where you are, as a healthcare system, inputting data on how your patients are doing in your care processes and using that to continuously improve quality of care, that can be done through the individual clinician, but also through structural uh techniques within the healthcare community.
00:21:20.880 --> 00:21:28.559
And I think it'd be interesting to see how that translates into a practice a year out, two years out, five years out, uh, and that as well.
00:21:28.559 --> 00:21:32.240
And so um that's certainly something that we have an eye on uh in the future.
00:21:32.240 --> 00:21:36.559
But for this next few years, we're really focusing on the residency uh period.
00:21:37.200 --> 00:21:37.440
Yeah.
00:21:37.440 --> 00:21:47.680
And that's uh, you know, I like that
our focus in the long term can be on how do we support lifelong learning for everyone.
00:21:47.680 --> 00:21:58.720
Uh, but in the short term, we're also going to, you know, I think we need to talk a little bit about uh the risks associated with collecting this data, right?
00:21:58.720 --> 00:22:03.359
Because anytime we collect Collect data, this data can be misused.
00:22:03.359 --> 00:22:10.400
And learners' assessment naturally drives learners to fear, right?
00:22:10.400 --> 00:22:19.279
They have fear, this fear of being watched, faculty have fear of being judged, institutions fear legal risk.
00:22:19.279 --> 00:22:39.119
So how do you, you know, how do you uh figure out when it comes down to trust, how can we ensure that data that's being collected is used in a supportive way and not in a punitive way?
00:22:39.440 --> 00:22:42.799
Yeah, I think I think you hit a really important topic right there.
00:22:42.799 --> 00:22:53.599
So I'll first just pause and say that the Impog organization has been around for about 20 years now and has learned a lot over that time under the directions of many people, but including Dr.
00:22:53.599 --> 00:22:54.480
Trimper and Dr.
00:22:54.480 --> 00:23:05.039
Keterpal, who were really instrumental in developing this and the safeguarding of data, how it's stored, how it's stewarded, you know, has been pretty well established there.
00:23:05.039 --> 00:23:06.000
And I think so.
00:23:06.000 --> 00:23:12.160
I'll kind of focus my comments on the educational aspects and kind of this the sensitive nature of that as well.
00:23:12.160 --> 00:23:20.000
And I think the approach that we're taking is to basically be as open and as transparent as possible in the development of this.
00:23:20.000 --> 00:23:36.000
And so, you know, our this is data is a shared resource that the residents and other learners should have an input into how data is used, how is it collected, how it's stewarded, and you know, what sort of downstream implications there are from that.
00:23:36.000 --> 00:23:43.759
And so, you know, I think there are with any new sort of data, there are going to be concerns around how it's used.
00:23:43.759 --> 00:23:53.599
And and I think it's it's incumbent upon us to sort of navigate that in a way that's transparent and is malleable over time as as we get feedback from people as well.
00:23:54.079 --> 00:24:08.160
And I think the the governance aspect and the trust aspect of data in education, I feel like it it has so many more nuances than probably we have time for in this podcast.
00:24:08.160 --> 00:24:25.440
But I, you know, I I relay it to my own observations, just that, you know, in in medical education right now, for instance, you know, we have we have the fresh start, I call it the fresh start, which is you go from one stage of your training to the next, and your past doesn't necessarily follow you.
00:24:25.440 --> 00:24:30.319
So you might have been the med student who took a little longer in the
clinics to grasp things, right?
00:24:30.319 --> 00:24:32.480
But that past doesn't follow you to residency.
00:24:32.480 --> 00:24:33.359
You get a fresh start.
00:24:33.359 --> 00:25:02.880
And and I wonder, and it's not maybe we're gonna solve it on this podcast, but you know, in in this emerging age of of data, and especially if, and it's not now maybe, but as we move to start collecting data through all stages of a learner's trajectory, those conversations about governance, and maybe not, you know, just the fresh start, but bias in the data, amplifying that bias, all sorts of questions come into play.
00:25:02.880 --> 00:25:16.400
And and I I like what you said, Matt, about involving residents, involving the end users in those conversations, because I think governance is going to be a big part of how to make sure those kind of concerns get addressed.
00:25:16.480 --> 00:25:16.640
Yeah.
00:25:16.640 --> 00:25:17.039
Yeah.
00:25:17.119 --> 00:25:20.160
And I think I think trust has to be earned over time as well.
00:25:20.160 --> 00:25:27.519
And, you know, I think hopefully by starting with experience data, that's not, there's not a judgment aspect in there.
00:25:27.519 --> 00:25:29.839
It's just, hey, did you have this experience or not?
00:25:29.839 --> 00:25:42.319
I mean, I think that's a pretty good starting point to work out kinks and to generate conversation and get feedback and essentially secure trust from all the parties that are involved.
00:25:42.319 --> 00:25:52.000
And that's one of the reasons that we're really emphasising the emphasizing the experiential aspect of it as opposed to evaluative aspect of clinical clinical care.
00:25:52.240 --> 00:25:52.400
Yeah.
00:25:52.400 --> 00:25:55.839
And that clarity really makes a big difference, right?
00:25:55.839 --> 00:25:59.039
It tells people this is for your growth.
00:25:59.039 --> 00:26:04.079
It's not to punish you or to grade you or judge you.
00:26:04.079 --> 00:26:06.799
And that's it's an ethical commitment, right?
00:26:06.799 --> 00:26:10.640
To say uh to set the tone of how innovation happens.
00:26:10.960 --> 00:26:11.200
Yeah.
00:26:11.200 --> 00:26:19.279
And just one example, you know, we looked at our the hemorrhage resuscitation of experience of our residents, you know, throughout their residency program.
00:26:19.279 --> 00:26:21.759
And as you would expect, there's a lot of variability.
00:26:21.759 --> 00:26:30.880
And so some of our residents left with a lot of experience with severe hemorrhage in the operating room, and a few with very, with very few, relatively few experiences there.
00:26:30.880 --> 00:26:34.880
And so that has implications as to, you know, how what do we do with that information?
00:26:34.880 --> 00:26:46.960
I mean, there's there's things that you can do on the programmatic aspect, but also, you know, you know, if a resident's going to have only a handful of experiences, that really ups the ante on what they need to get out of that experience.
00:26:46.960 --> 00:26:52.960
And so there's a lot of an opportunity for more deliberate reflection and goal setting along those.
00:26:52.960 --> 00:27:06.160
If I as a resident know that on average, I get a handful of these experiences during my residency, and I had one today, man, it is it is my job to try to figure out how I can get the most value out of that.
00:27:06.160 --> 00:27:15.759
Because you know, when you're coming already, you just you don't necessarily have the perspective within that program of you know what the downstream experiences are that you're going to have.
00:27:15.759 --> 00:27:21.440
And so having a little bit of insight to that, I think could could tie into more efficient learning for those types of things.
00:27:21.759 --> 00:27:24.880
I just want to follow up on that, Larry, on that particular study.
00:27:24.880 --> 00:27:31.200
Did you use that just to collect the information or did you go further and actually act on that information?
00:27:31.519 --> 00:27:32.720
For the hemorrhage resuscitation.
00:27:32.880 --> 00:27:34.079
For the hemorrhage resuscitation.
00:27:34.400 --> 00:27:39.440
We published one study, just look at a single center study just looking at our hemorrhage experience of residents.
00:27:39.440 --> 00:27:48.880
We have a follow-up study in process right now that has a it's a qualitative study where we're doing interviews with faculty and residents to understand what their experience was.
00:27:48.880 --> 00:27:50.240
And it's it's fascinating.
00:27:50.240 --> 00:27:51.440
And I'll just highlight Dr.
00:27:51.440 --> 00:27:54.480
Lara Zizblatt, who's one of our PhD educators, and Dr.
00:27:54.480 --> 00:27:56.799
Grant also are heavily involved in this work.
00:27:56.799 --> 00:28:00.799
But you know, when you for myself, I do a lot of vascular anesthesia.
00:28:00.799 --> 00:28:03.920
It seems like every day you're transfusing people and all that kind of stuff.
00:28:03.920 --> 00:28:05.519
But as a resident, it's amazing.
00:28:05.519 --> 00:28:10.559
Like the first time that you do that, like which button do I push on the on the resuscitation machine?
00:28:10.559 --> 00:28:12.079
Why are you why are you giving calcium?
00:28:12.079 --> 00:28:12.640
Why is that important?
00:28:12.640 --> 00:28:14.400
Like all these questions come out.
00:28:14.400 --> 00:28:26.240
And so we have a follow-up study right now, just trying to understand, you know, what what I think might be most important to the resident may not be actually what is what the resident needs, depending on what their prior experience in learnings were.
00:28:26.240 --> 00:28:30.559
And so I think we're we'll have some interesting insights to share with that when that concludes.
00:28:31.039 --> 00:28:41.839
So I think, you know, we've had some pretty high-level conversations about data, about governance, about how it would inform precision education.
00:28:41.839 --> 00:28:57.279
But I would love to take a moment to
bring it down to the listener level, people who are, you know, the practicing clinician or the educator working with residents, the CCC committee member or chair.
00:28:57.279 --> 00:29:18.880
For those listeners in our audience who are interested in maybe working more with data or making more of their decisions in better informed by data, but they they don't have a Sacha and Ketarpal in their department, maybe, or they're not maybe uh MPAG and Matt Caldwell for sure.
00:29:18.880 --> 00:29:28.160
What advice could you give them about starting and thinking about data or using it in their work as educators?
00:29:28.480 --> 00:29:30.640
Yeah, that's that is a great question.
00:29:30.640 --> 00:29:35.920
I I think they're I guess my recommendations would be to be trying to solve pain points, right?
00:29:35.920 --> 00:29:40.960
So identify where the needs and the gaps are in terms of education and training.
00:29:40.960 --> 00:29:45.759
You know, all of us are now on some sort of electronic healthcare record.
00:29:45.759 --> 00:29:58.079
And so even for institutions that are not part of the IMPOG network, you know, there are there are reporting type tools within the electronic healthcare record suite of applications as well.
00:29:58.079 --> 00:30:03.039
And so there is there are opportunities for those outside of IMPOG to do this as well.
00:30:03.039 --> 00:30:14.400
And I'll just highlight the work that we're doing is is meant to be open and our data standards will be published within a sort of a website format so that people can can access that.
00:30:14.400 --> 00:30:17.279
Because if you haven't worked with data before, it's it's amazing.
00:30:17.279 --> 00:30:19.440
You're like, okay, I just want to measure this, right?
00:30:19.440 --> 00:30:22.160
Whatever that I just want to measure what a polar bear is.
00:30:22.160 --> 00:30:24.559
But then you have to define what a polar bear is.
00:30:24.559 --> 00:30:26.000
Like, does it have to be this big?
00:30:26.000 --> 00:30:27.359
What color does it need to be?
00:30:27.359 --> 00:30:29.200
How big do its claws need to be?
00:30:29.200 --> 00:30:34.240
Like all those decisions are actually way more challenging than you might think at first blush.
00:30:34.240 --> 00:30:41.279
And so hopefully some of the work that we're doing is to, is to define some of those things in a multi-center collaborative work.
00:30:41.279 --> 00:30:49.759
But then the other people and other with other types of resources or can then take those data standards and be like, oh, look, here are the data definitions.
00:30:49.759 --> 00:30:58.480
Take that to your colleagues within the data informatics team and say, how can I translate this using our resources into something that I can that I can use locally?
00:30:58.480 --> 00:31:07.039
And so, you know, we're we're building within the IMPOG network, but we're really trying to make this accessible to all residents within the United States.
00:31:07.359 --> 00:31:12.960
That's great because one of the things that Larry and I care deeply about is this equity, right?
00:31:12.960 --> 00:31:23.839
Technical equity or techity, because a lot of places don't have access to personnel or resources that a lot of other institutions do.
00:31:23.839 --> 00:31:35.440
And trainees there should receive no less of uh experience than the ones who are in places that have you know high resource settings.
00:31:35.599 --> 00:31:35.759
Yeah.
00:31:35.920 --> 00:31:38.559
So I'm glad that that will be available to everyone.
00:31:38.880 --> 00:31:43.680
Yeah, and it's really gotten me thinking also, uh, I completely resonate with your comment there.
00:31:43.680 --> 00:32:00.000
And the other thing that I'm thinking about is that you know, we have this kind of federated system of education within the US, where we have residency centers that are across geographic locations, across urban and rural settings and community settings and tertiary care hospitals.
00:32:00.000 --> 00:32:03.119
And so, you know, what is it that a resident needs?
00:32:03.119 --> 00:32:05.680
There's not one standard answer, right?
00:32:05.680 --> 00:32:17.279
It it, you know, our residents are going on to different practice environments, and you know, different programs might have various strengths and weaknesses or or various goals that they are training for.
00:32:17.279 --> 00:32:21.119
And they're not necessarily better or worse, but they're just different.
00:32:21.119 --> 00:32:34.799
And so I think one of the gaps that that we need to be mindful of while we're building this out is that you know the community-based um training programs, their needs may be different than you know, a large academic hospital's needs.
00:32:34.799 --> 00:32:44.160
And so making assumptions, as you as you meant mentioned, you know, as soon as you start to decide what you're gonna measure, you've already made some judgments about what's important and what's not important.
00:32:44.160 --> 00:32:47.200
And so I think we just need to be mindful of that as well.
00:32:47.759 --> 00:32:48.640
Perfect.
00:32:48.640 --> 00:32:59.759
And uh so again, like you know, we can zoom out a little bit and look at precision media medicine required new tools, new norms, right?
00:32:59.759 --> 00:33:05.039
And similarly, precision education is gonna, you know, need that too.
00:33:05.039 --> 00:33:11.039
So uh when it works well, it can make teaching easier, not harder.
00:33:11.519 --> 00:33:11.839
Great.
00:33:11.839 --> 00:33:15.839
Well, I think this is a great conversation today.
00:33:15.839 --> 00:33:22.319
And it reminded us that data already shapes, as has been mentioned, how we care for patients.
00:33:22.319 --> 00:33:28.480
And the opportunity now maybe is to let it shape how we can support learning.
00:33:28.480 --> 00:33:36.880
I think not only for our residents, but hopefully in the future for continuing medical education for practicing clinicians.
00:33:37.200
--> 00:33:40.480
Yeah, thank you so much for the opportunity to join you in conversation today.
00:33:40.480 --> 00:33:43.680
Uh, this is something that I'm sure is very exciting to me.
00:33:43.680 --> 00:33:45.279
I could talk about this for a very long time.
00:33:45.279 --> 00:33:47.279
And I think both of you could as well.
00:33:47.279 --> 00:33:49.519
So I appreciate the opportunity to kind of share there.
00:33:49.519 --> 00:33:53.519
I would be hate to miss the opportunity to just thank a few people as well.
00:33:53.519 --> 00:33:56.000
And so within my mentorship network, Dr.
00:33:56.000 --> 00:33:58.319
Caterpillar has been instrumental, as have Dr.
00:33:58.319 --> 00:33:59.519
Naughton as well.
00:33:59.519 --> 00:34:12.159
I would just like to specifically thank the Foundation for Anesthesia Education Research as well for funding and support in the past, and the American Medical Association for our current grant and the change meta precision education thing.
00:34:12.159 --> 00:34:29.199
And that's a great community where we hope to learn not just from anesthesiology communities grappling with these issues, but across the spectrum of, Larry, as you mentioned, the cool thing about this grant portfolio that the AMA is putting together is it's got programs geographically deserved diverse.
00:34:29.199 --> 00:34:40.480
It's got programs that focus on undergraduate medical education, the, the, the transitions from UME to GME, the transitions from GME to practicing faculty across different specialties.
00:34:40.480 --> 00:34:43.679
So I'm really hopeful to learn a lot in this process as well.
00:34:43.679 --> 00:34:53.840
And so it's just humbling always to work with fantastic people and just realize how committed and smart people are and to try to contribute whatever we can in that regard as well.
00:34:54.239 --> 00:34:58.880
Well, Matt, thank you for joining us uh for pushing this conversation forward.
00:34:58.880 --> 00:35:06.400
I know data is uh not necessarily the easiest topic for some, you know, for many people to understand.
00:35:06.400 --> 00:35:10.480
And I appreciate you sharing what you know with us today.
00:35:10.480 --> 00:35:12.320
Uh, VG, any final words?
00:35:12.719 --> 00:35:27.679
No, I just want to thank uh Matt for being here and for just you know sharing his uh thoughts and helping us and helping our listeners understand a little bit more about how we can think about precision education.
00:35:27.679 --> 00:35:46.960
And today was all about how data is uh, you know, which underlies everything else, how data can be um used uh well uh ethically and uh with guardrails placed before we even begin instituting all of this.
00:35:46.960 --> 00:35:50.239
So thank you so much, uh Matt, for being here.
00:35:50.239 --> 00:35:56.320
Uh and uh we want to thank uh the AMA Changement, I guess, because I'm excited to see what's gonna come out of it.
00:35:56.719 --> 00:36:01.599
Thank you to our listeners for being part of the Precision Educator podcast.
00:36:01.599 --> 00:36:07.840
In our next episode, we'll explore how these ideas come together across learners, careers, and institutions.
00:36:07.840 --> 00:36:16.159
Until then, you've been listening to the Precision Educator, where we explore how data, coaching, and innovation are reshaping medical education.
00:36:16.159 --> 00:36:26.159
Our goal is to move beyond one size fits all approaches and towards learning that is thoughtful, personalized, and grounded in real-world practice.
00:36:26.159 --> 00:36:35.039
If you found today's episode valuable, please subscribe, share it with your colleagues, and connect with us through the Society for Education in Anesthesiology.
00:36:35.039 --> 00:36:36.559
Until next time, I'm Dr.
00:36:36.559 --> 00:36:38.880
Larry Chu with my co-host B.
00:36:38.880 --> 00:36:42.559
And this has been the Precision Educator.