Dr. Eric Holmboe on the ACGME’s Milestones Project [ASN Podcast Podcast]
Dr. Eric Holmboe on the ACGME’s Milestones Project
ASN Podcast
Published August 11, 2014 * 00:14:13
ACGME Senior VP Eric Holmboe, MD, discusses implementation of the Milestones Project in nephrology fellowship training with ASN Executive Director Tod Ibrahim.
Please note that this transcript was automatically generated and may contain inaccuracies. It is intended for informational purposes only. Refer to the audio for full context.
----Speaker 1: On July 1, 2014, Nephrology Fellowship Training Programs began implementation of the Milestones Initiative, part of ACGME's NEXT accreditation system. The milestones outline skills and competencies that fellows must demonstrate in order to practice unsupervised in real-world clinical settings. In this issue of the ASN Kidney News Podcast, ASN Executive Director Todd Ibrahim speaks with ACGME's Senior Vice President Dr. Eric Holmbo about the Milestones effort and how they will shape the future of medical education.
Speaker 2: Dr. Holmbo, could you help us understand sort of the overall philosophy behind the Milestones effort?
Speaker 3: I think the primary purpose of the milestones effort is really for the first time in my career to try to better define a discipline across all the competencies. I think the best way to think about the milestones is kind of a more comprehensive blueprint or roadmap that moves us beyond our primary focus in the past on medical knowledge and clinical skills, as important as those still are, to make sure that we now include other aspects of clinical practice that are really critical, and that's where I think the competencies have been helpful, but we also know that the competencies when they were first released were often hard to understand. We didn't really have good descriptions of what they were and the kind of things that we were looking for in our trainees. And so the milestones effort is really an attempt using more narrative and description to describe in developmental terms where we want our trainees to be over the course of their educational trajectory.
Speaker 2: And so we've had milestones at a residency level for a couple of years. How have those worked?
Speaker 3: The milestones for residents have really only gone into kind of widespread use this past November/December. We had five specialties that reported for the first time, and we did have some internal medicine and pediatric programs kind of voluntarily enter. But a third, actually, of internal medicine programs enter the system, and they'll kind of all go live, if you will, in May and June. Having said that, there's also been a fair amount of pilot work that's been done. As part of the milestones initiative, working with, for example, ABIM and the AIM group, we've done some survey work over the years, and we've also run some pilots with competency committees to try to get some early learnings around the milestones. A couple themes came out of the surveys. One was that it certainly was nice to have a better description of what we mean by competence and a better description of the competencies. The second thing actually has come from the residents in those surveys and also from the council review committee residents that the ACGME has here, that they too enjoy having a more explicit blueprint and roadmap. One of my favorite quotes from a survey from a couple years ago was like, Oh, Now I know what's expected of me. This is what you want me to look like when I finish. And I think it's a pretty powerful message that it really helps the trainee to have a better idea, in this case the resident, about where it is we hope they'll end up and that the purpose of the educational program. is honestly to help them get there. With regard to the competency committees, as you can imagine, those who had been working with some of the more granular milestones, sometimes called curricular milestones, that were published in 2009, and had already begun to evolve their tools in that way, they found use of the kind of reporting milestones a little bit easier. And they found that it did help guide their conversations. The other programs that were just beginning the kind of alignment between the assessments and the milestones as you might imagine, struggled a bit more because the assessment tools that they were using didn't line up quite as well with the kind of milestones concept. Some of this work was published by Ava Agart in the Journal of Graduate Medical Education recently, and so I'd encourage anybody who's interested to take a look at that article. It does a nice job of kind of talking about some of the early experiences. So we do have some early signals. And certainly moving forward, we're going to need to do a lot of evaluation work to make sure the system is accomplishing the goals we set out for it.
Speaker 2: So as we look at the subspecialties in internal medicine, recently all the subspecialties worked together to develop a set of core milestones for fellowship training, regardless of specialty. My understanding is that those will be implemented on July 1. How will they link to the internal medicine residency milestones?
Speaker 3: First, I would just call out what I thought was just an incredibly effective and wonderful collaboration among the specialties of internal medicine. Obviously, ASM was involved with that. Kudos to all of you for being part of that. And I think watching that community work together was really terrific. I think the link to residency is something that honestly is yet to be defined, Todd. We hope that as people come out of residency and they've reached a certain level, that they'll be prepared to kind of enter effectively fellowship and kind of pick up from there. Right now, we don't have a system of feed forward. Certainly, the resident could take that with them, and that at this time will be up to them as they enter the fellowship, and it would be a really nice starting point and sit down with their fellowship director and say, hey, this is where my program thought it was across these competencies. I think that'll be particularly useful in things like professionalism, teamwork and their personal communication skills, systems-based and practice-based learning and improvement, to kind of see where they are so they can continue to advance that. Fellowship is shorter, you know, for most of the specialties. So there's an acceleration or kind of a density with fellowship that's a bit unique. Obviously, the content portions of fellowship are critical. And the medical knowledge and clinical skills obviously are a really important part of that. And I think what the medicine folks have decided to do is not get specific around the conditions and content within the milestone, but leave that to each specialty to define. Some are doing that through a curricular milestone process others obviously have curriculum in that particular domain. Other specialties, by the way, have not they've decided to embed them in their milestones. We'll see we'll have to follow that pretty closely, but one of the things we hope with the milestones over time... is that it will break down these kind of abrupt transitions or silos between the various stages of training. What's interesting, the milestones have driven activity in the undergraduate medical education space. You may be aware that the AAMC just recently published what they call graduation and trustable professional activities that are mapped to a set of the competencies and some milestones. And certainly now medicine has begun that journey of looking at kind of starting with the residency set and then building those out into fellowship. So a good example of one in fellowship that's kind of what I would call the next stage is particularly around procedures. So the procedural milestone, which is divided up into kind of invasive and non-invasive, really is a recognition of the kind of the advanced training and deeper focus that you get in fellowship. But this is something we're going to need to follow pretty closely. And again, this is something that I'll be really looking to the community to kind of help with as we make sure that these links, this integration across the continuum, you know, continues to occur.
Speaker 2: I'm trying to work on the analogy of the blueprint. So is this a reasonable description, which is for residency training, the blueprint, let's say we're building a house, would be the basic structure and the layout of the different rooms. And then the fellowship milestones would be the differentiation of this is a bathroom, this is the living room, this is an office, and the detail at that level. And then perhaps the curricular opportunity for a group like ASN or for the nephrology community would be the interior designer who says this is going to be the color scheme, this is going to be how we're going to decorate, say, the living room. Is that a way to think of this?
Speaker 3: Yeah, I think that is a reasonable way to think about it. I mean, obviously, as you begin to specialize, your focus setting, if you will, does become more specific. And I think that's where groups like ASN can really be very important. I think the other thing that your analogy alludes to, Tom, that's important. is that from an accreditation and really even a certification point of view, both those entities are not dictating the curriculum. In fact, the review committees have pulled back on curriculum requirements. I mean, you and I both remember the days when you had to have four and a half hours of attending rounds, you know, stuff like that. And that didn't work very well because, you know, context is so important. And so the curriculum, the experiences to attain the outcomes or the goals is really going to be an important part what the community does. And that's where I see particularly organizations like yourself really helping to fill that in, you know, to make that really rich and robust and integrating that curriculum with the assessment so that we know that when folks come out of training, they're really ready to be a highly competent nephrologist when they leave that fellowship.
Speaker 2: So if you were to walk forward over the next couple of years, as both the milestones, I guess, specifically are implemented, but also the next accreditation system comes to reality, what would be the big steps? I mean, just sort of help our audience understand the process and the steps moving forward over the next couple of years.
Speaker 3: Seven specialties began using the milestones in July of 2013, five reported for the first time in November/December of 2013, and then internal medicine and pediatrics will do their first reports in May/June of 2014. The subspecialties of internal medicine, including nephrology, will begin using the milestones in July of 2014 and will report for the first time in November/December of 2014 as well. And so we'll be, you know, tracking the implementation pretty closely. So, for example, within the Milestone Reporting System, we can track how long it takes somebody to enter the data. We can look at medians, we can look at ranges, averages, we can see if there's problems, and we'll really want to take a close look at what I call the kind of feasibility issues around the IT systems and the utility of that. The last thing we want to do is make this really difficult, because then the system will fail. So we'll be tracking those sort of metrics pretty closely. The second thing we'll be looking for, obviously, is kind of patterns and trajectories and trying to learn what those look like and begin to think, what is it about various programs and experiences that lead to certain types of trajectories? That's going to take a little bit longer. It's probably more of a two-year timeframe, unless we're implementing it. The third thing we'll do is we'll certainly, now that the milestones are almost all completed, by the way, there's over 60 sets of these things, we'll begin to do a crosswalk of all the milestones, again, looking for those that really appear to have some strengths where they might help others. Look for where, as you mentioned, Todd, we notice gaps that may be already existing, some of the milestones, begin to think about how we might help that particular discipline move forward as they think about the next revision around some of them. So we'll be doing a fair amount of that work and trying to link it as much as possible to the evidence and the literature around the various competencies about what's important, what skills are needed, et cetera. The other important part of the system that we're going to begin to take a look at, and there's some research going on now that I was involved with at ABIN and will continue to expand here. is that part of the crux of the system is this clinical competency committee. And the reason for that is the need for group decision making to really enhance the probability of a good judgment, if you will. We don't know a whole lot about that. There's been some work around group process. Most of it is actually outside of medicine. The stuff that's occurred inside medicine has been very encouraging. One of the things even early on that people ask me about is, listen, we really don't know how to run a clinical competency committee. We need some help. And so what I'm working on is trying to get some helpful guidance up onto the web so that people at least have some ideas about how to run that process. Because that's kind of one of the important inflection points where you make decisions on how somebody's doing. And obviously, that's the kind of last step before it gets into the kind of report that comes with HCGME. So we're going to want to take a pretty good look at that. The other big bucket is obviously continuing to work to help align assessments with the milestones. And I think there's some interesting work that's been done already. For example, Chris Neighbors at West Chester has done a wonderful job mapping the original curricular milestones to specific goals of rotations, so that when they ask faculty to look for stuff, it's very focused. It's like, listen, these are the eight to 10 things I want you to look for in this rotation. This is what this rotation's about. Don't worry about the other stuff. This is what I want you to look for in the trainees and tell us how they're doing. And really thinking in terms of a sampling frame and how that experience and that assessment's going to tie to the larger whole when the competency committee meets.
Speaker 2: Dr. Humbo, thank you for joining us for today's discussion.
Speaker 3: Thank you, and I wish you all the best.
Speaker 1: This podcast is copyrighted by the American Society of Nephrology. All rights reserved. All content in this podcast is for informational purposes only and is not intended to be medical advice. This podcast should not be used in a medical emergency for the diagnosis or treatment of any medical condition. Please consult your doctor or other qualified healthcare provider if you have any questions about any medical condition or before taking any drug, changing your diet, or commencing or discontinuing any course of treatment. Thank you for listening to this podcast of the American Society of Necrology.----Episode URL: https://www.asn-online.org/media/podcast.aspx?ID=79
Dr. Eric Holmboe on the ACGME’s Milestones Project
ASN Podcast
Published August 11, 2014 * 00:14:13
ACGME Senior VP Eric Holmboe, MD, discusses implementation of the Milestones Project in nephrology fellowship training with ASN Executive Director Tod Ibrahim.
Please note that this transcript was automatically generated and may contain inaccuracies. It is intended for informational purposes only. Refer to the audio for full context.
----Speaker 1: On July 1, 2014, Nephrology Fellowship Training Programs began implementation of the Milestones Initiative, part of ACGME's NEXT accreditation system. The milestones outline skills and competencies that fellows must demonstrate in order to practice unsupervised in real-world clinical settings. In this issue of the ASN Kidney News Podcast, ASN Executive Director Todd Ibrahim speaks with ACGME's Senior Vice President Dr. Eric Holmbo about the Milestones effort and how they will shape the future of medical education.
Speaker 2: Dr. Holmbo, could you help us understand sort of the overall philosophy behind the Milestones effort?
Speaker 3: I think the primary purpose of the milestones effort is really for the first time in my career to try to better define a discipline across all the competencies. I think the best way to think about the milestones is kind of a more comprehensive blueprint or roadmap that moves us beyond our primary focus in the past on medical knowledge and clinical skills, as important as those still are, to make sure that we now include other aspects of clinical practice that are really critical, and that's where I think the competencies have been helpful, but we also know that the competencies when they were first released were often hard to understand. We didn't really have good descriptions of what they were and the kind of things that we were looking for in our trainees. And so the milestones effort is really an attempt using more narrative and description to describe in developmental terms where we want our trainees to be over the course of their educational trajectory.
Speaker 2: And so we've had milestones at a residency level for a couple of years. How have those worked?
Speaker 3: The milestones for residents have really only gone into kind of widespread use this past November/December. We had five specialties that reported for the first time, and we did have some internal medicine and pediatric programs kind of voluntarily enter. But a third, actually, of internal medicine programs enter the system, and they'll kind of all go live, if you will, in May and June. Having said that, there's also been a fair amount of pilot work that's been done. As part of the milestones initiative, working with, for example, ABIM and the AIM group, we've done some survey work over the years, and we've also run some pilots with competency committees to try to get some early learnings around the milestones. A couple themes came out of the surveys. One was that it certainly was nice to have a better description of what we mean by competence and a better description of the competencies. The second thing actually has come from the residents in those surveys and also from the council review committee residents that the ACGME has here, that they too enjoy having a more explicit blueprint and roadmap. One of my favorite quotes from a survey from a couple years ago was like, Oh, Now I know what's expected of me. This is what you want me to look like when I finish. And I think it's a pretty powerful message that it really helps the trainee to have a better idea, in this case the resident, about where it is we hope they'll end up and that the purpose of the educational program. is honestly to help them get there. With regard to the competency committees, as you can imagine, those who had been working with some of the more granular milestones, sometimes called curricular milestones, that were published in 2009, and had already begun to evolve their tools in that way, they found use of the kind of reporting milestones a little bit easier. And they found that it did help guide their conversations. The other programs that were just beginning the kind of alignment between the assessments and the milestones as you might imagine, struggled a bit more because the assessment tools that they were using didn't line up quite as well with the kind of milestones concept. Some of this work was published by Ava Agart in the Journal of Graduate Medical Education recently, and so I'd encourage anybody who's interested to take a look at that article. It does a nice job of kind of talking about some of the early experiences. So we do have some early signals. And certainly moving forward, we're going to need to do a lot of evaluation work to make sure the system is accomplishing the goals we set out for it.
Speaker 2: So as we look at the subspecialties in internal medicine, recently all the subspecialties worked together to develop a set of core milestones for fellowship training, regardless of specialty. My understanding is that those will be implemented on July 1. How will they link to the internal medicine residency milestones?
Speaker 3: First, I would just call out what I thought was just an incredibly effective and wonderful collaboration among the specialties of internal medicine. Obviously, ASM was involved with that. Kudos to all of you for being part of that. And I think watching that community work together was really terrific. I think the link to residency is something that honestly is yet to be defined, Todd. We hope that as people come out of residency and they've reached a certain level, that they'll be prepared to kind of enter effectively fellowship and kind of pick up from there. Right now, we don't have a system of feed forward. Certainly, the resident could take that with them, and that at this time will be up to them as they enter the fellowship, and it would be a really nice starting point and sit down with their fellowship director and say, hey, this is where my program thought it was across these competencies. I think that'll be particularly useful in things like professionalism, teamwork and their personal communication skills, systems-based and practice-based learning and improvement, to kind of see where they are so they can continue to advance that. Fellowship is shorter, you know, for most of the specialties. So there's an acceleration or kind of a density with fellowship that's a bit unique. Obviously, the content portions of fellowship are critical. And the medical knowledge and clinical skills obviously are a really important part of that. And I think what the medicine folks have decided to do is not get specific around the conditions and content within the milestone, but leave that to each specialty to define. Some are doing that through a curricular milestone process others obviously have curriculum in that particular domain. Other specialties, by the way, have not they've decided to embed them in their milestones. We'll see we'll have to follow that pretty closely, but one of the things we hope with the milestones over time... is that it will break down these kind of abrupt transitions or silos between the various stages of training. What's interesting, the milestones have driven activity in the undergraduate medical education space. You may be aware that the AAMC just recently published what they call graduation and trustable professional activities that are mapped to a set of the competencies and some milestones. And certainly now medicine has begun that journey of looking at kind of starting with the residency set and then building those out into fellowship. So a good example of one in fellowship that's kind of what I would call the next stage is particularly around procedures. So the procedural milestone, which is divided up into kind of invasive and non-invasive, really is a recognition of the kind of the advanced training and deeper focus that you get in fellowship. But this is something we're going to need to follow pretty closely. And again, this is something that I'll be really looking to the community to kind of help with as we make sure that these links, this integration across the continuum, you know, continues to occur.
Speaker 2: I'm trying to work on the analogy of the blueprint. So is this a reasonable description, which is for residency training, the blueprint, let's say we're building a house, would be the basic structure and the layout of the different rooms. And then the fellowship milestones would be the differentiation of this is a bathroom, this is the living room, this is an office, and the detail at that level. And then perhaps the curricular opportunity for a group like ASN or for the nephrology community would be the interior designer who says this is going to be the color scheme, this is going to be how we're going to decorate, say, the living room. Is that a way to think of this?
Speaker 3: Yeah, I think that is a reasonable way to think about it. I mean, obviously, as you begin to specialize, your focus setting, if you will, does become more specific. And I think that's where groups like ASN can really be very important. I think the other thing that your analogy alludes to, Tom, that's important. is that from an accreditation and really even a certification point of view, both those entities are not dictating the curriculum. In fact, the review committees have pulled back on curriculum requirements. I mean, you and I both remember the days when you had to have four and a half hours of attending rounds, you know, stuff like that. And that didn't work very well because, you know, context is so important. And so the curriculum, the experiences to attain the outcomes or the goals is really going to be an important part what the community does. And that's where I see particularly organizations like yourself really helping to fill that in, you know, to make that really rich and robust and integrating that curriculum with the assessment so that we know that when folks come out of training, they're really ready to be a highly competent nephrologist when they leave that fellowship.
Speaker 2: So if you were to walk forward over the next couple of years, as both the milestones, I guess, specifically are implemented, but also the next accreditation system comes to reality, what would be the big steps? I mean, just sort of help our audience understand the process and the steps moving forward over the next couple of years.
Speaker 3: Seven specialties began using the milestones in July of 2013, five reported for the first time in November/December of 2013, and then internal medicine and pediatrics will do their first reports in May/June of 2014. The subspecialties of internal medicine, including nephrology, will begin using the milestones in July of 2014 and will report for the first time in November/December of 2014 as well. And so we'll be, you know, tracking the implementation pretty closely. So, for example, within the Milestone Reporting System, we can track how long it takes somebody to enter the data. We can look at medians, we can look at ranges, averages, we can see if there's problems, and we'll really want to take a close look at what I call the kind of feasibility issues around the IT systems and the utility of that. The last thing we want to do is make this really difficult, because then the system will fail. So we'll be tracking those sort of metrics pretty closely. The second thing we'll be looking for, obviously, is kind of patterns and trajectories and trying to learn what those look like and begin to think, what is it about various programs and experiences that lead to certain types of trajectories? That's going to take a little bit longer. It's probably more of a two-year timeframe, unless we're implementing it. The third thing we'll do is we'll certainly, now that the milestones are almost all completed, by the way, there's over 60 sets of these things, we'll begin to do a crosswalk of all the milestones, again, looking for those that really appear to have some strengths where they might help others. Look for where, as you mentioned, Todd, we notice gaps that may be already existing, some of the milestones, begin to think about how we might help that particular discipline move forward as they think about the next revision around some of them. So we'll be doing a fair amount of that work and trying to link it as much as possible to the evidence and the literature around the various competencies about what's important, what skills are needed, et cetera. The other important part of the system that we're going to begin to take a look at, and there's some research going on now that I was involved with at ABIN and will continue to expand here. is that part of the crux of the system is this clinical competency committee. And the reason for that is the need for group decision making to really enhance the probability of a good judgment, if you will. We don't know a whole lot about that. There's been some work around group process. Most of it is actually outside of medicine. The stuff that's occurred inside medicine has been very encouraging. One of the things even early on that people ask me about is, listen, we really don't know how to run a clinical competency committee. We need some help. And so what I'm working on is trying to get some helpful guidance up onto the web so that people at least have some ideas about how to run that process. Because that's kind of one of the important inflection points where you make decisions on how somebody's doing. And obviously, that's the kind of last step before it gets into the kind of report that comes with HCGME. So we're going to want to take a pretty good look at that. The other big bucket is obviously continuing to work to help align assessments with the milestones. And I think there's some interesting work that's been done already. For example, Chris Neighbors at West Chester has done a wonderful job mapping the original curricular milestones to specific goals of rotations, so that when they ask faculty to look for stuff, it's very focused. It's like, listen, these are the eight to 10 things I want you to look for in this rotation. This is what this rotation's about. Don't worry about the other stuff. This is what I want you to look for in the trainees and tell us how they're doing. And really thinking in terms of a sampling frame and how that experience and that assessment's going to tie to the larger whole when the competency committee meets.
Speaker 2: Dr. Humbo, thank you for joining us for today's discussion.
Speaker 3: Thank you, and I wish you all the best.
Speaker 1: This podcast is copyrighted by the American Society of Nephrology. All rights reserved. All content in this podcast is for informational purposes only and is not intended to be medical advice. This podcast should not be used in a medical emergency for the diagnosis or treatment of any medical condition. Please consult your doctor or other qualified healthcare provider if you have any questions about any medical condition or before taking any drug, changing your diet, or commencing or discontinuing any course of treatment. Thank you for listening to this podcast of the American Society of Necrology.----Episode URL: https://www.asn-online.org/media/podcast.aspx?ID=79
