Teaching Observations with Dr. Michael Kelly의 전체 녹취록

And so I wanted feedback on my teaching and maybe just how it was delivered in a new setting. So I had had other teaching observations at my previous job, but I wanted to kind of see what the experience looked like from the Jefferson point of view. 00:05:48.800 --> 00:06:14.166 Dr. Michael Kelly And then I wanted something specific and tangible to put into my promotion dossier that would reflect my teaching style and and maybe the strengths and any weaknesses. And so I think the process, I believe I met with some of these folks two weeks in advance to kind of go over what the topic was, what the plan was for delivering the lecture, and any concerns that I had regarding the process. 00:06:14.200 --> 00:06:31.200 Dr. Michael Kelly And so I think it was really helpful to to kind of have that pre meeting to discuss what is what's going to look like, what the process looks like. And then for me to have a plan for what am I going to do. How am I going to do it. And kind of why I'm doing certain things. 00:06:31.266 --> 00:06:57.300 Leah Miller That's great. And I believe I actually did your observation. Yeah. So I remember those meetings. So aside from the kind of like pre meeting that like pre-observation check in. Do you do anything different to prepare for an observation versus how you would prepare for a normal lesson. 00:06:57.366 --> 00:07:23.433 Dr. Michael Kelly Yeah I think having the the observation really kind of forces me or encourages me to spend more time like planning out the lecture and what am I going to do actually in the classroom for this example that I was observed for, I created a short like 15 minute prerecorded video that I wanted students to watch before they came to the lecture, as most of that lecture time was dedicated to kind of that case based learning. 00:07:23.533 --> 00:07:40.700 Dr. Michael Kelly So again, it kind of forced me to think about what am I going to do in class, what do I want students to do before class and then after class? So having that observation planned, I guess, really kind of put the emphasis on me to think about what am I doing, why am I doing it? And then carefully kind of craft the lecture. 00:07:40.800 --> 00:08:01.233 Jacob Santos I love that teaching observation kind of puts us in, that puts us on our toes a little bit. As a class former classroom teacher myself, knowing I was going to be observed, maybe I paid a little bit more attention to what I was doing that day to make sure my all my, you know, my Is were dotted, my t's across, you know, everything I had on my handouts, everything was ready to go. 00:08:01.366 --> 00:08:23.000 Jacob Santos So you do the lesson. You, you you put your best foot forward out there, and then you have you and then have that post meeting with the person who observes you. How do you best manage the feedback you receive from the outside observer and decide what to adjust for next time? 00:08:23.100 --> 00:08:52.633 Dr. Michael Kelly Yeah, I think again, it's really helpful to have kind of this outsider point of view. You know, I know I have an intuition of what went well. When I deliver the class or the lecture, I have some kind of feedback from students, but I think the comments from the observers is really helpful. For example, my kind of suggestions for improvement reveal that some of the cases maybe didn't require as much time as I had allocated, but some of the cases actually needed more time. 00:08:52.633 --> 00:09:12.033 Dr. Michael Kelly So it was really helpful for me to go back and think about, do I need the same amount of time for each case, or can I adjust it based on the complexity of the case? So that was really helpful. And I think the other piece of feedback that I got was that I could, I could lean on the students a little bit more to kind of tell me when they were done. 00:09:12.066 --> 00:09:37.200 Dr. Michael Kelly So the other piece of feedback I got that was really helpful was that consider asking the student who needs more time just to make sure that everyone's getting enough practice time to complete the case before I move on. So I think the feedback that they provided was well intended, intended, and it was useful for for me to kind of consider and making changes to to the next time I do that lecture. 00:09:37.300 --> 00:09:52.866 Leah Miller That is great to hear that the feedback was useful. And that kind of leads nicely into our next question and kind of impacted your style and methods. 00:09:52.966 --> 00:10:15.833 Dr. Michael Kelly Yeah, I think again, one of the pieces of feedback I got from the first time I did this was that because the observers were stationed in the back of the classroom, they kind of got a different point of view. And what they had noticed was that some of the students, particularly ones that may place themselves in the furthest part of the classroom, maybe were less engaged during the case discussion piece. 00:10:15.866 --> 00:10:47.800 Dr. Michael Kelly So one of the potential suggestions for improvement was to offer some sort of student engagement so that everyone can kind of participate, not just groups of student or those that are more likely to contribute. So as a result, I've started to use more kind of anonymous polling technologies like Nearpod or Kahoot to let all students really have the ability to participate if they want, as some students are a little bit shy and maybe don't want to verbally offer their answers. 00:10:47.866 --> 00:11:00.800 Dr. Michael Kelly So having that kind of anonymous, low stakes, formative feedback allows everyone to kind of participate. And so that's probably the biggest thing I've incorporated as a result of my teaching feedback. 00:11:00.866 --> 00:11:30.800 Jacob Santos We love to see the use of our edtech tools like Nearpod in the classroom. Always want to always love to hear that they're being used. It makes us happy. So we've talked about your using the feedback from the observer, how it's

kind of changed a little bit, how you structure your lesson, that example you just talked about how you're using this tool to help get more student feedback in the moment, allowing them to interact and engage with you perhaps more than they were in the past. 00:11:30.866 --> 00:12:00.266 Jacob Santos So what I'd like to move into then with this focus on students. I'd like to like to jump into your work with the pharmacy residency and its students, and it's clear that you value feedback in your teaching. You've listened to it, applied it, and it's it's benefited your teaching. So how did that develop into the pharmacy resident teaching certificate program? 00:12:00.366 --> 00:12:16.700 Dr. Michael Kelly Yeah. So I had been a member of teaching certificate programs for pharmacy residents as both a resident and then as a new faculty member. And so it was something that I was really interested in, in having an affinity for academia. It's something that, you know, I want to be able to share with others who have that same interest. 00:12:16.900 --> 00:12:44.200 Dr. Michael Kelly And so in 2024, we launched our Jefferson Teaching Certificate program for our pharmacy residents, PGY one and PGY two residents, as a really a way to kind of engage them in the teaching process and to hopefully provide some sort of structured framework for how to deliver a lecture, how to create meaningful learning objectives, and how to, I guess, appropriately assess those learning objectives through assessment questions. 00:12:44.433 --> 00:12:54.866 Dr. Michael Kelly And so we've incorporated the the modified version of kind of the observation tool for all residents starting in 2024 and it's something we continue today. 00:12:54.866 --> 00:13:13.166 Leah Miller So when you this is kind of a follow up to that. So the teaching certificate program observations were kind of considered as part of the process from the get go. Those weren't like added on later like, oh we should do this. It was. 00:13:13.233 --> 00:13:36.733 Dr. Michael Kelly Yeah. No very much. We wanted to have again, kind of that unbiased point of view because the residents will get feedback from their faculty mentor, they'll get feedback from students. But again, I think having kind of an unbiased person, just providing observations is sometimes helpful as as we, the faculty maybe don't see things that they do. And then student evals can be kind of biased, positively or negatively. 00:13:36.733 --> 00:13:48.066 Dr. Michael Kelly So having someone just providing observation, I think is really important for the residents to take into account and kind of think about what would they do different for their second, their second teaching experience. 00:13:48.233 --> 00:13:49.633 Leah Miller Wonderful. 00:13:49.700 --> 00:14:10.333 Jacob Santos Now, you Michael have been in a unique situation in this, particularly through this program, and that you have been in the role of an observer, having been observed as

part of your teaching career and providing feedback. How does that process affect your own teaching practices? Being the observer, providing feedback? 00:14:10.400 --> 00:14:42.533 Dr. Michael Kelly Yeah, I think it's given me a much more kind of templated approach to designing lectures, which then we we share with the residents to kind of just have a structure. I think it's easy to maybe disengage from that and just think, all right, I'm going to show up with 45 slides and blow through it. But having been observed and had that opportunity to to think about what I want to do and how do I want to do it, I think then kind of changes my approach to teaching more so of, you know, I think it used to be kind of like, I'm the teacher. 00:14:42.533 --> 00:14:58.266 Dr. Michael Kelly This is what I'm going to do. I think my approach now is much more, all right, I'm a facilitator of learning. What am I going to do? And I think, you know, having that observation and thinking about what you're going to do and why, what the purpose is, has given me the kind of perspective of less. I'm the teacher I’m the expert. 00:14:58.300 --> 00:15:13.200 Dr. Michael Kelly Listen to every word I say, but I'm going to design an activity or activities that will help support your learning. And then using that approach or template for the residents, I think has been really helpful in guiding their teaching style. 00:15:13.300 --> 00:15:17.866 Dr. Michael Kelly And I don't know if that actually answers your question or not. That's what I wrote down. So no. 00:15:17.966 --> 00:15:40.866 Jacob Santos No, that that that I love that you are looking at the impact on yourself in the sense of thinking about how are you structuring your lessons and your lectures and having a template and structure that makes it easy for others to pick up the practice and implement in their own. And how you implementing that yourself helps these students benefit and grow in the teaching practice. 00:15:40.866 --> 00:15:42.400 Jacob Santos So that's good. That's perfect. 00:15:42.400 --> 00:16:07.566 Leah Miller I also love what you said about like being a facilitator of learning and watching some of the changes from like the first observation of the students to the next semester. And that actually, I'm curious, what impact have you noticed? Like, can you give us specifics about what impact have you noticed that these observations have on the pharmacy residents? 00:16:07.666 --> 00:16:28.166 Dr. Michael Kelly Yeah, I think the residents really appreciate the feedback as it's it's coming from a place of of intent. Right. It's not students who really liked it and just say everything's great. You're the best person or students who didn't like their presentation going, I wish you wouldn't teach anymore. It really is kind of an unbiased, an outsider observation, right? 00:16:28.200 --> 00:17:02.200 Dr. Michael Kelly Like the observation team has no has no influence over whether the resident does or doesn't do anything regarding their program. And I think it's really helpful in that the way we've structured their teaching observation is looking at kind of the lecture organization, the presentation skills and then student engagement. And I think having the residents see that that observation rubric, if you will, gives them the opportunity to then put more thought into how am I going to organize, how am I going to engage, what's going to be my teaching style? 00:17:02.266 --> 00:17:19.300 Dr. Michael Kelly And so and then at the end, once they receive the feedback from the students, from their faculty mentor and from the observational team, I think it's been really helpful for them to kind of think about, what did I do well, what what could be improved, and how would I make that improvement with the suggestions from from all of the feedback they get? 00:17:19.366 --> 00:17:38.033 Jacob Santos I love that you, as part of this process with them, you engage, you have them engage in that reflection element that is so crucial to the practice of teaching every [...] In an ideal world, if we had all the time in the world, after each of our lessons, we would be able to sit down and kind of think about what went well, what didn't. 00:17:38.066 --> 00:17:50.033 Jacob Santos Unfortunately, we can't always do that every time, right? But having that outside voice of the observer to help us with that process is so is such a gift in many ways for us. 00:17:50.100 --> 00:18:11.033 Dr. Michael Kelly And I think the the voice that or the observation that's coming really is well intended. I haven't read any remarks that seemed negative. It's really like supportive of what they do. Well, and then it's framed in a way that's, you know, consider doing the following to increase learner engagement or consider doing the following to better structure your lecture. 00:18:11.033 --> 00:18:18.366 Dr. Michael Kelly So what it really is a helpful tone, if you will, so that I think it's well received. 00:18:18.466 --> 00:18:47.500 Leah Miller That's wonderful to hear. We do we do try and at least from our perspective, like we do try and offer like, hey, this is a suggestion just to take things to the next level or this might help the lesson have better pacing if you try this. So everything is suggestions. It's a back and forth process. You mentioned something actually in the observation of these pharmacy residents that I kind of hadn't considered. 00:18:47.500 --> 00:19:16.633 Leah Miller And now I want to ask a little follow up about it. But you said there are there's like the faculty mentor, there's the faculty observer, and then there's the outside of server. I'm curious about the different kinds of feedback they get from those three different perspectives, and how that all like, that's a lot of feedback for a new to teaching learner. 00:19:16.700 --> 00:19:34.000 Leah Miller And Im just curious kind of how that all works. If different observers pick out different

things, like, I know I can't comment on the accuracy of what they're saying because I know nothing about pharmacy practice. But yeah, so if you could expand on that. 00:19:34.000 --> 00:19:58.266 Dr. Michael Kelly And I think that's a great benefit of it, is the teaching observations from the instructional design team really are kind of focused on how was this lecture delivered and how is it received, perhaps from from their point of view, me as the faculty mentor, I'm looking more at, you know. Did you hit the like clinical high points? Did you deliver the information that was necessary? 00:19:58.266 --> 00:20:16.566 Dr. Michael Kelly So my point of view really is kind of on the clinical piece of, did you bring home the most clinically important points that are practicing pharmacists needs to know? So that's one one piece of it. But I think the observation piece then supplements, you know, the things that maybe I'm not looking at or not looking at, as with a greater spotlight. 00:20:16.566 --> 00:20:37.733 Dr. Michael Kelly And then the student evals, while mostly positive again, I don't think really provide any like information on what to do differently. Right. It's kind of like, yes, you did a good job or like, no, I don't like your slide fonts or whatever, but it's not really actionable. I don't know what to do with that information. So for the most part, I think residents see the student evals go, okay. 00:20:37.733 --> 00:20:52.566 Dr. Michael Kelly I feel like I did good based on what the student said. I feel like I kind of hit the clinical points, but it is that that unbiased, observational piece that I think is, is what they kind of take away and say, oh, I hadn't thought about that, or I didn't think that that's how this came off. So something to consider. 00:20:52.566 --> 00:20:58.333 Dr. Michael Kelly So again, I think they all work separately but together. I love that symbiosis. 00:20:58.400 --> 00:21:35.166 Jacob Santos Yeah, that symbiosis of you get an observation from a content perspective and then a teaching and learning practice perspective. And merging those two together is wonderful. So Michael, unless you have any other topics or points you wanted to address from teaching and learning for for teaching observation rather for yourself, you're going to enter our last question of our interview today, and I'm going to ask what 1 or 2 pieces of advice you would like to give to someone, such as another faculty member here at Thomas Jefferson University who is interested in pursuing a teaching teaching observation, maybe hasn't done one. 00:21:35.166 --> 00:21:39.366 Jacob Santos Maybe they’re a bit nervous. What advice would you give? 00:21:39.433 --> 00:22:01.000 Dr. Michael Kelly That's a great, great question, Jacob. And I think I have a two word answer for this one, which would be do it. I think the process of having the teaching observation is beneficial for anyone, whether they're new to teaching. And this is maybe like their first year of lecturing or someone who's been doing the same lecture for, you know, dozens of years. 00:22:01.033 --> 00:22:28.900 Dr. Michael Kelly I think having that observation is really helpful to make one deliberately think about what they're doing, how they're doing, and why they're doing it. And so anyone who's maybe looking for a new looking to incorporate a new teaching strategy, I think this would be a great way to kind of test that out and think about it with the team before they execute, and then get feedback on maybe how that the delivery or execution of that new teaching style or technology landed. 00:22:29.033 --> 00:22:44.900 Dr. Michael Kelly But I think it could also be really helpful for someone who's been kind of doing the same lecture for the same audience for, for multiple years. Maybe there are new ways or new things that could be incorporated. Again, I think I learned a lot from the first time I did it, just different perspectives and different things to try. 00:22:44.966 --> 00:22:49.800 Dr. Michael Kelly So I think anyone could benefit from this. I would encourage everyone to take advantage of it. 00:22:49.833 --> 00:23:17.900 Leah Miller That is wonderful. We're so happy to hear that you look at this so positively and I love Just Do it. Especially new, experienced whatever if you want to try something different. I love the way that you put that together. So that brings us to the end of our interview for the day. Michael, thank you for taking the time out of your morning to be here with us and share your knowledge and expertise. 00:23:17.900 --> 00:23:19.533 Leah Miller We appreciate it. 00:23:19.633 --> 00:23:21.333 Dr. Michael Kelly Truly a pleasure. 00:23:21.433 --> 00:23:27.166 Jacob Santos Thank you very much Michael. This is wonderful. 00:23:27.266 --> 00:23:40.166 Jacob Santos Thank you for joining us this month on The Learning Curve. You can find more information and resources related to this episode at Academic Commons. Jefferson. 00:23:40.233 --> 00:23:52.400 Jacob Santos And we hope to see you at some upcoming Academic Commons workshops. If you would like to be featured on a future episode of The Learning Curve. Please contact us using the button on our show page. Thanks for tuning in.