Group Efforts, Part One: Getting To Know IDC’s Data Manager Data Quality Peer Group

Reach out to us if you want to access Podcast resources, submit questions related to episodes, or share ideas for future topics. We’d love to hear from you!

 

You can contact us via the Podcast page on the IDC website at https://ideadata.org/.

 

### Episode Transcript ###

00:00:01.52  >> You're listening to "A Date with Data" with your host, Amy Bitterman.

00:00:07.34  >> Hey, it's Amy, and I'm so excited to be hosting "A Date with Data." I'll be chatting with state and district special education staff who, just like you, are dealing with IDEA data every day.

00:00:19.50  >> "A Date with Data" is brought to you by the IDEA Data Center.

00:00:24.65  >> Hello, welcome to "A Date with Data." We are kicking off a special series of episodes that are focusing on IDC's data quality peer groups. These groups are facilitated by IDC TA providers to bring state role groups to discuss and collaborate on the data-quality issues that are of greatest importance in states. For this episode, I am joined by the facilitators of the Data Manager Data Quality Peer Group, Kelley Blas and Kristen DeSalvatore. Thank you so much for being here and welcome.

00:00:56.74  >> Thanks, Amy. Thanks for having us.

00:00:59.78  >> Yes, thanks, Amy. It's a pleasure to be here.

00:01:03.13  >> So to start things off for those who might not be familiar with these peer groups, can you tell us what these peer groups are like? Who tends to participate in this particular group? What is the structure and the format like? And what are some of the topics that you cover?

00:01:19.76  >> The Data Manager Data Quality Peer Group is a space for folks to come together to discuss and learn all things IDEA data. So as IDC facilitators, we provide timely information and resources, but there is also sharing by the states and learning from each other.

00:01:41.45  >> I agree with that, Kristen. I feel like though we do discuss topics and come in with data topics each month, the bulk of the learning really comes from the discussion and the information that states are able to bring forward, so we'll set the stage, and then they really take it from there and support each other. So it's also an opportunity for real networking.

00:02:10.15  >> The name probably says it's data managers that are a part of this group.

00:02:15.25  >> Yeah, so all IDEA Part B data managers are welcome.

00:02:18.92  >> Mm-hmm.

00:02:19.56  >> But they can also invite and bring other staff in the SEA that they think will benefit from participation. The meetings are held virtually on Zoom once a month, and as Kelley mentioned, we set the stage, so IDC comes prepared with a structured PowerPoint, but as we also already said, there is ample time and encouragement for interaction and dialogue with us, IDC and with the other states that are on the calls. So IDC is really there to support the states, so discussion and collaboration on all IDEA data subjects are open to discussion whether they are on the agenda or not.

00:03:13.49  >> But we also give states an opportunity to share their resources, so, for example, if we're talking about one of the 618 data collections, and they have a state-developed resource, we love to give them an opportunity to share their resource, which then could also turn into another opportunity for states to network together and collaborate on resources. And states that use IDC-developed resources are also often invited to share how they use it, demonstrate the use of it and share how they may use it either for pre-edit checks, so if they're using an edit check tool, they may share how they use that for their edit checks and for their data quality, or they may share how they use it for their public reporting. But we really like for states to come in and demonstrate the use of their own tools as well as IDC-developed tools.

00:04:09.88  >> I know when I was involved a while ago with those groups, that was one of my favorite parts and I think the states' too was getting a chance to see how other states were tackling some of the same challenges, what they were coming up with in terms of solutions and resources. They can kind of piggyback off each other. I know in some states they might even have reached out after they saw a particular resource, reached back out to that state and said, "Hey, can we hear more about this? We'd really love to maybe adapt something like it for our state." One of the best parts of these groups.

00:04:44.44  >> Yeah, that's always ... That's wonderful when states ask for contact information of each other and plan to follow up offline to take advantage of knowledge in one state that could be useful in another state.

00:04:59.92  >> And thinking about that, another really positive aspect of these groups is that it does feel like a safe space for data managers to share that information and talk about any of the concerns that they may be having with their data, whether they're having errors or even just sharing their resources because those calls are not recorded, and it is a smaller group. They may be more apt to share and discuss openly than the might be if we were trying to invite them to present at a nationwide conference or something, so we're finding that, that safety and that small-group feeling tends to open up more discussion.

00:05:41.63  >> Yeah, definitely.

00:05:43.13  >> In terms of some of the topics that we cover ...

00:05:46.02  >> Mm-hmm.

00:05:47.15  >> ... We really do cover IDEA data collection and reporting from A to Z., right, from the collection and requirements of the collections to the actual submission of the data. In particular, reporting requirements, of course, the SPP/APR is always a big one. EDFacts Data Files timelines and processes are some of our frequent fliers, and something that Kelley and I do is, we work really hard to make sure that folks are aware of all of the resources and services that IDC has available to states. We find that sometimes there are states that just aren't really aware of everything that is available, and the resources and services can really be amazingly helpful to states, so we find that it's really important that states realize what is out there and do our best to share things.

00:06:54.32  >> And one of the things that we've noticed from that sharing, and so in our just common template that we use every month, we touch on the data selection calendar that's posted on IDC website, and we also make sure that we bring up the comprehensive list of resources, and we've found that each month, that there are a-ha moments or discussions about some of the resources that are available. So that's been very helpful to just keep that as a forefront, and one of the first things that we discuss is what's upcoming and then touch on those, that comprehensive list.

00:07:34.44  >> So being very ...

00:07:35.01  >> Yeah.

00:07:35.38  >> ... relevant about what's going on at that time of year for the states and making sure that, that gets covered or at least brought up as a topic that ...

00:07:45.55  >> Right.

00:07:45.67  >> ... folks would have questions about or want to get into.

00:07:48.31  >> Absolutely, and really to provide a service to the states to keep them focused and kind of take a tiny bit of the burden off them to keep them on track, it can be very difficult as a Part B data manager to keep all the balls in the air.

00:08:06.32  >> Yeah, and know what's ... everything that's out there, like you said. IDC has a lot of resources. All of the centers do, and it seems like these groups are able to sort of elevate and point out the ones that are most relevant at that particular moment in time, which, like you said, kind of takes away some of the burden of the data manager having to pour through websites and be looking for things themselves.

00:08:30.60  >> Yes, they don't know what they don't know.

00:08:34.28  >> Very true, so you mentioned more generally a number of topics more generally that you cover, but what are some of the common themes that have been emerging in the last couple months of calls that you've held with this group?

00:08:52.39  >> Well, certainly in the last couple of months, SPP/APR-related issues.

00:08:57.30  >> Mm-hmm.

00:08:58.83  >> There has been a lot of discussion and focus on gathering the data, the process that's used to write the APR, if there's any kind of submission issues, responding to OSEP-required actions and then of course clarification, which has just finished. Some other things, frustration and confusion with the differences between significant discrepancy and significant disproportionality.

00:09:32.28  >> Mm-hmm.

00:09:33.56  >> And the reasonableness of indicator for thresholds, those have really been a topic of discussion lately as well as we've really dove pretty deeply into documentation and processes in general.

00:09:54.52  >> Right, and particularly around the documentation and processes, we've had states talk extensively on how helpful it has been for them to access the resources at IDC for their liaison to come out and document their data processes when they're thinking about their DMS 2.0 work, and in addition to what Kristen stated around SPP/APR and Indicator 4, those questions seem to come up pretty much monthly. There's also often discussion around Indicator 18 and general supervision, thinking about the new things that have come out over the past year and how they're going to collect information. Everybody is ready for the Indicator 18 calculator to come out, so they're really looking forward to ii24, where they're going to be able to see some of the new things that are available to support them and their data.

00:11:00.41  >> A lot going on around the SPP/APR and changes that are coming soon. That's a hot topic, I'm sure.

00:11:10.30  >> Yes.

00:11:11.72  >> Well, and then the other thing that we have found to be just a really, really big topic is that a lot of states are in the process of either developing or redesigning their public reporting dashboards, and so we've had states come on and share what they've developed. We've had states come on and share kind of partial development and ask questions about, how should we present a certain indicator? For example, Indicator 7 is a really rough one, and because there there has so many components and categories to it, how should they present it within the BI tool that they're using? And so there's been a lot of collaboration between states talking about those tools and talking about how they want to present their data publicly, so that has been very rich discussions, and we've found, like we said before, a lot of states are finding opportunities to engage with each other after the calls and say, "Oh, I really like what Georgia is doing, so I need to have discussions later on with their data manager, so that's been a very rich theme that's been coming up.

00:12:19.16  >> And using those calls, too, it sounds like for states to kind of get feedback from their peers who are engaged in the same thing in their state, kind of ideas and what we started on the right track. What ideas do you have for how to continue the work? Those sorts of things, that's kind of another side benefit.

00:12:38.34  >> Absolutely, we had a rich discussion at our last meeting about how to make the data more digestible for stakeholders.

00:12:50.20  >> Hmm.

00:12:51.01  >> Right? And it was a really great discussion with different states chiming in on what they do and struggles that they have had with presenting data to stakeholders.

00:13:04.36  >> Mm-hmm, yeah, that's definitely something that comes up quite a bit, especially around SPP/APR time.

00:13:10.64  >> Yep, that is going to be part of our topical discussion at ii24.

00:13:16.35  >> Great.

00:13:16.90  >> That's what I was going to say. Yeah, that was such a rich discussion. That that's what our data managers chose as their topic to continue that discussion.

00:13:27.45  >> Great, looking forward to hearing that. What are some tips or recommendations that maybe you've shared during these groups, resources, things that maybe you all as former data managers have found to be helpful that other states might benefit from knowing about kind of related to the 618 SPP/APR data, the data-quality topics that come up in this group?

00:13:53.83  >> So as Kelley already mentioned, we have had several states share Data Dashboards and visualizations, and that has really been amazing and helpful. It's been really cool for states to see what their colleagues in other states are doing and for states to be able to ask questions and have a discussion around how states are making it happen, as Kelley said. What ... software, who does the work? What is their role? Et cetera.

00:14:27.61  >> Mm-hmm.

00:14:28.33  >> So that has been ... It's not a specific tip or recommendation, but it's been just great knowledge-sharing in general. And then of course tips on APR language and submission are always welcomed and found useful. At our last meeting, we had a discussion, as I mentioned, about making the data more digestible, and one of the states shared information on a product or app that's called the Hemingway Editor, and it helps you tell a story and how you can look at data in real time, edits in real time and tells you what's hard to understand and I believe even what grade level things are written ... that it's a very specific tip ...

00:15:27.38  >> Yep.

00:15:28.04  >> ... [Indistinct] and the result of it, of being in the group. And then I wouldn't say another huge bonus in the, quote, tip category is, as we've already said, learning about resources and services that are available through IDC. There's so many offered that I was not fully aware of when I was a data manager, and I do really regret that I didn't participate in the Data Manager Data Quality Peer Group more often when I was a data manager in a state.

00:16:08.82  >> I totally agree with that, and I feel like the value in being able to align those resources with the specific topic that we're talking about is very helpful to data managers. We also really try to listen for any changes that are coming up with 618 data, 616 data submissions, if there's going to be any changes in the EDFacts file specification. For example, we heard of a change in language in FS009, and so our first meeting after that change, we kind of highlighted it, and then of course the data managers had some questions, so we went back to OSEP and to some of our TA providers at IDC and asked some questions, and so now at our next meeting we'll be able to provide some more clarification. So we really try to listen to what's happening in the moment and real time so that we are addressing questions and concerns that they're having right now.

00:17:10.48  >> And sometimes things come up not expecting. Again, I'm going to reference our last call that we just had last week, and we had a participant from one of our entities who stayed on after everyone else had left to ask a question about EDFacts reporting that turned out it was not IDEA-related.

00:17:35.80  >> Mm-hmm.

00:17:36.44  >> But I was able to point him in the right direction and give him enough information to get going, so it really is ... It's just a great place to come to for support.

00:17:50.55  >> Thank you, Kelley and Kristen, so much for being on. You shared such great information about this wonderful group, really appreciate your time.

00:17:59.11  >> Thank you. We appreciate the opportunity to share.

00:18:02.86  >> Yes, thank you so much. It's been great.

00:18:07.93  >> To access podcast resources, submit questions related to today's episode or if you have ideas for future topics, we'd love to hear from you. The links are in the episode content or connect with us via the podcast page on the IDC website at ideadata.org.