Influencing the Quality of IDEA Data, from Data Collection to Communication and Use

A Date with Data

Reach out to us if you want to access Podcast resources, submit questions related to episodes, or share ideas for future topics. We’d love to hear from you!
You can contact us via the Podcast page on the IDC website at https://ideadata.org/.

### Episode Transcript ###
00:00:01.52 >> You're listening to "A Date with Data" with your host, Amy Bitterman.

00:00:07.34 >> Hey. It's Amy, and I'm so excited to be hosting "A Date with Data." I'll be chatting with state and district special education staff who, just like you, are dealing with IDEA data every day.

00:00:19.50 >> "A Date with Data" is brought to you by the IDEA Data Center.

00:00:24.65 >> Welcome to "A Date with Data." I am joined by Brandon Myers, who is the Part B Data Manager with the Indiana Department of Education, and we are going to be talking about data-quality influencing. We at IDC have been talking to states about this idea, about how all of us have the power to impact and improve our data quality. So, Brandon, welcome. Can you first tell us ...

00:00:47.87 >> Hello! Lovely to be here.

00:00:48.84 >> Thank you. Can you first tell us just a little bit about yourself and your role?

00:00:54.23 >> Sure. I'm with ... As you said, I'm the Part B Data Manager in Indiana. I started in April of 2016, so basically a manager is a pretty good title for it because the data comes in from the schools, and we have to do various things with it to get it recorded to our federal partners and then also to make it usable and for the schools and reportable to the public. So kind of every aspect of it coming in, being analyzed, organized in one way and then getting it reported back out, I'm involved with to one degree or another. And then our monitoring specialists work on the specific areas from a more substitive point of view, from a more program point of view.

00:01:46.13 >> Great. So you're really touching the data at all points in the process. Can you just start out telling us a little bit from your perspective, what does it mean to be a data-quality influencer?

00:02:00.15 >> Well, we've had a lot of issues with making sure that our data is of high quality. If you're going to use this data to make strategic decisions at the local level or monitoring decisions at the state level, including accountability decisions, you want to make sure that everything that is coming in is reflecting the reality to the extent that it can be of what's going on out there in our schools. So from a data-quality perspective, we make sure that the schools are aware of the specifications for all the data and then also, maybe even more important, why it's ... what it's going to be used for and, thus, why it's important. Otherwise, it's just, "Well, we have to fill out this particular template and put this information," and you get obsessed with the coding, and that's very dull for most people, even me sometimes. But it does matter on the other end to make sure that everything is lined up so that our decisions can be made with the best information possible.

00:03:06.93 >> Absolutely. So being a data-quality influencer, part of that is being able to message out why this data is important, what it's being used for because that sets the stage for making sure that they are collecting and reporting high-quality data if you know what the purpose really is in the end.

00:03:26.62 >> Yes, and involving the schools in that process, particularly our school stakeholders and parents and community members to a certain extent. Now, there's certain things, obviously, that we have to do that are in regulation, but additional things, we want the input from people, and also how we go about collecting this information, we want input from our school partners because that's going to make the process go the best and make sure that everybody is on board with providing the best information.

00:04:03.33 >> Yeah. And how would you say that you influence the quality of your state's IDEA data?

00:04:10.76 >> Oh! Well, I don't know. Is that a valuative question or a technical question? I try to be visible with our data collection, so I partner with our IT office because, obviously, that's where the vast majority of our data is coming in, through various IT systems and the IT systems we have here at the Department of Education, so very fortunate in our data-collections people here because from the moment I started, I was always heavily involved in both the organization of, say, specifications and files and things like that but also in the training. So we have joint trainings. Just spent an hour yesterday on our twice-a-week Data Exchange. That's our data-reporting system. Data Exchange office hours.

00:05:03.55 >> Hmm!

00:05:03.68 >> It's run by the head of our data reporting, so sometimes, I'll be like a presented speaker during that, but most of the time, it's just answering questions, and I'm kind of on-hand to deal with issues surrounding special education reporting. So I would say trying to be visible and available is actually the biggest part of it. After that, it is really the material and making it as clear as possible, trying to make it as simple as possible, which is not always very easy because sometimes, these specifications and the rules for reporting and the rules for calculating and so forth are very complicated and technical, but I try to present that information in as clearest and as simple way as possible, lots of pictures in my presentations and charts the things like that. But then, of course, we do have the very dry data layouts and things like that, where if you really want to look into, "What's the difference between a one and a three in this particular situation?" you can go that far as well.

00:06:08.81 >> Yeah. So really meeting them where they're at, whether it's: They want the more complex and to dig in deeper or sort of more novices and need the simpler introductory-type language.

00:06:23.63 >> Right. And sometimes, I need the introductory language as well.

00:06:26.70 >> Yes. I think we all do, for sure. You mentioned doing PowerPoints and materials that are friendly for families and other stakeholders. Do you have examples in what areas or what types of materials you've produced in what content areas?

00:06:47.35 >> Sure. So like I said earlier, there's really three stages, I think, to the Data Manager position. One is getting the data in. The second is organizing and performing, kind of monitoring tasks, and then the third part is how you communicate that out, and it depends on who the audience is and how you're going to get that out and how much information you're going to provide. So the things I was talking about earlier mainly had to do with the collections. So there, we're talking about mostly the special education directors at the local level, data-reporting staff at the local level, which they can have a variety of jobs. We also have a lot of special education cooperatives, which are groupings of local education agencies who've grouped together to provide special education services, so we have to keep those in mind as we're presenting those materials. But what's really more interesting to me is the information you're putting out after the data comes in.

00:07:49.65 >> Mm-hmm.

00:07:49.94 >> And we do that not as well as I'd like, to be honest, as one of our kind of priorities because it does take so much time to just get it incorrectly.

00:07:59.52 >> Right.

00:08:00.07 >> But through our monitoring system, our Results Driven Accountability system has a data reporting that I think is very simple in presentation, and we've gotten a lot of good feedback on how easy it is to understand and to figure out and to kind of use to get conversations started at the local level for strategic planning. It's not going to tell you exactly what to do, but it's going to start to ... It's going to give you the places that you need to start looking or provide you with information with where you can best expend your limited resources at a human and other level. So that's one big example.

00:08:47.07 >> Mm-hmm.

00:08:47.36 >> Our public reporting, we're working on doing a better job of getting more information that is a little easier for parents to understand, to see how the school that their child may be attending is performing. Right now, we have our INview, and we're developing a new DOE, Indiana Department of Education Initiative called GPS, which is Graduates Prepared to Succeed. That is going to provide ... It's trying to simplify the information and provide the most useful information. There's a special education component to that, but then we'll be adding on some more specific special education issues, such as least restrictive environment, things like that, that don't apply to those students that are only in general education.

00:09:37.28 >> So what are some of the challenges? You mentioned one of them being making sure that the data are accessible and understandable to different audiences. Are there other challenges to being a data-quality influencer? And what are some ways that you've tried to address those?

00:09:52.36 >> Yeah. I think that fear is a lot of the challenge because of fear of a lack of understanding or a fear that ... Data is very powerful ...

00:10:04.25 >> Yeah.

00:10:05.04 >> ... in the sense of ... Since it is so useful at making information, a lot of big decisions can be made off of it.

00:10:13.49 >> Mm-hmm.

00:10:14.30 >> But at the same time, it's important to keep in context what it is telling you and what it is not telling you. So it's not describing what's going on in your school from a day-to-day basis. It's showing the results of that or the organization of that. So if you can communicate that to stakeholders ... And to be honest, it's important that local administrators know that as well because they're looking at benchmarks, and that's a very effective way to do it, but it's the detail that they need to take that next step, so I think that is an important aspect of communicating things.

00:10:57.62 >> Mm-hmm.

00:10:59.87 >> Another thing that we noticed particularly with our parent stakeholders, when we have meetings, is that they are much more interested in what's going on in the classroom than the overall performance of the school system, in general. I'm generalizing here, but I think I've been in enough meetings that I can say that that's the case. So you really have to try to connect those things to talk about how we are using this data to start discussions and then evaluate interventions. So school having a new program or new organization, a new way of supporting their students within the classroom, a new way of integrating the special education services that are going on outside the general-ed classroom and in. Those are some examples of things that we're really trying to focus, and to that is making ... I think people have really come to accept the idea that data is powerful, but they don't always see that it's relevant to what goes on day-to-day, which is really where the magic happens.

00:12:03.20 >> Right.

00:12:03.91 >> Magic doesn't happen at the Department of Education, so as much as I'd like it to or ...

00:12:09.66 >> Right.

00:12:09.87 >> I don't know, maybe I don't want the responsibility, but it can provide the information to improve the process by which those outcomes are happened.

00:12:20.08 >> Yeah. So really understanding, especially with parents, why is this data meaningful to them? What is important about this process for them personally? And trying to hit that home and really make that connection with them, especially.

00:12:35.66 >> Yeah, yeah.

00:12:36.48 >> Yeah. We just wrapped up our In-Person Interactive Institute a couple weeks ago, and it was wonderful to see you there, and this was a big theme at the Institute, the data-quality influencer and the power we all have over our data. Is there anything that you learned at the Interactive Institute that's really going to help you, you think, in your data-quality-influencer role?

00:13:04.42 >> Yeah. I think, actually, I probably have highlighted the meaningfulness in my discussion here with you today more as a result of going there. It's not something that I didn't really know before, but this really helped to ... We did an activity to really ... And it was difficult, to be honest. It was difficult to come up with these connections, but its difficulty to me showed why it's even more important that we focus on relevance as a huge issue and knowing your audience as a huge issue. The other thing that's great about the interactive institutes is that they do deal with some very specific issues that most people off the street wouldn't be the slightest bit interested in, and most people working in schools wouldn't be the slightest bit interested in, but if we aren't looking at those issues with our colleagues, relying on the support of all of the 60 grantees that are invited to the Institute, it's going to make our ability to do all that other stuff a lot harder, and we're going to be spending a lot more time on the collection part and the specifications part, and I actually like the collections part and the specifications part in some sort of sadomasochistic way, maybe. I don't know. But I also know that that is just the very first stage. So we had conversations around that and always connecting that to the relevance question.

00:14:48.55 >> Yeah.

00:14:49.44 >> So really moving ... Getting that first part down pat, so we know that we're collecting the right data, and it's complete and accurate and timely and all those pieces and really moving beyond that. So now that we have this good data, what can we do with it?

00:15:04.73 >> Yes. Yes. That's always what is, to me, on the other side.

00:15:11.01 >> That's always what you have to keep in mind is, we're coding this LRE data, least restrictive environment, in a particular way. And I think, by and large, that is also ... We see that at the Department of Ed in Washington. They try to organize things and connect it very much, as much as possible, to what's going on at the local level. Now it's not perfect. It's never going to be perfect. You can't have ... And you're always losing information by making some sort of categorical choice. You are influencing the outcome of that data, but if you try to think about at the ground level what that looks like and how that will connect to the more formal data organization, that's, I think, always what we should keep in our head.

00:15:59.19 >> Yup. And what do you have coming up next as a data-quality influencer?

00:16:05.44 >> So, yeah, as I mentioned earlier, we are developing a new system in the state that will present ... Right, currently, it will be for schools, but then we hopefully will have a public-component part of it later. To share our monitoring information with schools, we've been doing it in a very sort of old-fashioned, by 2022 standards, way, so we're really improving our IT infrastructure so there will be a place to communicate, a place to look at the information through an online connection, rather than static documents.

00:16:41.73 >> Mm-hmm.

00:16:42.31 >> It will improve our ability to do visualizations a little bit more because right now, with a manual process, you can do visualizations, but they take so much time, and we are limited in our capacity here. We only have a few specialists, and I'm the only Data Manager, so ... Data Person. So whatever we can rely on, IT infrastructure to improve that process, that's a new initiative. And the other thing, of course, is, we're looking at what we've seen from the coronavirus pandemic, as is everybody. It really hit our kids with disabilities hard in terms of their access to education for a while, and it hit everybody else hard as well, and we're trying to see how we're recovering from that, how we're going to deal with some of the deficits, some of the learning deficits that were created just by a lack of instruction or instruction in a different form that we really haven't quite perfected yet because virtual learning is still very much a work in progress for most people.

00:17:50.54 >> Yeah.

00:17:51.64 >> And so that's another focus that we are folding in to our monitoring process. We try to keep things as clear as possible and not have four or five or six different separate initiatives going on. We have, within our monitoring system, different points of emphases.

00:18:13.50 >> Great. Well, is there anything else that you want to share about the without going on in Indiana and being a data-quality influencer?

00:18:24.67 >> The only thing that I would share, and I'll just kind of reiterate, is, for me, anyway, is the importance of connecting this information to classroom instruction and services support.

00:18:39.22 >> Mm-hmm.

00:18:39.46 >> So and this is a very interesting thing, I've been talking about this for a couple of years, and the more that I look into it, the more I'm convinced that it's the right way for us to. We've seen a great deal of improvement in children with disabilities in the general-education setting in the last 10 years. So that was an extremely ... And that predates me when this started, but that was an extremely data-driven process where we gave the schools very specific information on how they were doing on Indicators 5A, 5B and 5C. If you are listening to this, you probably know what that is. If you don't know what that is, I'm not sure whether I'd look it up. But anyway, it's about information on where the children are receiving their services, and that data-driven process has been very effective in getting more and more of our students with disabilities in the general-education classroom.

00:19:43.07 >> Mm.

00:19:43.60 >> And so it's a success story, but of course, it's an incomplete success story because it focused on that aspect of the data, and that was fine, and that made sense, but really now have to look at, what's happening in those classrooms? Which is a much bigger data challenge.

00:20:00.59 >> Right.

00:20:01.43 >> And so that, and something that I think ultimately, we can only assist at the state level. It's hard to collect really good systematic data on that, even at the local level, but what we're seeing is, children in those general-education settings,m but their outcomes are only modestly improving, and we think it's mainly due to the access to the general-ed curriculum. So the alignment between services and the academic instruction there becomes the biggest issue, and that's really the next step in our kind of monitoring vision, for lack of a better word, is to look at specially designed instruction and how it goes into practice, and some of that might have to be done by more qualitative observation at the local level, and then obviously, at the state level, we can be part of that to a certain extent. And then also some collection of systematic data, so we've provided some suggestions on that, but that is the thing that I am the most interested in from a program point of view at this point.

00:21:08.81 >> Wow. That's really interesting, and congratulations on the success you had around the least restrictive environment and Indicator 5, and hopefully, what you've learned from that work, you'll be able to apply to what you're doing now moving forward and have as much success there.

00:21:28.02 >> Thanks. It's very nice to see that graph go up, as you had mentioned.

00:21:31.62 >> Yes. I'm sure.

00:21:32.39 >> You don't always see that, so when you do see it, yeah, you really should celebrate it.

00:21:35.96 >> Definitely, definitely. Well, thank you so much, Brandon. This was wonderful. It was so good to chat with you, and would love to hear from you again when you get some of those initiatives going, and would love to hear how that's going on.

00:21:51.64 >> Thanks very much. I enjoyed it.

00:21:54.16 >> To access podcast resources, submit questions related to today's episode, or if you have ideas for future topics, we'd love to hear from you. The links are in the episode content, or connect with us via the Podcast page on the IDC website at ideadata.org.