Step 7: Developing Effective Data Usage - with James Pembroke and Richard Selfridge의 전체 녹취록

that goes on in schools that doesn't really have much impact at all. Paul Ainsworth: Richard, what would you like to add to Can Jamie's thoughts? Richard Selfridge: Again, our view generally is that you need to make sure the information which you've got is high quality. ⁓ You want the highest quality data you can get and the least amount of it. Basically, you don't want to collect any more than you need. You need to work out what it is that you do actually need, make sure that you do all kinds of things to ensure that that data is actually valuable and is telling you something meaningful about things that are happening for the children in your school, across your school. And then over time, you want to reduce the amount of effort it takes to gather that information and to collate it. We talk about the thinking about data in terms of the data which you gather, the data which you collate, the data which you analyze, and then the actions that you take as a result. In English schools, particularly, you're gathering information all the time. You you can't teach in an English school without... Effectively assessing, gathering, thinking about the children you're teaching, thinking about the classes you're teaching, the cohorts, the schools that you lead. You're gathering information all the time. Only some of that needs to be collated. And therefore you need to be kind of ruthless, as Jamie says, in thinking about, what do we actually need to collate? What are the key metrics that I need so that I can understand the school, the class, the individual, child? What are those key things? And then making sure that you actually analyze that data. well and then that there are actions. Our view as well is that most children in most of our schools do very well. If you look at expected outcomes in primary, the majority of children do well in our primary schools, not all, but the majority. And once you recognize that, you can begin to see that then for those children, you need a light touch system that allows you to ⁓ monitor those pupils and to monitor their experience in school, their learning in school. But not to obsess too much about it. How much do you really need to obsess about a seven year old who appears to be in school in class focused on learning? You know, if that if all those things are working then brilliant the things that all those other things which you've done as a school leader are working but for the children who aren't in school all the time who aren't in class all the time who aren't focused on their learning and aren't actually learning the things which we want them to learn then you need more information about those children. So it's kind of focusing your effort being ruthless understanding what it you're actually going to collate and what you're actually going to then analyse and then ultimately you do you need to have a plan so that every every individual in the school particularly teachers in classes they have a really clear idea at this point in time this is my focus. James Pembroke: Yeah. Paul Ainsworth: that leads us beautifully onto the next question. So if you are kind of being ruthless, you you kind of take over two primary schools and there's maybe two different methods of data collection going on in those two schools. Where should a head teacher start? So Richard, what different types of data would you be advising that ⁓ executive head teacher to collate and then begin to analyse? What's your starting point? Richard Selfridge: It's a good question and this is a common situation that a lot of people find themselves in. And I think again, so when we work with schools, we say, always start by just auditing where you are. Just have a good look and have an honest conversation to say, what is the picture within the schools that I lead? So therefore, you need to have a good look at your data and you need to ask those questions. Is this the highest quality data? Have we got the least amount? What have we got in here? that we don't really need. And also, what is this data possibly telling me? Now, broadly, would talk about, Jamie and I would talk about teacher-assessed data and how useful that is. And it is incredibly useful to have teacher assessment. The problem with teacher assessment is you're probably wrong and you don't know where you're wrong. So teacher assessment is, whilst you can't teach without it in England, it's very, very, it's extremely useful, particularly over time if you build up big pictures. But you're not sure where you're wrong. So you can have moderation. ⁓ processes and so on but recognise that teacher judgment is biased, we're all biased and there's plenty evidence of the research into this. Christodoula is particularly good on this, good name to look at if you want to know about bias in teacher assessment. ⁓ Once you recognize that you need some teacher assessment, but you also need some kind of independent assessment. And again, we would recommend strongly that you have some kind of standardized assessment. Standardized assessment is a particular thing and there's lots of ⁓ areas ⁓ or issues to consider, but essentially something where particularly in primary schools where your main focus is reading, writing and adding up. Can these children work? know, can they can they decode language, understand language? Can they can they express themselves in writing? and can they work with numbers at an age-appropriate level. So you need information about that and then you need to have some kind of way of just having some standardized information and again standardized assessment there's also measurement error in that but over time if you pick up if you build up a picture of children's experience over time you could do that. So as a teacher as a head teacher I would look at say what information do we have let's have look at those two things say teacher assessment, standardized assessment I would also be then saying are any of these schools doing anything

innovative? Do they collect any James Pembroke: you Richard Selfridge: other useful information and they may collect other useful information could be effort grades, be attitude to learning, they could be really very good at analysing attendance, they could be very good at analysing additional provision that's put in place. But again it's taking that step back having a good look, having an understanding of what you're looking for and what you'd like to see and what you can begin to picture. Anything you'd add to that Jamie? James Pembroke: I think that schools need to beware of the illusion of accuracy. This is something I talk about quite a lot, that we fell into this trap years ago with dear old levels, that we would assign a child a level. We made levels do a lot of work, you know, we kind of put a lot of emphasis on them, a lot of pressure on them to be all things to all people. ⁓ But they gave an illusion of accuracy, you know, basically, unless it was based on a test, so it became more like a sort GCSE grade, levels are often assigned by a person. A human would come to this decision that this child was a level A, ⁓ a level 3 or a level 4, whatever. And this idea that even that you could compare one person's level with the same person's level that they give at another point, so this is the same person, I said that this child is level 3 and then I said this child is level 4, even comparing those two levels assigned by the same person is problematic. let alone comparing the levels assigned by two different people. So ⁓ in year three, child gets a level three and then in year four, the child gets a different level by a different teacher and you think that you can then compare those two, let alone across subjects, let alone across schools. You know, this idea that you can take some schools and you can take human opinion in different schools and that you could compare them. And of course we do do this. We try and iron out the wrinkles. by using moderation. With writing, at Key Stage 2 is a really good example and we know there's been some real problems with writing and there continues to be problems. So if you're a senior leader, if you're an executive leader across a number of schools, the idea that you can compare standards in that school, those schools, which may have, ⁓ you you've got different teachers, you've got different relationships, you've got different approaches to assessment. You might have different accountability regimes. Those teachers in one school might be under more pressure than the other because one school is old money outstanding in the new school, whatever the language is these days, we don't need to go into that. Another school is kind of needs improvement. You know, so the pressure will distort the assessment and pressure exerts a lot of a lot of sort of external accountability exerts a lot of pressure on assessment and can cause There's biases inherent anyway, but it goes beyond that and becomes distorted. And that's why we do need some form of external reference. We do need some form of independent reference there, which is why standardized tests are a good idea. ⁓ But if you are going to use some form of standardized test, then you're going to need to use the same test, right? There are levels to standardization. There are some things that are outside of your control. So there is the way that the test is designed and the way that the actual scores are standardized. That's all done for you, you don't really need to worry about that. But you do need to make sure that you're using the same tests across the schools. Otherwise, if you're using, I don't know, NFR test in one school and you're using a STAAR test or an NTS test or something in another school, you can't really compare those, you just can't. So you need to have the same suite of tests. But the other thing is they need to be taken at the same time and they need to be taken under the same conditions. And those things are some, those are factors that you have control over. If one school is being really strict about the administration of these tests ⁓ and they all have to be taken on the same day at the same time, whereas in another school they're going to go, well, these ones are done after lunch and these ones are done before lunch and ⁓ like, well, you can sit in groups and then it's not going to be, the data is not going to be comparable. So you have to have quite strict regimes. And this is difficult because you're then imposing some, yeah, some quite sort of, some quite rigid boundaries on the way that assessment ⁓ takes place. But the payback is you get good data, but you do get pushback from staff maybe who don't like the way that this is being done, but you want good data. you're kind of, it's a difficult thing to balance. Without that data you can't monitor the standards across the schools. So... Yeah. Paul Ainsworth: I think that's really helpful and something that I learned a lot from you, when I first met you two in a data busting session, was that use of standardised tests and the way that you talked about the importance of those and that's really stuck through me in my work in system leadership. Now, I think some of the ground you've just talked about there, James, I'd just like us to unpick a little bit more. So one of the things that I think is really important and I'm sure you'd agree is that executive head teachers have some kind of assessment strategy that runs across their schools. And I sometimes say to people, have you got an assessment strategy? And people look at me a little bit blankly, as what do you mean, Paul? What should be in there? What do I need? So you talked a little bit about kind of consistency of tests and those kind of things that I think would probably be an assessment strategy. But if you were starting from the beginning, Jamie, what would you advise that executive headteacher to put in their assessment strategy? James Pembroke: ⁓ I think that they need to decide

what the end point is. What do they want to get out of this? What is it they're trying to do with the assessments that they run? Because broadly speaking, are two parts. If you just take a standardized test, there are probably two reasons for doing this. There may be more reasons, but in terms of reporting to parents and things like that. But generally speaking, you're using it two purposes. One is... you might be using them to find out what children can and can't do, what they do and don't know. So there's like an informative element to it. I'm going to pass this over to Richard in a minute. He's going to talk about this because I can see he's itching to talk about this. The other element is there's an accountability side to it. What you want is you want to know, you don't want any surprises, you know, at the end of a key stage two, for example, you don't want to go, whoa, Why are results like this? I didn't know this was going to happen. So you want to know what's going to happen in advance. you want these tests to be ⁓ predictors of outcome, to give you a really good sense of how many children are on track to meet expected standards, essentially. also now, if you think about Ofsted, and we think about the IDSR, and we think about how gaps are back, And I don't mean gaps in knowledge. I mean gaps between disadvantaged and non-disadvantaged children primarily, but the gaps between different groups. We want to be able to analyze those gaps over time because we want to see if those gaps are closing. So that's sort of part of the strategy. Part of the strategy is to have assessment that effectively tells us what children do and don't know so that we could plug those gaps. Part of the strategy is to try and predict whether children are on track to meet those key standards. ⁓ at the end of key stages and to monitor the gaps between groups to see if those gaps are closing over time. There are other elements to our strategy. That's to give parents useful information as well. We want to inform parents about how well their child is doing. We might want to have some ⁓ assessment of progress. I mean, I could talk about progress to the cows come home because progress is a massively complex thing. And I think it's been sort of oversimplified and levels have a lot to answer for that. So yeah, think that's It's starting like, what is it that we want to, where do we want to get to? What do we want our assessment to show? What is the purpose of doing all these things? And then almost like to work back from there. But I'm going to hand over to Richard because Richard actually does sessions with schools on developing data strategies. So he's probably better equipped to talk about this than me. Richard Selfridge: ⁓ add a few in there about, again, thinking about this as a a executive and you're thinking about an assessment strategy. ⁓ One of the key things, one of the phrases that we had in the last inspection ⁓ framework ⁓ was ⁓ that leaders should understand limitations assessment and those working with assessment should understand the of assessment. It's still in there in the the current toolkits, but it's not as clear as was this came from some work which Becky Allen did ⁓ again on the looking at the workload reports from a while ago, you the last 1012, eight, 10 years ago, and understanding the limitations of assessment, which is that ⁓ that the assessments, even standardized assessments, they they, they are, I like to use the word fuzzy. They're not accurate assessment in the way that people want them to be often. And this is a bit of a hangover from the data wave that washed through schools in the 2010s when people, know, people, you would genuinely believe that they could assess a child as being a 3C ⁓ or a 2A and could allocate a point score to that child and that child children made one sub level or sub sub level or bit of a thing from one point in time to another point in time. And there was this sort of illusion of accuracy as Jamie's talked about, So the limitations of assessment are key for a leader to understand. Standard assessments are incredibly useful, but they don't often do the things that people might want them to do or assume that they do. So this is a key thing as a leader that you need to understand. And I think it's also worth recognizing that we talk about primary education here as well. And primary education is a different thing to secondary education. The key stages within primary education are different. Key stage one is different. Key stage two is different to EYFS. the main things which I would say is that for a lot of a lot of the comments you hear about this comes from secondary school teachers who recognize that testing is a useful educational tool and it is at secondary school level certainly in key stage 3 and key stage 4 testing the testing effect is real and you know children learn through being tested but they also have a lot of agency in secondary school and have a lot of incentive to get better if you're 13 14 15 16 17 and you're being set tests you probably want to do on those tests, not all children and the children who don't are a different group and there's reasons why. But as I say, most children do well in our system, that's because they want to do well in assessments. Therefore they take ownership, so the testing effect really works for them. There isn't a lot of evidence that that's really the case for six or seven year olds or eight year olds. And also when you look at a assessment for a six, seven year old, we're asking a relatively small number of items. You're probably only asking 30, 40 items. So there's only a small amount of information you're getting from the children. And people in primaries of schools, when I talk to leaders and to those using assessments, you can see that people have assumed a lot more of the assessments than is reasonable. You know, the thing which Jamie knows is one of my book bears is question level analysis done for

younger children. You've had 40 items and people will spend a lot of time working out whether they can really clean a lot of information from that. I think the honest answer is probably not. And that's that's worth you considering as a school leader. It's definitely worth doing occasionally if it's done quickly, easily, if you're doing online adaptive tests, which give you basically question and error analysis quickly. But to spend a lot of time on that is not sensible thing to do. understand the limitations of assessment. Assessments are really good at ranking children, broadly putting them into kind of groups. There's some measurement error in there. But if a child is recording 120 on a standardized assessment, that is definitely different from a child who's scoring 80 on a standardized assessment. Paul Ainsworth: I'm ⁓ Richard Selfridge: And there probably is a difference between those two pupils and the pupil scoring 100. But I would be wary about that because when you, if you look at measurement error within standardized assessments, particularly in primary, it can be quite large. So, so therefore just be aware of that. But as I say, if you build a picture over time, if you build a big picture, if you use standardized assessments, you can begin to use them in year one. as a written standardised assessment. But by year two, you can definitely use them by year three, certainly. By the middle of year three, you should probably have four five data points in reading, in maths. And so you should have a fairly good idea now which of the children are finding school works for them and which of children who are finding school doesn't work for them and they're finding it more difficult. So as an executive leader, I would say go back to really making sure you understand what are the data, think about what we've got there. Have we got any standardized assessments? And just try not to be either misdirected or to overinterpret the results of those assessments. Jamie, think some of you wanted to add in as well. Paul Ainsworth: games. James Pembroke: Yeah, there's another element to having a strategy and that then everyone knows where you're going, you know, and I think that purpose is like a really important thing. That I think there are a lot of schools where and therefore a lot of MATs probably where teachers don't really know why they're collecting data and there's a huge risk to this. ⁓ Historically, think data has been collected in schools for the purpose of accountability rather than the purpose of learning. It is to, as I often say, it's to keep someone else happy. That's why we collect data. If you have a data strategy, and Richard talks about data strategies a lot, if you have a data strategy that outlines exactly why you're doing these different types of assessment at different points in time and what the purpose of that data is and why you're going to use it, how you're going to use it, who it is being reported to. parents, governors, senior leaders, executive leaders. If people see that and they see there's purpose in it, then you get buy-in, know? People buy into the system. And then the payback is that you get timely and accurate data because people see that there's a reason for doing it. If people don't see there's a reason for doing it, then they won't take it very seriously. Or maybe, even worse, they could take it too seriously and they think, is this to do with me? Is this data, is this about measuring my performance as a teacher? Therefore, and this is the age-old problem with data, let's make it really low at the start of the year and then make it really high at the end of the year. And then you get these weird kind of sawtooth effects, go up and down, up and down, up and down. And every, no teacher trusts the last teacher's data. And every key stage blames the previous key stage. You know, it's so you need buy-in and a strategy gives you that. It gives people purpose. The other thing I think is, which maybe this fits into the strategy as well, is about workload, about reducing workload. I don't think teachers, teachers don't mind doing things that take a lot of time if they see that there's purpose to it, but they don't want to do things that they just think, I'm just ticking boxes here. I'm just jumping through hoops, going through the motions. I don't really know why I'm doing this. then don't expect to get good quality data. And if they think the data is being collected to measure their performance rather than to track how children are getting on, then you will get distortion. Distortion will happen. You'll get those sawtooth gradients where they're trying to prove progress over time. Paul Ainsworth: So I think we've started to cover some of the barriers that executive heads might find in trying to work on those data strategies. You've talked really well there, Jamie, about staff not understanding the purpose of it. You've also kind of talked about workload as well in there. Are there any other barriers that you've seen when colleagues have tried to implement data strategies across, well, just in one school or maybe in a group of schools? Richard Selfridge: One of the key things which we struggle with all the time is the fact that our schools are comprehensive schools. We have all of the children in a primary school. And therefore, as I say, most children are probably keeping up with expectations for their age group and so on. But a number of children aren't. And all teachers will have to then to deal with that and say, well, what do we do about those pupils who aren't working at age-related expectations or below that or well below that? And that's definitely one of those areas that we definitely we talk a lot Jamie and I talk a lot about and that you have to have within your assessment strategy some way of managing ⁓ how you're going to approach the assessment for children who working below the age group expectation again because there are all kinds of interesting

practices that schools end up doing which aren't particularly helpful Jamie can probably expand on a number of those things but that's one of the things which I would say is a key area you have to take into account the fact that we have the full spectrum and that we have to assess all of the pupils. James Pembroke: Yeah. Children working below the age-related... Age-related expectations is an interesting phrase because actually when we say age-related expectations, we should be clear, we're really talking about curriculum expectations. It's only... ⁓ And I'm being a pedant here, but it's only really early years. When early years talk about age-related expectations, they mean it. They're actually referring to a child's age. and the typical level of development for a child's age and they're talking about child age in years and know months and anyway. So we've got children who are working below their peers, working out of year group and that causes all sorts of problems. It can cause some bad practice actually. So one of the classics is using ⁓ tests designed for younger children. That's a, it's an obvious thing. You think, well, this child is in year five, they are And they'll often say, are they working in the year two curriculum? No, they're not. They're not working in the year two curriculum. They might be tackling elements of the year two curriculum. If you want to go by, on average, yeah, it's sort of around that kind of level. I suppose they're maybe operating at the level in terms of their reading, maybe is the level of a six year old or a seven year old or something like that. But anyway, so schools will go, well we do these standardized tests, we've just been talking about standardized tests, we do these standardized tests and all the year five children take the year five test, but what about this child and this child and this child who are working below curriculum expectations and they're working like a year two, three years below. Let's give them a year, let's give them a year two paper, let's give them a year three paper. That's fine in terms of they have a test that they can attempt, that is maybe it's appropriately pitched, that it's interesting to see how they get on with that test. And maybe at the question level it can show you something interesting about how they tackle those particular questions. But the data is of no use. know, this 10-year-old child did a paper designed for a 7-year-old and they got 104 which makes them above average. And then you go, ⁓ they did quite well on that. Let's give them a year 4 paper to try then. You give them a year 4 paper to try and they get 86. Well, they got 104 in the last one, they got 86, but that was a year two paper and they've just done a year four paper. That doesn't mean anything. You can't track in that way. So you have to be, I understand why schools do it. The problem is at the executive level, the leadership level, when you're trying to put all this data together, that doesn't really tell you something useful about those children and you can't mix it in with all the other data. So this is where things get messy. At the executive level, guess, I'm I'm a governor, Richard and I are governors. If we think about governors, and I know that's kind of ⁓ a separate group, but we broadly speaking, we need to know in year groups, in reading, writing and maths, what proportion of children are working at or above age-related expectations, and therefore what proportion of children are not. And we should be, rather than asking for data on those children, we should be asking about support for those children and the effectiveness of the support for those children, the provision that is put in place and what's tried and worked and what's tried and hasn't been tried and hasn't worked. So that's quite messy because sometimes data just doesn't... We've got this kind of headline data so I can see in this year group we've got far more children working below in maths than any other year group so what's going on there but when you really want to know something about that year group but it's got a lot of children working below you've got to get into the weeds. No data is going to really explain... So there's a group of children that... As Richard started by saying, know, there's some children it's quite light touch. You just if they're fine, we use this word all the time. They're kind of working around expectations and they're fine. You don't really need a great deal more data. It's these other children that you probably need more data on, but it becomes quite individualized. You know, some children, they they're working at a very low level of attainment and they have some quite complex needs. And in terms of curriculum coverage, it might be tiny, but the progress that they've made in making that tiny bit of coverage, which in relation to other children is extraordinary. They can make extraordinary progress, but it's very unique. Very unique. It's unique. It's individualized. It's a personalized sort of rate of progress. So it's, yeah, it's, it's really complex. And I think that that's the biggest challenge that, ⁓ that, that schools face now with data is, is trying to make sense of how these children are doing. Paul Ainsworth: One of the great things I love about talking to the two of you is you come up with a barrier and then you immediately give a solution and kind of explain to colleagues how they can work through that. I don't need to say how do you overcome these barriers because you've talked about them really well. that's kind of so helpful for kind of colleagues. of the challenges that I've consistently faced is working across a number of schools is when I take on a second school and there's a different IT system. in that school. know and it is that challenge isn't it? I think we would all agree wouldn't we that if we're going to be executive heads we need to have the same data system across both schools otherwise it's just really

painful very quickly. I've been very fortunate to have used insight in a number of trusts that I've found really helpful but what advice would you give to somebody? First of all in in trying to make a decision about which data system or IT system to use in their schools, what should they be looking for? And then when they've made that decision, have you got any advice in how they manage that change process in one of those schools? So Jamie, I know this is something I feel that you're really interested in, so could you share your thoughts? James Pembroke: So first of all, ⁓ for total clarity, I work for Insight and so does Richard and other systems are available. So yes, ⁓ you need a system that, or you need to unify systems really. ⁓ Otherwise it just becomes incredibly messy and incredibly frustrating and your precious time is spent trying to what? pull spreadsheets from different places and try to bind them all together, clearly that's ⁓ not what you want to do. You want something that enables you to have all of your data in one place. But this is another really important point about the systems that we use. I referred earlier to progress and progress being a problem, and I maintain that progress is a problem because the history of systems in schools up until quite recently was they were tools to measure progress. The whole purpose of having a system, a tracking system, as they were called, and it's still called, but I don't really like the word tracking system because I find the word tracking quite a sinister term. anyway, right, the purpose of these systems was to assign, in old money, in old days, assign a child a level and that was converted into a number. And then assign a child another level at some other point, probably by a different human being. And then that was also converted into a number and you subtract the first one from the second one. and that apparently equaled the amount of progress the child had made and then you would show that to your governors and you would show that to your LA advisor or whoever, your SIP and you would show that to the Ofsted inspector and they would go, excellent, that's great, I'm very happy and that was the whole purpose was to invent progress numbers to keep various external agencies happy. That is not the purpose of these systems. The purpose of having a system in a school is to have all of your data in one place or as much of it as possible. Going back to what Richard said though, not all data needs to be collected. Don't try and collect everything, but you need sort of headlines. So you need a system that enables you to capture teacher assessments, probably terminally in reading, writing and maths, if it's a primary school, a whole bunch of other subjects if it's secondary school. You probably want something that's quite nuanced for reading, writing and maths because you're trying to capture whether children are are on track to meet expected standards, get greater depth, hit high scores, or working towards or be well below or pre-key stage or whatever. So there's some nuance to that. I think with foundation subjects in a primary school, it's literally are they working at expectations or not? And you might only want to do that once a year. So that's very light touch. There are no frameworks to assess children by. It's like something that that's actually something else you need to kind of work out how you're going to assess foundation subjects, which maybe we can talk about. That's quite a difficult thing. Then you want to have your standardized tests and that also needs to go into your system. You might have a reading age assessment. ⁓ That needs to go into a system. You've got a phonics scheme and the phonics scheme that generates phonics scores and it generates phonics phases. You've got a maybe the reading scheme also has like book bands that children are on. They're on green books, they're on red books. Maybe you want to put that into your system. You've also got stuff around social, emotional, mental health maybe and well-being and how happy children are. You've obviously got attendance data. There's a massive amount of attendance data in schools. There's so much information. You really want to try and avoid having some of it over here, some of it in a spreadsheet, some of it in a tracking system, some of it in different files and folders. All the provisions that you're running for children with SEND that we were just talking about, you know, the effectiveness of that and how much time is devoted to different types of support, trying to bring all that into the system as well. So the person with oversight and the teachers as well. So all users of the system have access to all this information without having to go from one place to another. And then what you want to do is be able to pull this together and be able to say, right, okay, so show me in those core subjects across all my schools, all these schools that have oversight, how are people getting on? How are children getting on? And I should be able draw from different types of assessment at different points in the year using the same system. And yes, that requires you to invest in a MAT wide approach. Paul Ainsworth: So Richard, how would you, if you decided to invest in that map wide approach, what would be the advice be on, you know, kind of starting that implementation journey with your two schools or your three schools? Richard Selfridge: Yes, I think it's again. It's also recognizing that as with all areas of school development school improvement that it's an iterative process. It develops over time. So again, setting up starting off by auditing what you've got at the moment. And once you've done that, and once you've then just done a bit of sharing what you're doing, set up some kind of assessment policy or a roadmap for an assessment policy. You might want to develop that over time. And then also as part of all of that, have a roadmap to develop an integrated system of

some kind and looking to do that and to see you know from the different schools that you've got who uses what works well what areas work best. The one thing which I will say is that the last time I was teaching was two years ago and ⁓ teaching year five and I'm fond of telling people this I always ask people how many different systems as a teacher do you think I had to log into in order to do my job because it's a lot you know it was was 15 plus, think was maybe 18, 17, 18 different systems I had to log into. We use lots of different systems in school, is the point I'm trying to make. Hopefully there will be a system that does nearly all of the things you want to do. You might find that actually certain aspects are done elsewhere. And one of the things which you need to be a little bit wary about, particularly if you're looking at MIS type solutions, is that MIS are growing and are getting bigger and they have lots and lots of modules within them. Some of those modules are really good. some of them less so. So therefore you just need to have a really good look at whatever systems you've got and work out does this system do what we want it to do and what other add-ons do we maybe need to add to that. But I think those are the key things. Just work out what is being used and if you think there's a system that one of the schools is using that you think that could work for us then do that. But then also it's always worth at this point again within your plan to say well just ask all of the companies who provide solutions. to present to you or make a short list and just find out what's going on because the one thing which you are absolutely certain of is that the systems aren't standing still you know they're changing over time so Jamie and I work for insight and insight does does a lot of things now which it didn't do last year or five years ago or ten years ago and that's because you know we're developing the systems and all systems will be like that so yep so keep an eye on what's available but then once you once you have made a decision and this is the thing which I say with all things Once you've made the decision, try and stick with it. My experience as a school teacher and somebody leading in schools is that there's too much of a temptation to just keep changing things and always have a new thing that you're doing. Try to make a decision and then say, OK, we'll also put some in place and then try and leave it there as long as possible so that we don't change things, so that we build up an understanding and a picture of how to use the system, how to get the most of it, and so on. So that's the other thing to do. Paul Ainsworth: Jamie. James Pembroke: Yeah, I was going to say that one of the important things is that the system should adapt to you. The system should bend to you. You shouldn't have to bend to the system. Don't compromise your approach to fit the rigid sort of structures of a system, which I think is, it's maybe less common now, but

you would buy a system off the shelf and the system would do things in a particular way. would... it would have a rigid set of assessment bandings, for example, you go, well, our teacher assessment, we don't use that language in our school, we do this and go, so we, our teachers talk about children being X, Y, Z and this system has got like ABC and they kind of, don't really align, but okay, well, we'll just go with it. So when we put the data into the system, we kind of have to do these kinds of conversions in our head. It's a bit like when you're buying something on holiday and you think, how much is that in pounds? You shouldn't do that. You know, that the system should be customizable. It needs to be customizable, which is something that we're quite proud of, insight that we've made a very customizable system. Sometimes that can cause problems that, you know, so you're talking about schools that want to change things all the time. You can customize things too much. You know, you can you can over complicate things like don't over complicate things. Don't it's not if you're if you you have to sometimes you have to have these out with body kind of conversations with yourself or just hover above yourself and listen down on your own explanation of the system you have. Yeah, come on. Does that really sound sensible? Does that really sound logical? Because yeah, things can get over complicated. But so yes, a system needs to be able to be should be customizable, but but be just be cautious about the amount of customization that you do and be realistic. Don't keep changing everything. But also don't stand still. You know, there's this two answers. We haven't changed anything since 2003. Yeah, you probably need to. Our head teacher changes everything all of the time and I have no idea what he's doing. Paul Ainsworth: And that's one of the things that I really loved about using insight in the two MATs that I used it was, you know, the kind of simplicity of the tool I found really helpful. listeners will will be realizing that the two of you have got so much knowledge in this area, and that's why you've written a book and that's why you present those day to day sessions. So I'd really advise people to kind of look into those. But just as we come to the end of our conversation, James and Rich, I'm going to ask you almost the hardest thing. know what would be those top tips, would be the kind of you know from the things that you've said in our conversation so far, what would be those you know those first three or four things that you think somebody should be looking at in terms of that effective data usage across two or three schools. James Pembroke: Well, I'm going to jump in here. I'm going to jump in with one. And it's one that I've been saying for over 10 years now. And it's don't obsess about measuring progress. I know I keep banging on about this, but if you set up a system with the sole purpose or the main purpose of measuring progress, you're probably going to break your system. We have to recognize what happened to levels. Levels are an effective system, actually, when they were first set up. There was nothing wrong with levels. But levels got broken because we tried to use them to measure progress, initially in terms of broad levels, and then in terms of sub-levels, and then by the end of it, when it got into a real mess, we were pretending we knew the difference in a 2B and a 2B +, you know, so it was ridiculous. And so levels ended up like, we'll get rid of levels. The same thing kind of happened to P scales, and the same thing happened to development matters, when we had 30 to 50 and 40 to 60, which overlapped, but we pretended that we could split them into 30... 30, 50 emerging developing secure and 40 to 60 emerging developing secure. And we're talking about children in early years making four or five points per year. And we completely fractured it to the point of it was irreparable. So then we move away. Every time we try and use a system, particularly a system of teacher assessments, teacher assessments should be about is this child okay or not? And if they're not okay, what can we do to help them? If you then try and take that they're okay but they're an okay minus and they're an okay equals and they're an okay plus then teachers are just to go well I'll put them in the lowest band I can get away with at this point I'll put them in the highest band I can get away with at that point which I know it sounds like I'm really I'm being negative about teachers it's not just human nature you think I'm under the cosh here this system is designed to reflect children's progress which reflects on me and therefore the data becomes distorted which is what we started off talking about so Have a system in place that captures really good quality, useful information, but don't bend it out of shape because you bend it and bend it beyond its elastic limit and it snaps. So don't obsess about measuring children's progress. Richard Selfridge: I'm going to jump in and say that my top tip, my summary is to use a slightly hackneyed phrase, but one that I quite like. I'm going to repurpose it, which is to take back control. I think take back control of. James Pembroke: haha Richard Selfridge: the information that you are that you as an executive head or senior leader as a teacher that you are interested in that you are developing your understanding of you are developing you all the time you're developing your understanding of the children in your classes the classes in your school the schools in your group you're trying to take back control of that and to say were there all these things that we've said understanding the limitations of assessment understanding the need for teacher assessment the need for standardized assessment, have all of those things, but to take back control and to do that, I think the key thing as with so many things in life generally is you need a plan, you need a structure at

any point you should have a reason for what you're doing. As well as our book, Data Proof of Your School, I also wrote a little guide to using student data, which I wrote for Sage, which is a teacher's guide. And in that one, I make the point that particularly in primary, you are teaching one group of children. largely. teaching one group of children and you have a huge opportunity to get to know those pupils so that you should build that knowledge. So you know what you need to do is to have a plan. So when you first meet the children you need to work out which of these children are left-handed? Which of these children are supposed to be wearing glasses? Which of these children look like they should be wearing glasses? There's things that you do but you need a plan. You need to say okay this week I'm focusing on this, this week I'm focusing on this and for you as a senior leader as an executive head teacher if you can create culture so that your teachers take back control of the data and that they have a plan they know this week this half term my focus is on these very small number of things because there's too many things that you could be doing you need to go for the best bets you know what is it that first half of the autumn term that you should be focusing on because that will be very different to what you're focusing on in you know the first half of the summer term but you need a plan take back control of that data build your understanding share that understanding so that as the children move through school, you know more about them. Again, one of my favourite phrases is, should know a lot more about your year fives than you do about your year ones, because you've had them for a lot longer. You should know a lot more about your year fives than your year ones. It is ludicrous that a teacher in year five knows what the children looked like the last time you took a photograph of them. No, you've got years of information about that child. The teacher should understand which of these children do we need to actually really focus on? and which these children would like to touch. So take back control is my main message. Focus on that. Paul Ainsworth: Really wise, really wise. could you just give some thoughts on the listeners about possible next steps for you and insight in form? Are there any more books on the horizon from the two of you? And most importantly, how do school leaders book you to lead a session in their school and how can they find out more about your work? James Pembroke: Well, if you're an insight user, ⁓ can book, ⁓ you can actually book a consultancy call ⁓ with Richard ⁓ or I, or both of us if you really want to. ⁓ In the mornings, ⁓ well, mainly mornings with me, but you know, ⁓ there is a calendar in there, you can book it, you can go into the help menu, you can book an insight informed data surgery. It doesn't cost you anything, so you can do that. ⁓ And yeah, we run sessions for groups of schools and for MATs So can get in touch with us via ⁓ media ⁓ Blue mainly, ⁓ not much X these days. ⁓ Or you can email us through Insight. yeah, we love to hear from you and we are really enjoying running sessions, our data buster sessions where ⁓ the most data buster sessions we've been running have been... ⁓ a sort of split really, a morning looking at accountability and really getting into the weeds of ⁓ the IDSR and DFE performance measures and how progress measures work and how they're going to measure progress using the reception baseline and all kind of the issues around that. kind of really ⁓ a deep dive into, there's an old phrase, ⁓ into how performance of schools is measured. And then the afternoon is much more about stuff, which is just saying that you have control over. So a morning on sort of stuff that you don't really have control over, but you need to know about, and then an afternoon of all the stuff that we can do in schools to ⁓ measure, monitor, track children's progress and performance and attainment ⁓ and needs and barriers and all that sort of thing. So ⁓ if you're interested in that, then please do get in touch with us. No book on the horizon at the moment, although we're constantly talking about it. So watch this space, we might get round to doing another one at some point. Richard Selfridge: We have been talking about books I'm quite interested in a book about you know, supporting children with either SEND or children who don't ⁓ find school easy. But we've kind been waiting for a bit of the legislation, which is kind of coming through there. So once we get a bit more of an understanding of that, we might look at that. One of the things which we're focusing on really beginning to work on is the fact that the data wave appears to be coming back. Becky Allen, Ben White, Matt Evans, they referred to the data wave coming through schools in the 2010s, which followed the differentiation way from earlier. was receding because of the COVID, the impact of COVID and the fact that had this big focus on us. ⁓ We've had a big focus on children's attendance, which has been important and key, but definitely things are coming back that way. One of the things which I'm certainly very focused on at the moment is the fact that we've had this sort of ⁓ all terms in education are disputed, to my mind, but progress means two different things. You expected progress was a number and progress, know, how much can children know more and do more. But the idea of gaps are definitely coming back and gaps between groups and so on and what people mean by that and so on. there's there's quite a bit to for us to unpick there. I think it's made you live in interesting times. I think because of the new Ofsted toolkit framework, because of the changes within the SND system and because of the white paper that's out at the moment, there's a lot of things to be looked at them. And I say certainly an insight and form we will be ⁓ commenting on on things and just trying to support. schools through this because I think there's a lot of opportunity. There's a lot of opportunity still. You know, the thing to remember is the expectation still for you as an executive leader is that you are leading your school. You are there understanding the wider picture, the national picture, understand the picture for your school. So we'll be there helping supporting there as they're our data discos and then doing our data busters days inside road shows for people. hopefully we will be well, we will definitely be continuing. that as we move forward because there's a lot to be done. Paul Ainsworth: Thanks so much, James. Thanks so much, Richard, for sharing your thoughts on step seven of the nine steps to executive headship. We focused on effective data use things today. I'd strongly advise listeners to take a look at data proof your school, the blogs and research from Insight Inform, and yeah, try and attend one of James's and Richard's sessions if you've got the chance to. Thanks so much for your time. I'm really grateful. Richard Selfridge: Thank you very much, Paul.