Release of Primary School Data Disingenuous & Deluded

The Impending Release of Primary School Achievement Data: Disingenuous, Destructive and Deluded

Opinion: Martin Thrupp

23 August 2012

On 8th August, something terrible happened in New Zealand. Unfortunately most of us didn’t notice because it was during the Olympics. On that day the Government revealed its plans for how National Standards achievement data will be released and how it intends to create more consistency between schools in their National Standards judgements and reporting. There is now a ‘Public Achievement Information Plan’ with numerous steps towards ‘incrementally improving the quality of the data’.

In this paper I describe the impending release of primary school achievement data as disingenuous, destructive and deluded. By disingenuous I am not trying to engage in conspiracy theories but I do point out that the Government can’t wash its hands of media-developed league tables and that print media coverage of the potential for harm is suspiciously thin. I then discuss why the decision to release the national standards data can be expected to be destructive to many schools, communities, families and children, indeed truly devastating for some. Following that, I argue that any notion that National Standards data that will get more meaningful over time because of the various consistency measures the Government is putting into place is deluded, as is the idea that many parents will have a better choice of schools because of the release of the data. Lastly, I suggest the situation is still recoverable if there was the political will to change direction.

Disingenuous

Is there anything disingenuous about the Government’s treatment of the release of National Standards data? Absolutely, because back in 2009 then Minister of Education Anne Tolley said repeatedly that the Government would not be creating league tables. This is likely to be why some schools were prepared to go with the National Standards rather than resisting them as many schools did. For instance in the RAINS research I am doing, the principal of the school most supportive of National Standards said last year that she wouldn’t have supported them if they were about league tables and performance pay. Then in June this year the Prime Minister and current Education Minister Hekia Parata signalled their support for league tables. The current stance is that ‘Public Achievement Information’ is not about creating National Standards league tables and strictly speaking that is true. But it is not preventing them either, rather it is facilitating the media being able to create league tables by pulling data together from New Zealand’s 2300 primary and intermediate schools and making it all available on the ‘Education Counts’ website. It is also requiring schools to present their National Standards data in increasing standardised fashion that will make league tables easier to construct. So the branding of this exercise as ‘Public Achievement Information’ is little more than a distraction when the Government is doing all that is needed to make league tables of primary school performance except actually rank the schools.

As for the media, the print media (as a whole, there are some notable exceptions) seems reluctant to engage with the arguments against league tables, perhaps even shutting them down. For instance what should we make of the fact that the Herald has only devoted one sentence (in the item ‘National Standards website a “good compromise” – PM’, August 8) to a public letter against league tables signed by nearly 300 Auckland principals? This represents roughly 70% of the primary and intermediate principals in the wider Auckland area, was it not more newsworthy than this? Another example is a recent Herald editorial ('Flawed school data no reason not to publish', August 10) that started off by pointing out that I had argued that ‘schools will use tricks to portray themselves in the best possible light in National Standards results that will be published next month’. By leaving out the details of what had been reported in the Herald two days earlier, this carried the implication I could be criticising the schools, which I certainly wasn’t. (What I have always been saying, and will say again here, is that if you want schools to be honest and authentic then don’t incentivise them to be otherwise by making the data public). And finally what about John Hartevelt from Fairfax Media who has been reporting on the developments around league tables, covering a range of views, but has now put in an OIA requests to all schools for the data, presumably in order that his employer can publish league tables?

In all of this I think we need to recognise that the print media, in particular, stands to gain from the publication of league tables. Based on overseas experience, it will create an annual spectacle that will sell a lot of newspapers. Perhaps it is this, as much as their general political stance, that explains why newspaper editors seem to prefer to engage in relatively general and abstract arguments about whether or not parents have the right to the information rather than focus on the important issue of harm, of why releasing the data is likely to be destructive.
Destructive

To me the Public Achievement Information plans are the educational equivalent of introducing a nasty disease into every primary and intermediate school in the country. Actually the disease is already there but relatively contained. Now it is about to go viral and there is not a school or school community in the country that will be unscathed, although as we will see, some will be afflicted more than others, in fact for some it is likely to be fatal.

I am talking about powering up a culture of performativity within schools. What this means in this situation is that the new professionalism that can be expected to grow up in the primary sector now will be to get as many children ‘at’ or ‘above’ in the National Standards since these will become the headline statistics in the league tables of school performance that the media will pull together.

“No bad thing” you might say, but note I did not say that the new professionalism will be to “authentically teach the children so well in reading, writing and maths that they can all genuinely achieve ‘at’ or ‘above’ in the National Standards as well as carry on doing all the other things that we expect in a rich and full primary curriculum”. No, that is difficult, indeed impossible, in most schools so there are quicker and easier methods that schools will be pushed to use to increase the proportions of children ‘at’ and ‘above’. They can be expected to include:

Turning away students who are unlikely to achieve ‘at’ or ‘above’. Children with special needs, English as a second language, troubled backgrounds or a track record of not achieving well will become a liability, indeed a threat, to schools once the data is released. But there are many subtle and not so subtle ways that schools can discourage enrolment of these children without actually breaking any laws. They include the extent to which they make provision for such children, the discussions they have with prospective parents and the ways schools draw up their enrolment zones.

Narrowing the curriculum. The National Standards are in reading, writing and maths, not in science, PE, art, drama, social studies, environmental education or the host of other things taught in primary schools. While in theory the National Standards can be taught by way of the broad school curriculum, the pressure towards a narrower, more direct approach will now intensify. Curriculum narrowing is not just a problem for highly creative children or those with particular abilities in other areas. Most children will find school more tedious with a narrowing curriculum and it reduces the range of options available to teachers trying to respond to the interests of children too.

Giving a lot more attention to some students than others. This is what is known as ‘educational triage’ after the medical process of making decisions about who is worth saving and who isn’t. It means we are likely to see intensive work with some students and things like ‘booster classes’ depending on what will make a difference for the school in terms of its published data. But if your child is not in a group that will make a difference for the school - say they are going to be very difficult to move up or they are already ‘above’ - then they are unlikely to get as much attention.

Advantageous approaches to assessment. There are many steps in the process of assessing students where teachers can make decisions that will lead to a more favourable outcome. For instance an ‘unassisted’ writing sample can range from being truly thrown in the deep end to being rather more supported, depending on the approach that a school takes. The specific ways in which formal assessment tools are carried out also provides many options for getting a more favourable result. So of course when teachers are put under pressure to maximise the number of children getting through ‘at’ or ‘above’, they can be predicted to take the path of least resistance.

Obvious dishonesty. We also know from overseas experience that in a very small number of schools, someone on staff will be so desperate to do better that they will just revise the figures up, they will cheat.

A few general points about all this. I don’t see the scenario I am painting being already in place in schools through the introduction of National Standards. There are some elements of it because schools have been mindful their data is going to the Ministry of Education and many have been considering the prospect of the data becoming public. But at the moment school staff still seem prepared to be pretty open about what is going on in their schools. For instance in the RAINS project we have a school where a lot of children were put ‘above’ in reading but the staff there are happy to admit they believe it’s an artefact of the test that was used and that if they had used another test many fewer would have been ‘above’ and they are making that change. Another school has lots of kids put at ‘well below’ for writing and the DP there just says that is realistically where those children should be. But I think all that will change, that we will see more children being put ‘at’ or ‘above’ standard and schools becoming more insistent that the achievement is at those levels.
This is the logic of the situation. We could say it shouldn't happen but it will because schools are being incentivised to take these ‘shortcuts’ and play these ‘tricks’. And as professional cultures shift, it will no longer be a trick or a shortcut; it will just become what you do to try to ensure your school prospers. Professional identities will change as responding to league tables becomes the ‘bread and butter’ of the job. Schools and teachers will come to measure their worth and value by their National Standards achievement and the position of their schools in the league tables. There will be greater anxiety around National Standards performance in classrooms, staffrooms, senior management and board meetings. A kind of ‘economy’ will develop where the energies and funding in each schools is directed to doing well in the league tables and away from other areas.

These patterns are well supported in the international literature and we can see them in other education sectors here too. In the tertiary sector, where staff and institutions are responding to various performance indicators, there is a lot of trickiness. For instance I understand the Tertiary Education Union has been getting complaints from staff who are being told what their pass rates need to be while others are having their grades modified by senior managers who are chasing particular targets and performance indicators. There is also the game of people being hired or ‘moved on’ or given new roles in order to do better in the PBRF, the Performance Based Research Fund. We can also see some of the same patterns with NCEA. One issue which has been repeatedly raised in the past is that some schools have offered more ‘easy’ unit standards than others, allowing them to look more successful when they achieve higher pass rates than schools doing more difficult ‘achievement standards’. Another concern often raised is the problem of ‘credit cleansing’, schools boosting their pass rates by cancelling the NCEA enrolments of students who are unlikely to be successful. But the difference between NCEA and the situation at primary levels is that NCEA goes across many subject areas and levels and the resulting league tables are complicated. Whereas at primary the headline statistic of the percentage ‘at’ and ‘above’ standard in just reading, writing and maths will be deceptively straightforward and just the kind of simplistic approach that will make those league tables really take off.

The destructive effects could occur in schools anywhere on the socio-economic spectrum, even in schools that are much sought after by the middle classes, because the fear of falling in the league tables will be everywhere and there is always the problem of a ‘rogue’ year group that will upset the pattern of year-on-year improvement that all schools will want to see. But it is in low socio-economic schools serving mainly the country’s more vulnerable families that the release of National Standards data will be most devastating. These schools will be on the back foot because their children typically come in less ready for school and there are continuing issues related to poverty and transience and many also have lots of issues with children with special needs and English Language Learners. All this will impact heavily on the published achievement data, as it will be in the form of raw rather than ‘value-added’ figures. There will be a regional and national comparison and based on present practices with the NCEA the media will probably try to contextualise by decile but that crude comparison will just makes things worse in some ways. So I’m expecting intensified local hierarchies of high performing ‘star’ schools and others demonized as ‘failing’ with these hierarchies mainly reflecting socio-economic and other contextual differences. The ‘star’ schools will find it even easier to recruit new staff, whereas a low position on the league tables will tend to further discourage applicants for lower socio-economic schools. Wider criteria for choosing schools will become increasing ignored. League tables will also affect teacher expectations. Frustration about ‘underachieving’ children will translate to them even if nothing is said. Unfortunately there is a lot of evidence that being told you or your school is not up to standard undermines students’ identities as learners because these are constructed through assessment processes.

Wider changes can be expected to occur as well. Numeracy and literacy will come to further dominate teacher education as providers respond to the demands from schools. This will be another pressure changing what it means to be a teacher. Schools will encourage parents to seek and pay for outside tutoring for their children: we are already seeing some tutoring services being explicitly targeted at children who have been assessed as ‘below National Standards’). The league tables will inevitably become the focus of target-setting and other policy and political commentary, an easy thing to focus on rather than digging into what is actually happening behind the figures. And of course the new charter schools will be waiting in the wings wherever existing provision is deemed to be failing.

Deluded

At the moment schools’ approaches to the National Standards are all over the place. The RAINS research illustrates that this is because they involve different trajectories that reflect school-specific contexts including student intakes and the history of assessment and curriculum development in each school. The Government clearly knows this variability is a problem too as it is now using the various means at its disposal to try to create more consistency between schools in both judgements and reporting. So they are introducing and making compulsory the Progress and Consistency Tool, an online platform that will help teachers to line up different kinds of assessment tools and other data and create an OTJ. They are also tightening up reporting requirements so that everyone provides data in the same format, not in different ways as they did this year.

Now the Minister’s expectation is that through these measures the data “will get better and better and be more useful over time”. Becoming more ‘meaningful’ is the expression often used. But of course at the same time the Government is taking these steps, it is creating more incentive for schools to be gaming the system. So talk of the data becoming meaningful is nonsense, it may well become less meaningful in real terms. The problem for the Government is that while the PACT tool might remove the benefits to be gained by choosing one assessment tool over another, it won’t be able to be able to control all the variation in assessment processes, the turning away of students, the educational triage and the random instances of obvious cheating. So long as the incentive is there, every time a loophole is closed with new directives, the people out in schools will find another one. And if the Government becomes more serious about policing schools’ processes, the costs of trying to ensure compliance will get very high.

Also deluded is any notion that school choices will be improved for most people. On the one hand the release of data will encourage consumerist behaviour by parents and yet the system will be unable to deliver to most of them because they won’t be able to get their children into the top schools in the league tables. So there will be many needlessly dissatisfied parents who will have been encouraged to exit their local school whereas had they engaged with it more positively, would probably find it works just fine for their child. It is also worth recalling that the data is only in reading, writing and maths and is already out of date by the time it gets to parents. Someone using the data to choose a school is actually interested in a prediction of future achievement but in practice there is little certainty because their child will typically go through with a different student cohort, quite possibly with different teachers or board members, and maybe even a different principal.

The way forward

I have argued that the impending release of primary school achievement data is disingenuous and will prove destructive and deluded. When it comes to ‘meaningful’ data, the emperor will have no clothes because of all that will be going on behind the scenes. So where to now? We are told there is no alternative to releasing the data because of the Official Information Act and the Ombudsman’s recent ruling would seem to confirm this. So this year’s ‘ropey’ data will have to be published although anyone who wants to take it seriously should be laughed off the planet. But then the problem should be fixed by removing the reporting requirements around the National Standards or, better still, getting rid of National Standards altogether and letting schools just work with the underlying curriculum levels and assessment tools. In my view the situation without repeated public release of the data would still be recoverable. I’m basing this on the preference to be honest and frank that I see in the RAINS schools. But the outlook for the culture of New Zealand primary and intermediate schools without shutting down the release of data is bleak. What the politicians and the general public need to grapple with is the paradox that the more high stakes pressure is placed on teachers, the less authentic their teaching will become and that there is no easy way to get around this problem.

Martin Thrupp is Professor of Education at the University of Waikato. This opinion piece is based on presentations given at ‘Stand Up for Kids’ public meetings organised by the NZEI at Kelston Community Centre, West Auckland, 21st August 2012 and Lakeside Novotel, Rotorua, 23 August 2012.

ENDS

© Scoop Media