The Impending Release of Primary School Achievement Data: Disingenuous, Destructive and Deluded
Opinion: Martin Thrupp
23 August 2012
On 8th August, something terrible happened in New Zealand. Unfortunately most of us didn’t notice because it was during
the Olympics. On that day the Government revealed its plans for how National Standards achievement data will be released
and how it intends to create more consistency between schools in their National Standards judgements and reporting.
There is now a ‘Public Achievement Information Plan’ with numerous steps towards ‘incrementally improving the quality of
the data’.
In this paper I describe the impending release of primary school achievement data as disingenuous, destructive and
deluded. By disingenuous I am not trying to engage in conspiracy theories but I do point out that the Government can’t wash its hands of
media-developed league tables and that print media coverage of the potential for harm is suspiciously thin. I then
discuss why the decision to release the national standards data can be expected to be destructive to many schools, communities, families and children, indeed truly devastating for some. Following that, I argue that
any notion that National Standards data that will get more meaningful over time because of the various consistency
measures the Government is putting into place is deluded, as is the idea that many parents will have a better choice of schools because of the release of the data. Lastly, I
suggest the situation is still recoverable if there was the political will to change direction.
Disingenuous
Is there anything disingenuous about the Government’s treatment of the release of National Standards data? Absolutely,
because back in 2009 then Minister of Education Anne Tolley said repeatedly that the Government would not be creating
league tables. This is likely to be why some schools were prepared to go with the National Standards rather than
resisting them as many schools did. For instance in the RAINS research I am doing, the principal of the school most
supportive of National Standards said last year that she wouldn’t have supported them if they were about league tables
and performance pay. Then in June this year the Prime Minister and current Education Minister Hekia Parata signalled
their support for league tables. The current stance is that ‘Public Achievement Information’ is not about creating
National Standards league tables and strictly speaking that is true. But it is not preventing them either, rather it is
facilitating the media being able to create league tables by pulling data together from New Zealand’s 2300 primary and
intermediate schools and making it all available on the ‘Education Counts’ website. It is also requiring schools to
present their National Standards data in increasing standardised fashion that will make league tables easier to
construct. So the branding of this exercise as ‘Public Achievement Information’ is little more than a distraction when
the Government is doing all that is needed to make league tables of primary school performance except actually rank the
schools.
As for the media, the print media (as a whole, there are some notable exceptions) seems reluctant to engage with the
arguments against league tables, perhaps even shutting them down. For instance what should we make of the fact that the
Herald has only devoted one sentence (in the item ‘National Standards website a “good compromise” – PM’, August 8) to a
public letter against league tables signed by nearly 300 Auckland principals? This represents roughly 70% of the primary
and intermediate principals in the wider Auckland area, was it not more newsworthy than this? Another example is a
recent Herald editorial ('Flawed school data no reason not to publish', August 10) that started off by pointing out that
I had argued that ‘schools will use tricks to portray themselves in the best possible light in National Standards
results that will be published next month’. By leaving out the details of what had been reported in the Herald two days
earlier, this carried the implication I could be criticising the schools, which I certainly wasn’t. (What I have always
been saying, and will say again here, is that if you want schools to be honest and authentic then don’t incentivise them
to be otherwise by making the data public). And finally what about John Hartevelt from Fairfax Media who has been
reporting on the developments around league tables, covering a range of views, but has now put in an OIA requests to all
schools for the data, presumably in order that his employer can publish league tables?
In all of this I think we need to recognise that the print media, in particular, stands to gain from the publication of
league tables. Based on overseas experience, it will create an annual spectacle that will sell a lot of newspapers.
Perhaps it is this, as much as their general political stance, that explains why newspaper editors seem to prefer to
engage in relatively general and abstract arguments about whether or not parents have the right to the information
rather than focus on the important issue of harm, of why releasing the data is likely to be destructive.
Destructive
To me the Public Achievement Information plans are the educational equivalent of introducing a nasty disease into every
primary and intermediate school in the country. Actually the disease is already there but relatively contained. Now it
is about to go viral and there is not a school or school community in the country that will be unscathed, although as we
will see, some will be afflicted more than others, in fact for some it is likely to be fatal.
I am talking about powering up a culture of performativity within schools. What this means in this situation is that the
new professionalism that can be expected to grow up in the primary sector now will be to get as many children ‘at’ or
‘above’ in the National Standards since these will become the headline statistics in the league tables of school
performance that the media will pull together.
“No bad thing” you might say, but note I did not say that the new professionalism will be to “authentically teach the
children so well in reading, writing and maths that they can all genuinely achieve ‘at’ or ‘above’ in the National
Standards as well as carry on doing all the other things that we expect in a rich and full primary curriculum”. No, that
is difficult, indeed impossible, in most schools so there are quicker and easier methods that schools will be pushed to
use to increase the proportions of children ‘at’ and ‘above’. They can be expected to include:
Turning away students who are unlikely to achieve ‘at’ or ‘above’. Children with special needs, English as a second
language, troubled backgrounds or a track record of not achieving well will become a liability, indeed a threat, to
schools once the data is released. But there are many subtle and not so subtle ways that schools can discourage
enrolment of these children without actually breaking any laws. They include the extent to which they make provision for
such children, the discussions they have with prospective parents and the ways schools draw up their enrolment zones.
Narrowing the curriculum. The National Standards are in reading, writing and maths, not in science, PE, art, drama,
social studies, environmental education or the host of other things taught in primary schools. While in theory the
National Standards can be taught by way of the broad school curriculum, the pressure towards a narrower, more direct
approach will now intensify. Curriculum narrowing is not just a problem for highly creative children or those with
particular abilities in other areas. Most children will find school more tedious with a narrowing curriculum and it
reduces the range of options available to teachers trying to respond to the interests of children too.
Giving a lot more attention to some students than others. This is what is known as ‘educational triage’ after the
medical process of making decisions about who is worth saving and who isn’t. It means we are likely to see intensive
work with some students and things like ‘booster classes’ depending on what will make a difference for the school in
terms of its published data. But if your child is not in a group that will make a difference for the school - say they
are going to be very difficult to move up or they are already ‘above’ - then they are unlikely to get as much attention.
Advantageous approaches to assessment. There are many steps in the process of assessing students where teachers can make
decisions that will lead to a more favourable outcome. For instance an ‘unassisted’ writing sample can range from being
truly thrown in the deep end to being rather more supported, depending on the approach that a school takes. The specific
ways in which formal assessment tools are carried out also provides many options for getting a more favourable result.
So of course when teachers are put under pressure to maximise the number of children getting through ‘at’ or ‘above’,
they can be predicted to take the path of least resistance.
Obvious dishonesty. We also know from overseas experience that in a very small number of schools, someone on staff will
be so desperate to do better that they will just revise the figures up, they will cheat.
A few general points about all this. I don’t see the scenario I am painting being already in place in schools through
the introduction of National Standards. There are some elements of it because schools have been mindful their data is
going to the Ministry of Education and many have been considering the prospect of the data becoming public. But at the
moment school staff still seem prepared to be pretty open about what is going on in their schools. For instance in the
RAINS project we have a school where a lot of children were put ‘above’ in reading but the staff there are happy to
admit they believe it’s an artefact of the test that was used and that if they had used another test many fewer would
have been ‘above’ and they are making that change. Another school has lots of kids put at ‘well below’ for writing and
the DP there just says that is realistically where those children should be. But I think all that will change, that we
will see more children being put ‘at’ or ‘above’ standard and schools becoming more insistent that the achievement is at
those levels.
This is the logic of the situation. We could say it shouldn't happen but it will because schools are being incentivised
to take these ‘shortcuts’ and play these ‘tricks’. And as professional cultures shift, it will no longer be a trick or a
shortcut; it will just become what you do to try to ensure your school prospers. Professional identities will change as
responding to league tables becomes the ‘bread and butter’ of the job. Schools and teachers will come to measure their
worth and value by their National Standards achievement and the position of their schools in the league tables. There
will be greater anxiety around National Standards performance in classrooms, staffrooms, senior management and board
meetings. A kind of ‘economy’ will develop where the energies and funding in each schools is directed to doing well in
the league tables and away from other areas.
These patterns are well supported in the international literature and we can see them in other education sectors here
too. In the tertiary sector, where staff and institutions are responding to various performance indicators, there is a
lot of trickiness. For instance I understand the Tertiary Education Union has been getting complaints from staff who are
being told what their pass rates need to be while others are having their grades modified by senior managers who are
chasing particular targets and performance indicators. There is also the game of people being hired or ‘moved on’ or
given new roles in order to do better in the PBRF, the Performance Based Research Fund. We can also see some of the same
patterns with NCEA. One issue which has been repeatedly raised in the past is that some schools have offered more ‘easy’
unit standards than others, allowing them to look more successful when they achieve higher pass rates than schools doing
more difficult ‘achievement standards’. Another concern often raised is the problem of ‘credit cleansing’, schools
boosting their pass rates by cancelling the NCEA enrolments of students who are unlikely to be successful. But the
difference between NCEA and the situation at primary levels is that NCEA goes across many subject areas and levels and
the resulting league tables are complicated. Whereas at primary the headline statistic of the percentage ‘at’ and
‘above’ standard in just reading, writing and maths will be deceptively straightforward and just the kind of simplistic
approach that will make those league tables really take off.
The destructive effects could occur in schools anywhere on the socio-economic spectrum, even in schools that are much
sought after by the middle classes, because the fear of falling in the league tables will be everywhere and there is
always the problem of a ‘rogue’ year group that will upset the pattern of year-on-year improvement that all schools will
want to see. But it is in low socio-economic schools serving mainly the country’s more vulnerable families that the
release of National Standards data will be most devastating. These schools will be on the back foot because their
children typically come in less ready for school and there are continuing issues related to poverty and transience and
many also have lots of issues with children with special needs and English Language Learners. All this will impact
heavily on the published achievement data, as it will be in the form of raw rather than ‘value-added’ figures. There
will be a regional and national comparison and based on present practices with the NCEA the media will probably try to
contextualise by decile but that crude comparison will just makes things worse in some ways. So I’m expecting
intensified local hierarchies of high performing ‘star’ schools and others demonized as ‘failing’ with these hierarchies
mainly reflecting socio-economic and other contextual differences. The ‘star’ schools will find it even easier to
recruit new staff, whereas a low position on the league tables will tend to further discourage applicants for lower
socio-economic schools. Wider criteria for choosing schools will become increasing ignored. League tables will also
affect teacher expectations. Frustration about ‘underachieving’ children will translate to them even if nothing is said.
Unfortunately there is a lot of evidence that being told you or your school is not up to standard undermines students’
identities as learners because these are constructed through assessment processes.
Wider changes can be expected to occur as well. Numeracy and literacy will come to further dominate teacher education as
providers respond to the demands from schools. This will be another pressure changing what it means to be a teacher.
Schools will encourage parents to seek and pay for outside tutoring for their children: we are already seeing some
tutoring services being explicitly targeted at children who have been assessed as ‘below National Standards’). The
league tables will inevitably become the focus of target-setting and other policy and political commentary, an easy
thing to focus on rather than digging into what is actually happening behind the figures. And of course the new charter
schools will be waiting in the wings wherever existing provision is deemed to be failing.
Deluded
At the moment schools’ approaches to the National Standards are all over the place. The RAINS research illustrates that
this is because they involve different trajectories that reflect school-specific contexts including student intakes and
the history of assessment and curriculum development in each school. The Government clearly knows this variability is a
problem too as it is now using the various means at its disposal to try to create more consistency between schools in
both judgements and reporting. So they are introducing and making compulsory the Progress and Consistency Tool, an
online platform that will help teachers to line up different kinds of assessment tools and other data and create an OTJ.
They are also tightening up reporting requirements so that everyone provides data in the same format, not in different
ways as they did this year.
Now the Minister’s expectation is that through these measures the data “will get better and better and be more useful
over time”. Becoming more ‘meaningful’ is the expression often used. But of course at the same time the Government is
taking these steps, it is creating more incentive for schools to be gaming the system. So talk of the data becoming
meaningful is nonsense, it may well become less meaningful in real terms. The problem for the Government is that while
the PACT tool might remove the benefits to be gained by choosing one assessment tool over another, it won’t be able to
be able to control all the variation in assessment processes, the turning away of students, the educational triage and
the random instances of obvious cheating. So long as the incentive is there, every time a loophole is closed with new
directives, the people out in schools will find another one. And if the Government becomes more serious about policing
schools’ processes, the costs of trying to ensure compliance will get very high.
Also deluded is any notion that school choices will be improved for most people. On the one hand the release of data
will encourage consumerist behaviour by parents and yet the system will be unable to deliver to most of them because
they won’t be able to get their children into the top schools in the league tables. So there will be many needlessly
dissatisfied parents who will have been encouraged to exit their local school whereas had they engaged with it more
positively, would probably find it works just fine for their child. It is also worth recalling that the data is only in
reading, writing and maths and is already out of date by the time it gets to parents. Someone using the data to choose a
school is actually interested in a prediction of future achievement but in practice there is little certainty because
their child will typically go through with a different student cohort, quite possibly with different teachers or board
members, and maybe even a different principal.
The way forward
I have argued that the impending release of primary school achievement data is disingenuous and will prove destructive
and deluded. When it comes to ‘meaningful’ data, the emperor will have no clothes because of all that will be going on behind the
scenes. So where to now? We are told there is no alternative to releasing the data because of the Official Information Act and
the Ombudsman’s recent ruling would seem to confirm this. So this year’s ‘ropey’ data will have to be published although
anyone who wants to take it seriously should be laughed off the planet. But then the problem should be fixed by removing
the reporting requirements around the National Standards or, better still, getting rid of National Standards altogether
and letting schools just work with the underlying curriculum levels and assessment tools. In my view the situation
without repeated public release of the data would still be recoverable. I’m basing this on the preference to be honest
and frank that I see in the RAINS schools. But the outlook for the culture of New Zealand primary and intermediate
schools without shutting down the release of data is bleak. What the politicians and the general public need to grapple
with is the paradox that the more high stakes pressure is placed on teachers, the less authentic their teaching will
become and that there is no easy way to get around this problem.
Martin Thrupp is Professor of Education at the University of Waikato. This opinion piece is based on presentations given
at ‘Stand Up for Kids’ public meetings organised by the NZEI at Kelston Community Centre, West Auckland, 21st August
2012 and Lakeside Novotel, Rotorua, 23 August 2012.
ENDS