Will Fairfax respond to unfairness of school comparisons?

The next stage of releasing this year’s National Standards judgements will be the release of data on individual schools later this month. When it happens Fairfax’s response should be of considerable public interest.

It has become increasing clear that it is impossible to use National Standards data for comparing the achievement of schools. It was obvious from the outset that the data was raw data that doesn’t take account of differences in the nature of school intakes. It has subsequently also become apparent from research that numerous sources of variation underlie the way the data is collected making it even more unsuitable for comparative purposes.

Despite this, Fairfax may be gearing up for publishing another round of National Standards data by way of its newspapers and especially the ‘School Report’ section of its various websites. A new heading has appeared on the Stuff education page: ‘School report: data now available for 2009 New Zealand Schools’. When compared to other media sources such as APN’s The New Zealand Herald and Radio New Zealand, Fairfax also appears to have little coverage of recent criticisms of the release of the data.

It is concerning that Fairfax might want to continue promoting the comparison of schools’ National Standards judgements despite the increasing evidence that this is misleading. I reproduce below a statement sent to Fairfax last month where I urged this media company to stop publishing the data.

Statement about Fairfax publication of National Standards data (newspapers and digital) in 2013.

Last year Fairfax made a point of trying to be responsible about its initial publication of ‘ropey’ National Standards data. Fairfax’s September 2012 release of the data was qualified by the editorial comments of then senior political reporter, John Hartevelt and by a range of other commentaries and case studies.

Since that time it has become apparent that because Fairfax put the data online in particular ways, the release of data now continues in a way that is largely unqualified. A reader can currently go onto Fairfax’s Stuff education page or just the home pages of papers like the Waikato Times or Nelson Mail, be invited by the ‘School Report’ button to pick up to ten schools and can start comparing their results without having to be exposed to very much in the way of context.

For this reason, whether Fairfax reporters bother with the same level of qualifying commentary this year as they did when the data was first published in September 2012 is not the test of commitment to a contextualised account of the data that it might have otherwise been. Fairfax is already failing in requiring on any continuing basis the contextualised view of National Standards it promised readers last September when Hartevelt claimed that Fairfax had “...not simply dumped all of the new National Standards data online”.

My latest RAINS research report released last week illustrates the further serious problem of much variability in what the data actually represents across schools. It may have some meaning within the assessment processes of individual schools but shouldn't be used for making comparisons between them - see especially section 3 of the report.

Just yesterday Nikki Kaye, Associate Minister of Education, said in the House that ‘we [the Government] will be publishing the data because we are confident of overall teacher judgments, not just one assessment tool’. Yet the RAINS research indicates the data does not deserve the confidence being expressed in it by Government.

In the circumstances I urge Fairfax not to publish this year’s National Standards data and to remove the 2012 data from its websites. The publication of such a poor data set has become increasingly indefensible.

Martin Thrupp is a Professor of Education at the University of Waikato.

© Scoop Media