John Gerritsen, Education Correspondent
Universities are blocking the publication of survey results that rate the usefulness of hundreds of their courses.
Photo: Sara Haj-Hassan / freeimages.com
The Tertiary Education Commission told RNZ it had sufficient responses to its survey to publish ratings for 639
university qualifications, and 1528 qualifications offered by other types of tertiary institutions.
The survey asked recent graduates to rate how well their qualification prepared them for work and whether they would
recommend it to others.
The commission said it needed at least 50 responses for qualifications with 250 graduates or more, and at least five
responses for qualifications with 50 or fewer graduates.
Universities New Zealand director Chris Whelan said statisticians had advised that the survey would have a high margin
of error.
"On average for these kind of self-selecting surveys, you can expect bias of about 20 to 40 percent and for some small
courses, under 50 students, which represent more than half the total number of courses offered by universities, they
were going to accept a minimum sample size of just five responses."
Mr Whelan said the low response rate required for publication meant disgruntled students could skew the outcomes for
some courses.
"If you've got a sample size of say just five people and they've got together and said, 'Well that was a miserable
course, why don't we all fill out this survey,' you might have 45 very happy other students out there who just aren't on
social media, or haven't seen the survey or haven't be bothered to answer it."
Mr Whelan said universities would support the survey if the threshold for publication of results was a 25 percent
response rate from graduates.
Chris Whelan, executive director, Te Pōkai Tara - Universities New Zealand. Photo: SUPPLIED
The Tertiary Education Commission said it would not publish results for university qualifications because universities
had opted out of the system.
However, the commission's deputy chief executive of information, Brendan Kelly, said the survey methodology was sound
and the margin of error was relatively low.
"Depending on sample size, it's up to 13 percent and we're putting safeguards and caveats around that. We actually have
a much higher response threshold before we will publish than a number of surveys overseas."
Mr Kelly said more than 12,000 graduates had responded to the survey, though only 7700 of those responses were valid.
Respondents had provided a representative mix of graduates and the commission had enough information to publish results,
he said.
"At a national level we can display results for around 1500 qualifications now, and at a specific qualification level
the numbers are lower, we've got around 26 that are ready to go."
The commission created the survey under the previous government and had spent $1.6m developing it.
It had expected to start publishing results in February 2017, but that was delayed in part because of efforts to ensure
the methodology was robust. There was still no date for publishing the figures.