Academics warned of research-assessment problems
Association of University Staff academic vice-president, Dr Grant Duncan, will today urge tertiary-education staff not
to support any future research-assessment schemes, especially those based on bibliometrics. Dr Duncan, in a paper to the
Tertiary Education Commission (TEC) forum, “Measuring Research Performance: What are the Options?”, will highlight
serious problems with the Performance-Based Research Fund’s use of the individual as the unit of research assessment,
the way in which scores are allocated to individuals, and the release and misuse of individual scores.
Speaking personally rather than as a representative of AUS, Dr Duncan will argue that the individual unit of assessment
creates a mismatch between what is being assessed (individual researchers and their achievements) and what is being
rewarded (universities and their budgets).
“Naturally, research is actually performed by, and depends upon, individuals and teams of individuals. But it is the
aggregate institutional ranking that determines the funding allocation; whereas the creation and release of quality
scores for individuals has resulted in much invidious behaviour and many misuses of those scores, or cheap internally
produced imitations of PBRF quality scores, and the pressure on staff to comply with the audit criteria means that a
public policy and management issue is force-fed to individuals,” Dr Duncan will tell the forum.
In his paper, Some principles for redesigning the PBRF, Dr Duncan will also tackle the additional problems that would be
associated with a bibliometric, rather than peer-review, approach to research assessment, especially if it too is based
on the individual. He will highlight the lack of accuracy in citations indexes and controversy over journal rankings;
the gap between being cited and being read, let alone understood; and the migration of research outputs into
“high-impact” off-shore publications and its deleterious effect on New Zealand-based studies.
In addition, Dr Duncan will deal with such problems as the possible rewarding of research that causes controversy and is
consequently widely cited despite any poor analysis or because of offensive implications; differences in citation
conventions and rates across different disciplines and languages; co-authorship exploitation of graduate students; and
assessment of contribution percentages among multiple co-authors.
Acknowledging the validity of the view that the PBRF has had some benefits for academics, Dr Duncan will add, “But also,
with the present PBRF, we have seen some academics being pressured, harassed, and bullied into research and publishing.
The absurdity and injustice of this should be evident to anyone. I can only see this getting worse with the narrow range
of criteria specifiable by bibliometric evaluation methods.”
Dr Duncan is a member of the TEC’s PBRF Sector Reference Group for the 2012 PBRF quality evaluation consultation
exercise.
Also in Tertiary Update this week
1. OECD confirms lack of support, says NZUSA
2. Participation urged in PBRF consultation
3. $40 million for genomics research
4. One-stop tertiary shop in Wanganui
5. Upsets expected in research assessment exercise
6. Melbourne academics face sack
7. Research and tobacco
8. Review recommends full research funding
9. Bastardisation or honour?
OECD confirms lack of support, says NZUSA
Figures released in the recent Organisation for Economic Cooperation and Development report, Education at a Glance 2008,
confirm New Zealand’s lack of investment in student support and the high cost borne by students and their families,
according to the New Zealand Union of Students’ Associations. “This report proves we have a high-cost and
limited-student-support tertiary system that leaves graduates burdened by debt simply for getting an education,” said
NZUSA co-president Paul Falloon.
NZUSA said that the 1.5 percent of GDP spent on tertiary education in New Zealand is equal to the OECD average; however
a greater share comes from students and their families than in many other countries. Private expenditure in the form of
tuition fees accounted for a “massive” 40 percent of expenditure on tertiary institutions, while the OECD average is
only 27 percent.
“It’s clear that the lion’s share of funding to the tertiary sector is coming from those who can least afford it, young
students and their families,” said Mr Falloon. “This is an inequitable and unsustainable situation given large tuition
fees and the high cost of living New Zealanders face. There’s nothing spare to give,” he added.
The report reveals that the average fee for a New Zealand degree-level qualification was $US2,671 in 2005, while eight
of the OECD countries surveyed charged no fees. Diploma-level average fees were $US2,489 and six countries charged no
fees at all at this level. “New Zealand is often compared to Ireland, and along with Norway, Denmark, and Sweden, that
country charges no fees for tertiary education with excellent results academically and for its economy. We would do well
to follow this example”, said Mr Falloon.
“Contrary to misleading and inflammatory claims by the New Zealand Vice-Chancellors’ Committee,” he concluded, “only
around 23 percent of government expenditure on tertiary education goes directly to student support, only slightly above
the OECD average of 18 percent.”
Participation urged in PBRF consultation
Two more consultation papers have been released by the Tertiary Education Commission in the 2012 PBRF Quality Evaluation
process. The papers, which follow the initial 2012 Quality Evaluation paper, cover the unit of assessment and special
circumstances, and have a deadline for response of 17 October 2008. Further consultation papers are to follow.
The unit of assessment consultation paper discusses issues with regard to the PBRF unit of assessment for the 2012
quality assessment; considers some options for addressing these concerns and makes a recommendation; and invites
feedback on the options and recommendations of the Sector Reference Group (SRG) presented in the paper and on any other
matters regarding the unit of assessment that should be considered as part of the redesign process.
The special circumstances paper discusses issues and concerns about the special circumstance provisions and their
application in the PBRF quality evaluation; considers some options for addressing those concerns; and invites feedback
on the options and recommendations of the SRG presented in the paper and on any other matters regarding special
circumstances that should be considered as part of the redesign process.
AUS academic vice-president and SRG member, Dr Grant Duncan, is urging maximum participation by AUS members in the PBRF
consultation process. “These papers raise important issues that directly affect our members. AUS will be making
submissions on the papers but I also encourage individuals to read them and make their own submissions directly to TEC
or through their branch to complement the AUS one,” he said. “The unit of assessment paper is of special significance
for our members.”
$40 million for genomics research
Researchers behind a new genomics research infrastructure involving several of the country’s universities and crown
research institutes (CRIs) say it will accelerate the progress of genetic research and technology in New Zealand. The
initiative will bring together scientists from three universities, the University of Otago, Massey University, and the
University of Auckland, and one crown research institute, AgResearch. It also has support from another four universities
and seven CRIs.
The infrastructure will receive $40 million in government funding through the Ministry of Research, Science and
Technology over the next nine years alongside parallel investment from collaborating institutions. Modern genomic
technologies, which allow scientists to examine the structure and function of thousands of genes at a time, are
considered vital to advancing research in health, agriculture, horticulture, biosecurity, and biodiversity.
Director of the University of Otago’s cancer genetics laboratory, Professor Anthony Reeve, a leader in the
infrastructure’s development, says such a collaborative national infrastructure will help New Zealand stay at the
forefront in genomic research. “New technologies and methodologies are emerging all the time. We need to keep pace with
that change for the benefit of New Zealanders,” he said.
Each of the key institutions will bring different areas of expertise to the infrastructure. As the lead institution, the
University of Otago has expertise in all areas, including genome sequencing, bioinformatics, and gene arrays, the
technology used to examine genetic variations. This is complemented by the strong expertise Massey University has in
sequencing, genomics, and bioinformatics, and the University of Auckland’s and AgResearch’s expertise in genomics and
bioinformatics. They also have complementary technologies which do different tasks.
One-stop tertiary shop in Wanganui
Students in the Wanganui region will soon have a new and possibly unique way to map out the best study route to their
chosen careers. A new one-stop-shop resource will be based at Wanganui’s new tertiary-education campus, Matapihi ki te
Ao, and is the first step in what has been described as a new concept for tertiary education in New Zealand. The
resource provides information on all tertiary-study options in the region, and links to other support and industry
information.
The Whanganui Tertiary Education Venture (WTECV) project launched the new initiative yesterday as a pilot that will also
give up to 200 students at private tertiary-education providers in the region access to UCOL facilities in Wanganui.
The WTECV project is a partnership involving the Tertiary Education Commission, Whanganui UCOL, and other
tertiary-education providers in the region, as well as support agencies such as Career Services, Study Link, and Work
and Income New Zealand.
The initiative offers new opportunities for present and future tertiary-education students in the Wanganui region. Two
state-of-the-art, touch-screen portals will allow access to a new website with information on more than 130 courses
available within the region. As well as links to tertiary-education providers, the portals also provide links to
education-related agencies and industry training organisations as well as a bi-lingual glossary of terms.
Whanganui UCOL principal Suzanne Frecklington says the launch is an important milestone towards creating a shared
learning community that represents the region’s tertiary-education providers. “We’re pleased to open up new
opportunities for students by giving them tools to plan their study paths and their careers. We’re also looking forward
to sharing our wonderful new facilities at Matapihi ki te Ao with a wider student group,” said Ms Frecklington.
World Watch
Upsets expected in research assessment exercise
Major upsets to the existing hierarchy of research excellence can be expected when the results of the research
assessment exercise (RAE) are released in December, according to an academic helping to judge the submissions. Ray Paul,
an emeritus professor at Brunel University, has warned that, when trying to select the best research papers to submit to
the 2008 RAE, some university departments have relied too heavily on the perceived prestige of the journals the research
was published in.
“Such rankings do not equate to research quality,” he said. “The results that come out in December may not be what
[departments] expect. They may have got themselves a whole pile of publications in the top-ranked journals but find
themselves not top-ranked.”
The RAE rules prevent assessment panels from using journal rankings to judge quality, requiring them instead to use peer
review. But Professor Paul said it was “common knowledge” that universities had used ranked lists of journals to decide
which papers to submit to the RAE. He cites the example of an information-systems journal that appears “near the top of
the table” but has a “striking” lack of correspondence with quality.
“The reason for this is that the journal, with a very large circulation, has concentrated on making the research results
clear to its wider and varied readership, and therefore wants little in the paper about the research content. So when
assessing such a paper for research quality, there is hardly any,” he added. “It would appear to me that journal league
tables are to do with promotion procedures and rites of passage in academia, not quality research.”
From Zoë Corbyn in Times Higher Education
Melbourne academics face sack
As many as 20 senior academics could be sacked from Melbourne University’s embattled arts faculty by the end of the year
in a cut designed to save about $NZ2.45 million. The announcement this week by dean of arts, Professor Mark Considine,
is in addition to the 30 voluntary redundancies announced earlier this year and coincides with the release of a staff
workload survey indicating widespread stress.
A draft proposal, released to staff for consultation, outlines the forced redundancy programme, stating that the sacked
staff would have to be out by the end of the year “to assist with reducing staffing costs”. The draft says targeted
staff would be those whose research activity over the past five years failed to meet academic board requirements.
The National Tertiary Education Union (NTEU) said this would include senior lecturers, associate professors, and
professors. NTEU Melbourne University branch president, Ted Clark, said the cut represented a significant proportion of
academic staff, almost 10 percent of the full-time teaching workforce in the arts faculty, and would endanger subject
offerings next year as well as increase the workload for remaining staff at a time when many felt overworked. He added
that the student-staff ratio would swell beyond the current 20 to one following the cuts.
The troubled arts faculty has already weathered the loss of 24 professional and six academic staff by way of voluntary
redundancies this year. That exit followed last year’s departure of 24 academic staff from the faculty, a move that cut
wage costs by $NZ5.26 million to $NZ50.5 million this year. Results of the union survey, conducted in May with more than
800 staff across faculties, show general and academic staff feel they are experiencing excessive workloads and work-life
imbalance.
From Bridie Smith in the Age
Research and tobacco
Earlier this year, the New York Times published an account of a contract between Philip Morris USA, America’s largest
tobacco company, and Virginia Commonwealth University in Richmond. The contract was highly unusual in giving the tobacco
company not only the right to bar publication or discussion of the research but also assigned to the company all patent
and intellectual property rights flowing from the research.
Responding to the report, Dr Rick Solana, Philip Morris’s senior vice-president for research and technology, said that,
once the company concluded that its competitive interests were protected regarding any research results, it could permit
publication. “Once the intellectual property is protected, then it’s usually OK to publish. Something being proprietary
does not mean something cannot be published,” he told the newspaper.
Some anti-smoking organisations maintain that research which the industry pays for is near to worthless, or at best
suspect. Monika Kosińska, secretary-general of the European Public Health Alliance, for instance, says, “Where there’s a
conflict of interest between the organisation producing the products and the health outcome, that research is always
compromised.”
By contrast, the industry argues that the more money engaged in research, the better the health benefits. “Maximising
the funding available for researchers can offer the greatest potential benefits in addressing the health risks
associated with smoking,” says Marija Sepic, manager of external communications at Philip Morris International.
In sharp conflict with that view, the leading US public-health agency, Centers for Disease Control and Prevention (CDC),
argues that opponents of public health (in which they include the tobacco industry) frequently attempt to “manufacture
uncertainty” by questioning the validity of scientific evidence on which health regulations are based. Producers of
tobacco and other hazardous products frequently ridiculed any research that threatened their interests as “junk
science”, said CDC. The agency always has in mind the funding source and other potential biases when considering
research.
From Alan Osborn in University World News
Review recommends full research funding
According to the National Tertiary Education Union, the release of a comprehensive review of Australian innovation,
Venturous Australia, will make a valuable contribution to the debate about the future direction of education and
research and development (R) policy for Australia. “The report highlights the importance of innovation for Australia’s future economic, social, and
cultural prosperity,” said NTEU national president, Dr Carolyn Allport.
“It also shows that the stalling of productivity growth over the last five years has occurred at the same time as levels
of public investment in education and R have been declining. The need for increases in the level of public investment in the education and skills of our
people, human capital, is emphasised as an essential and central component of any policy response that aims to
strengthen Australia’s innovation performance,” she said.
“NTEU strongly supports the recommendations for an immediate increase in funding for research undertaken by our
universities to ensure that grants cover the full costs of undertaking specific research projects, without reducing the
number of projects being supported. The union also applauds the report’s recommendations for greater collaboration in
research and increases in the level of financial support for research students,” Dr Allport continued.
“If our universities are expected to provide a quality education that produces graduates with the capacity to adopt and
adapt new innovations, it is critical that both universities and the proposed funding model encourage greater engagement
of academic staff in research. The report correctly highlights that maximising the benefits from innovation research
requires greater numbers of people to participate in research,” concluded Dr Allport.
Bastardisation or honour?
“I was party to it, willingly, I had a hell of a lot of fun, [but] I totally refute the implication ... that it was an
exercise in bastardisation,” said Dr Ray Roberts, who participated in what has been described as the pitchfork ritual of
the elite applied mathematics department at the Australian National University. His explanation follows reports that
postgraduates unfortunate enough to fall behind in their work in the department were paraded, bent over as a mock poem
was read, and had a pitchfork pointed at their bottoms.
However, not everybody agrees it was fun. Andrew Stewart, an ANU physicist who feels he was pressured into early
retirement in 2005 for denouncing these rituals as bastardisation, has reprised the affair for the Senate inquiry into
academic freedom.
The academic organisers of the pitchfork rituals insisted they were voluntary affairs and vice-chancellor Ian Chubb has
assured Dr Stewart by letter that “no student of the university will be required to participate in ritual-type
activities”. Dr Stewart says that, when one student refused to take part, an email summoning staff in the department to
a pitchforking spoke of “important secret business”. The rituals took place in the early 2000s.
Dr Roberts said his 2004 doctorate was “one of the proudest achievements” of his life. “I was the second recipient of
the pitchfork award and I can assure you I consider it a great honour,” he said. “It was done off-site from the
university, [at] a private person’s residence. I just find it amazing that anybody could consider it bastardisation.
Quite the reverse: all the academic staff were terribly supportive.”
Nigel Palmer, president of the Council of Australian Postgraduate Associations, was bemused by the rituals. “I’d assumed
that these sorts of practices had gone out of fashion with flares,” he said.
From Bernard Lane in the Australian
More international news
More international news can be found on University World News:
http://www.universityworldnews.com
AUS Tertiary Update is published weekly on Thursdays and distributed freely to members of the Association of University
Staff and others. Back issues are available on the AUS website: www.aus.ac.nz. Direct inquiries should be made to the
editor, email: editor@aus.ac.nz