Nothing secret or unofficial about exam marking

12 December 2005

Nothing secret or unofficial about exam marking

The Qualifications Authority Acting Chief Executive Karen Sewell has again emphasised that there is no secret or unofficial exam marking taking place and that only students who reach the standard will be awarded the credits.

"I have been open and transparent about the monitoring and marking processes we have in place this year. The processes are designed to ensure the results are fair and consistent for students.

"It is unfortunate that there seems to be some public confusion about the marking. I don't want this confusion to undermine the excellent and professional work being carried out by the 2000 markers who are now half way through marking 1.9 million papers.

"Markers are working hard under significant time pressure to ensure the results are fair and consistent this year. I have emailed all markers today to thank them for their commitment.

"I also want to run through the details of the marking processes. In the early stages of marking, the marking schedules usually need clarification or changing, and as a result, some papers are always remarked, often more than once. This has always been the case.

"Check marking is also a critical part of the process. I know that the check markers are working hard to make sure that the interpretations markers are making of the marking schedules are consistent. This also always leads to some remarking.

"The difference this year is that when a sufficient percentage of papers has been marked, if the results are significantly different from the profiles of expected performance, we investigate possible causes. If there is an apparent difficulty with interpretation of the marking schedule or the way it is being applied, we call a halt to marking and investigate. In some cases the marking schedule is modified and remarking, usually of only part of a paper, is carried out," says Karen Sewell.

"This is still a standards-based approach for making judgements. To achieve, a student must reach the standard. The profiles are not set in stone. Markers still need to apply their experience and professional judgement to the students' work.

"There is also an assumption that the remarking is focussing on standards where not enough students are achieving. This is not the case. In some instances the marking schedule may not be sufficiently discriminating.

"It is likely that in some instances, the final results will not match the profiles. If that is the professional judgement of markers and panel leaders, then so be it," says Karen Sewell.

To date 13 (or 3.9 percent) of the 335 standards have been remarked after initial benchmarking and check marking. The standards are:

Subject and Standard Number Credit Values
English level 1 – 90054 - 2 credits
English level 1 – 90057 - 3 credits
English level 2 – 90380 - 3 credits
Economics level 1 – 90198 - 5 credits
Maths level 1 – 90152 - 2 credits
Maths level 3 – 90636 - 6 credits
Maths level 3 – 90635 - 6 credits
Japanese level 1 – 90102 - 6 credits
Drama level 3 – 90612 - 4 credits
Classics level 2– 90247 - 5 credits
Accounting level 2– 90224 - 5 credits
English level 3 – 90724 - 3 credits
Te Reo Maori level 2 – 90441 - 6 credits

NOTE: The email to markers follows.

ENDS

12 December 2005

Dear Markers

First, I would like to acknowledge and thank you for the hard work you are doing at what is always a very busy time for teachers. It is not easy to work under significant time pressure and under such public scrutiny.

Fair and consistent results for students depend on the work you are doing and I am very grateful for the high level of professionalism you are demonstrating. If any of you have any issues about the process being followed this year, please ring me.

It is unfortunate that there is some public confusion about the marking process for achievement standards. I have tried to ensure that the public understands the processes we have in place this year so that the results will be fair and consistent. I have been open and transparent about all remarking to the extent of releasing a media statement every couple of days.

I will be putting out another statement today reiterating the process. I will be making the point that in the early stages of marking the assessment schedules almost always need clarification or changing and as a result some papers are remarked, often more than once. This has always been the case.

Check marking is also a critical part of the process. I know that the check markers are working hard to make sure that the interpretations markers are making of the assessment schedules and the way the schedules are being applied, are consistent. This also leads to some remarking.

What is different this year is that when a sufficient percentage of papers has been marked, if the results are significantly different from the profiles, we investigate possible causes. If there is an apparent difficulty with interpretation of the assessment schedule we call a halt to marking and investigate. In some cases the assessment schedule is modified and remarking, usually of only part of a paper, is carried out.

This year we have developed expected profiles of performance for all standards based on past statistics and the professional expertise and advice of examiners, panel leaders, teachers, principals and Qualifications Authority staff. The profiles of expected performance give the median and upper and lower bands for expected numbers of achieved, merit and excellence in each standard. This gives a range that is a guide for expected student performance - not an absolute.

This is still a standards-based approach for making judgements. These ranges of tolerance are not set in stone because you are then asked to apply your experience and professional judgement to the students work.

To apply this process we have new ICT software which allows us to capture and use the information you are feeding into the system daily. This system captures grades, analyses them overnight and provides information to panel leaders and national assessment facilitators. They are then able to modify assessment schedules if it appears early on that the exam or questions have undershot or overshot in terms of the level of difficulty or appropriateness, or that the assessment schedule has not taken account of what is an appropriate student interpretation.

But the fundamental question through the exam marking process remains - does the performance of the student demonstrate achievement of the standard?

It is quite likely that in some instances, the results will be below or above what is expected. If that is the professional judgement of the panel, then so be it. It will then be up to us to explain why student performance was different from the profiles and different from that in previous years.

If you have any concerns and feel they haven't been heard, I urge you to contact me on [number removed by Scoop] or, if you prefer, contact the Manager Assessment (Andrew Kear) or the Group Manager Secondary Education (Kate Colbert).

You can do this, confident that we want to respond to your concerns in a professional and positive manner.

Thank you again for your commitment to ensuring this year's results will be fair and consistent.

With my thanks and best wishes

Karen Sewell

© Scoop Media