NCEA and the Digital Age

Handwritten exams for NCEA are on the cusp of becoming a thing of the past

Bernadette Stockman Principal of St Mary's College St Mary's Bay Auckland Photo Hayley Stevenson

Gone will be the days of students sitting nervously in an exam room with their highlighters and four pens in every colour if one breaks down ready to attack an exam.

The New Zealand Qualification’s Authority’s digital transformation journey with National Certificate of Educational Achievement assessments are on the brink of a significant breakthrough.

They are offering schools the option of participating in Digital Pilot examinations for three Level 1 and Level 2 NCEA subjects this year.

Set to be all digitally assessed (online) by the roll out period of 2020 however others have raised questions and issues including if there will be a blanket policy whereby all schools should make the move at the same time.

Nikki Kaye National's Spokesperson for education said the NZQA and the government need to seriously consider a blanket policy because it would be risky and without knowing what the implementation plan is on NZQA is they need to scrutinise it.

In her view, it would be better to phase it in so they could learn from what's happening rather than to going online on a particular date.

Involvement in the Digital Pilot examinations is voluntary.

Schools choose to participate and give students of the selected subjects the option of sitting a digital examination rather than the paper examination.

The printed examination paper is available as a back-up.

Just over 5,100 students nationwide are entered to sit their examinations digitally this year.

Fifty-five schools are planning to participate in Digital Pilot examinations, which are available at NCEA Levels 1 and 2 in English, Media Studies and Classical studies.

A Digital Pilot examination is an examination students sit instead of the equivalent paper examination, and therefore does count towards NCEA.

Digital Trials are digital examinations that do not count towards a student’s NCEA but may provide evidence towards a derived grade.

So yes, students have been using computers in real exams for the last 3 years.

Andrea Gray Deputy Chief Executive Digital Assessment Transformation said the Digital Trials and Pilots provide us with invaluable information and enable schools to participate in digital assessment at a pace they are comfortable with.

“We recognise that not all examinations may be suitable for delivery in a digital format in 2020.

"Not every school will consider themselves ready to fully participate in digital examinations in 2020 too, but the aim is to have digital exams widely available then.

“We are taking measured and considered steps to ensure schools and students have the best possible experience with digital examinations - at present, we are still in a research and development phase.

“It’s critical that we work closely with schools – as we’re doing – in a co-creation approach, and to work with schools of varying characteristics to understand their needs. It’s important to take things in stages,” said Mrs Gray.

From NZQA's monitoring of the results distributions there doesn't appear to be an intention to force all schools to go digital for assessment all at the same time.

Next year NZQA is moving to investigate more closely the schools that aren't making moves in this direction to find out what's stopping them.

The reasons are probably much bigger than just what NZQA is doing in this area.

It might help to get more resources into schools where students are disadvantaged.

Of course there have been glitches, but the big issue is whether there are remedies in place at the time.

NZQA are satisfied that the pilots haven't put students' opportunities to succeed at risk.

Jack Boyle, President of The New Zealand Post Primary Teachers' Association (PPTA) said there are challenges in terms of suitability of rooms, availability of power points to keep computers charged, the capacity of schools' Wi-fi to cope with all those students logging on at the same time, etc.

“It's just one of a whole lot of steps that schools are taking and digital assessment is a subset of a much bigger picture about adapting to a digital age.

PPTA's biggest concerns are around equity of access, both at the whole school level and the disparities of access within a school.

As part and parcel of the 2016 Digital Trials Pilots project, feedback was sought from the students, teachers, markers, and examination centre managers who participated in the trials and pilots.

They have been careful to not proceed with pilots where testing showed that there was a risk to students, e.g. the cancellation of a French pilot in 2016 at a fairly late stage because testing showed problems.

"So yes, PPTA is happy that they’re moving carefully and evaluating how it is working rather than barrelling on regardless.

"The language NZQA is using about the "2020 roll out date" has changed quite a lot in the last year or two so that it is now about all exams "where appropriate" being "available" digitally by 2020, which is very different from the message that schools thought they were hearing in the early stages, i.e. that everyone would have to be having all their students doing all their exams digitally by 2020.

"So yes, PPTA is happy that this is being managed sensibly and allowing for the big diversity in schools' readiness,” said Mr Boyle.

The big challenge for NZQA will be to make sure that students who don't do their assessments digitally aren't disadvantaged, but also that students (who do most or all of their learning using digital devices) aren't disadvantaged by having to turn to pen and paper when it comes to the exams.

Mr Boyle has said that students who don't have access to devices are disadvantaged in a number of ways, but that is a much bigger issue than digital assessment, it's about equity more generally, that some schools and students have access to all sorts of technologies and many don't.

He has said this feedback has been respected and valuable in providing insight into their experience which is already informing the next steps NZQA takes with digital assessment in collaboration with schools and the wider education sector.

Many professionals have their take on the scheme however Derek Handley, New Zealand entrepreneur, speaker and author said this is such an interesting point in relation to the handwriting verses computers in NCEA exams debate.

“ I taught myself to type when I was at high school as I knew it would be the future. Today, if I was at school I can’t imagine not being a totally proficient at speed typing – it just speeds things up about 100 times. But we still all need to learn to write.

“The future, I believe, is actually voice dictation – the technology for which is almost perfect and I use a lot in drafting emails and texts.

“It’s by far the most natural way we communicate (obviously!) it’s also much better for body posture, our hands and arms and other things that suffer from excessive typing.

“We need to crack how this is going to work socially because in a normal environment in today’s layouts (with offices) it’s not possible for people to all be talking to their machines (phones, computers) to dictate their content.

“Typing I think is really bad for the brain. It’s like a stream of consciousness and no thought.

“Writing is super awesome for different types of hand -brain-mind-think connections,” said Mr Handley.

Back to the chalk board, 100 schools and 4386 students took part in at least one of the 2016 NCEA Level 1 digital trials and pilots.

Two hundred and sixty three schools have been involved in NZQA’s digital trial and pilot work in 2015 and 2016.

It appears from surveys and through feedback from the students it’s all ‘thunderbirds a go’ and on the right track and these are positive and indicative findings on which to build future work.

The results of the pilots showed no evidence of disadvantage for those students sitting a digital examination, compared to the paper equivalent.

The opt-in trials and pilots model is working smoothly. Schools are able to participate at a speed they are comfortable with.

They can access their capacity for digital assessment and its relevance, subject by subject, to the way they’re delivering their teaching and learning.

Bernadette Stockman Principal of St Mary's College said that they were doing a strategic piece at the moment on parents’ reaction and what they hope for in the next few years which would give them some feedback at the beginning of next year.

"The opt in scheme is for schools to choose different subject areas that they might trial or pilot differently and different faculties make that decision based on their own criteria. We have been using it for health and in particular, " said Mrs Stockman.

NZQA will continue with a co-creation model for digital assessment wherever feasible.

What this means in the big picture is a moderately big step in education and technology that has been in the pipeline for a while now.

The idea of this has stirred up much debate with academics, professionals in the teaching fraternity, parents and of course students.

While Ms Kaye has had some information from NZQA she has not seen their final evaluation.

She said as she would expect with anything like this that there would be lessons learned as part of that testing process but what the test will be for New Zealand, is that they learn the lessons from that testing.

"That the testing is comprehensive, that people can have confidence in the platform technically but also this issue of the schools being confident that they have the digital environment, not only technically but also from a people. perspective to enable these examinations run effectively for children.

Without knowing specific situations, what is important she said is to really carefully test the different types of physical environments and it doesn't have to be a one size fits all approach but again it's important they're confident of security, connections, confident of the fact that schools may need different types of environment whether it's classrooms or halls to be able to delivery this.

She said the government needs to make sure they're across both the evaluation but also the detail of what needs to happen in terms of the implementation plan.

"From my perspective, I've been heavily involved in ensuring that we shift to a more 21st century education system and we had 'The Economist Magazine' name us as number 1 in the world for preparing children for the future but we have a lot more to do.

"As opposition spokesperson for education I think it shouldn't just be about online examinations, there's lots of mega trends happening in education in 21st century learning from data analytics to this whole issue of people having multiple qualifications and what that looks like.

"Communities of online learning. From my parties perspective we believe in freedom, and lots of other opportunities, so there will be some challenging debates around issues like communities of online learning and the future of assessments and I look forward to leading some of those debates," said Ms Kaye.

This year 15 Level 1 Digital Trial examinations have taken place in New Zealand around the country in different schools.

This year the planned trials were Art History, Business Studies, Classical Studies, Economics, English, French, Geography, Samoan, Health, History, Media Studies, Physics, Spanish, Science, Te Reo Rangatira.

Externally assessed standards will be used in all these subjects. The Trial Examinations will be available at two times – the first opportunity will be 11- 29 September and the second lot will occur 16 – 27 October.

Schools will reign over when they offer the Digital Trials within this period.

The setup, monitoring and marking of the Digital Trials will be undertaken by schools.

The Digital Trials can either be used as a practice examination completed in one sitting, completed over multiple sessions as a classroom activity, or used for revision.

In the three language Trials French, Samoan and Spanish contenders will have full support over audio playback.

This means they will be able to pause, rewind, fast-forward, skip and have unlimited playback of audio.

In 2017 up to 12 schools will be asked to become a Focus Group.

This is to work closely with students to advance the learnings and concepts of how Digital Assessment will operate.

The Focus Group will be introduced to participate in co-managed Trials for Level 1 Classical Studies, Level 1 English and Level 1 Media Studies.

These examinations will be invigilated (supervised) and marked by NZQA.

If the 2017 Digital Trial examinations are accomplished as a practice examination in one sitting, they can be used as evidence for (derived) consequent grades because they’ve been adjusted and improved using materials from combined papers from previous years.

These examinations will need to be kept protected so they’re not accessible to students from other schools who may be sitting them at another time.

The marking of these papers must conform and obey with the derived grade requirement for verification or justification or validation before results are reported.

Digital Pilots will be offered for three Level 1 Digital Pilot examinations. Level 1 Pilots will be available for all candidates in the following subjects: Level 1 Classical Studies, Level 1 English, Level 1 Media Studies.

Level 2 Pilots will be available to students who participated in the Level 1 Pilots in 2016: Level 2 Classical Studies, Level 2 English, Level 2 Media Studies.

TRIALS AND PILOTS TIMELINE FOR NEW ZEALAND SCHOOLS - source NZQA reinvented graph Hayley Stevenson.

Involvement in a pilot examination will be voluntary and those students who opt-in will sit a digital examination rather than the paper examination (the printed examination paper will be available as a back-up).

The pilot examinations will be held on the same dates and times as the paper- based examinations.

There is the aspect which is addressed in an interview with Tony Kane Headmaster of Kapiti College and that is trials of enhanced functionality in digital assessment.

In the 2017 Digital Trials and Pilots project, the digital assessment is like the paper based examination, but is amplified by technology.

Trialling such implements will help NZQA find out what they need to improve on in regard digital assessment.

Some of the tools that students will understand will be txt editing, copy and paste within the digital exam, word count in some subjects, spell check in some subjects.

The following tools are available in the Trials, Videos, Audio playback in the language trials of French, Samoan, Spanish and Te Reo Rangatira.

Special assessment conditions are also important so candidates who need special assessment conditions can participate in both the trial and pilot examinations.

Spellcheck will be available for all candidates in the following subjects. Art History, Business Studies, Classical Studies (trial and pilot) Economics and English (trial and pilot) Geography, Health, History, Media Studies (trial and pilot) check student’s responses as typed.

The candidate will have control over the spellcheck function and will select the passage and/or words they want spellchecked.

The subjects that spellcheck will not be available in the following subjects French, Samoan, Science, Spanish, Physics and Te Reo Rangatira.

Mr Tony Kane, principal of Kapiti College said spellcheck will not automatically check student’s responses as typed, the student will have authority over the spellcheck function and will select the passage and/or words they wish to check.

This is due to the mandatory use of scientific language and the use of languages other than English, candidates will use when responding to questions in these subjects.

Students will not have access to spellcheck to ensure they have the best experience

© Scoop Media