Thursday April 19 2018
Now is the time to consider a regulatory body on AI predictive analytics
With Immigration New Zealand recently joining ACC and The Ministry for Social Development on the list of Government
departments courting controversy over the use of predictive analytics systems, leading University of Otago researchers
say now is the time to consider a regulatory body to oversee the rising use of artificial intelligence (AI) systems in
New Zealand Government departments.
Members of the University of Otago’s NZ Law Foundation funded Artificial Intelligence and Law in New Zealand Project (AILNZP) have produced an article, Proposed Oversight of Predictive Analytics,published today on the online current affairs and culture blog hub Pundit highlighting concerns being prompted by the rise of computer-based predictive analytic systems.
While the AILNZP members are unanimous in acknowledging there is a place for predictive analytic systems in New Zealand,
they say there should be greater public oversight of the systems used in government, and greater transparency of how
they work.
“At present, public information about predictive analytics in government comes mainly from media ‘exposés’” says
Associate Professor Colin Gavaghan of Otago’s Faculty of Law, and Director of the NZ Law Foundation sponsored Centre for
Law and Policy in Emerging Technologies.
“In the latest case with Immigration New Zealand, even the Immigration Minister himself was in the dark about the
system. These systems can be of great use, but there must be more transparency about how predictive systems are being
used in government.”
Rather than heaping condemnation on the increasing number and use of AI systems, the AILNZP say the key is to provide
greater understanding of how they work.
“Even for those systems we do know about, we only have partial information about how they work, what data they use, and
how accurate they are. We’re told, for instance, that the Immigration NZ tool is ‘just an Excel spreadsheet’. But many
algorithms can be run in Excel: what algorithm is being run in this case? On what data? With what results? And what
margin of error?” adds AILNZP member Associate Professor Alistair Knott, of Otago’s Department of Computer Science.
AILNZP member Professor James Maclaurin, of Otago’s Department of Philosophy, believes there are several factors an
overseeing body needs to consider around the performance of predictive analytic systems in Government departments.
“How precise are particular systems? Are they prone to bias? Can the system explain how it came to decisions, and how
well trained are the humans using them?”
AILNZP member Dr John Zerilli (Philosophy and Law) notes that these questions are particularly pressing now, in light of
the recent scandal surrounding Facebook’s use, and misuse, of personal data.
“There are growing calls—which we fully support—to regulate the use of personal data gathered by social media sites. But
the process of regulating giants like Facebook will be a complex matter involving lengthy international negotiations. In
the meantime, there is no reason why New Zealand should not put its own house in order over the use of these same tools
in its own Government.”
The AILNZP are proposing an agency should be established in New Zealand to oversee the use of predictive analytics by
publicly funded bodies. The AILNZP suggest such an agency would publish a complete list of the predictive tools used by
Government departments, and other public institutions such as ACC. For each system, it would also supply some basic
information about its design; which variables constitute its input and output, and which techniques are used to learn
mapping from inputs to outputs. In addition, it would answer some of the key questions about the performance of the
system, and about its use.
ends