As the use of artificial intelligence (AI) grows, particularly with the introduction of “generative” tools like ChatGPT,
experts are calling for transparency in their use in Aotearoa New Zealand government agencies, especially healthcare.
In the latest Public Health Communication Centre Briefing, Victoria University of Wellington Professor of Artificial
Intelligence, Ali Knott and co-authors examine the latest reports on AI and look at what is needed to ensure its safe
and transparent use as it is increasingly employed across the public sector.
Professor Knott stresses the necessity of evaluations of AI systems to find out how well they work. “We need to know how
often it is right. Does it work as well as a person performing the same task? Does it work equally well for all
demographic groups? Is it fair and equitable?”
He says the public also has a right to know how well healthcare AI systems perform. “Performance information should
provide an understanding of what level of human oversight is required for a given tool, and what expectations about
human involvement there should be.”
Bias is a factor that must be considered when designing and evaluating AI in healthcare according to fellow author, AI
and technology ethicist Dr Karaitiana Taiuru. He says Māori experience significant inequities and bias in health
compared to the non-Māori population. He recommends all new AI teams be diverse and account for the introduction of bias
at each stage of the AI lifecycle. "Addressing questions of performance and bias is crucial for public sector AI
systems, where open government principles demand high transparency," he says.
The Briefing outlines two types of AI systems: "predictive models," which are trained to perform specific tasks, and
"generative models," like ChatGPT, which can be applied to a range of tasks they were not specifically trained for, such
as answering questions or translating. Professor Knott emphasises that research on evaluating generative AI systems is
still evolving, so their use—particularly in healthcare—should be approached with caution.
Along with robust evaluations, Professor Knott says there should be a regularly updated register of all AI algorithms
used by government agencies including reporting on their performance and impact on equity. “As AI technologies continue
to evolve and become more prevalent in healthcare and other public services, the time has come to implement the register
of AI algorithms and ensure rigorous evaluations. This will foster transparency, build public trust, and ensure that AI
systems are safe, reliable, and equitable,” he says.