The register
Every known AI-related Incident in New Zealand.
-
Severity 5 of 5, Severe
Auckland man died after a sextortion scam that used an AI-generated woman
A coroner found that a 41-year-old Auckland man took his own life in 2024 after being blackmailed by scammers. He had sent intimate images to what he believed was a…
-
Severity 4 of 5, Serious
Deloitte’s $435,000 Wellington City Council staffing report found to contain AI-generated errors
Wellington Mayor Andrew Little said “large chunks” of a $435,000 Deloitte staffing review were written by AI, after the report was found to contain significant errors including double-counted roles, employment…
-
Severity 2 of 5, Moderate
Tenancy Tribunal swamped by long AI-generated claims, prompting new guidance
The Tenancy Tribunal issued guidelines on AI use after a wave of AI-generated applications, some hundreds of pages long and seeking large sums. In one case a tenant filed over…
-
Severity 4 of 5, Serious
Ministry of Education’s AI marking tool “SMART” scored student writing by word count
NZEI Te Riu Roa urged parents to check whether their school uses SMART, an Australian-sourced AI assessment tool adopted by the Ministry of Education, after teachers reported it assigned identical…
-
Severity 2 of 5, Moderate
Supreme Court ordered costs against an applicant who relied on AI-hallucinated cases (Chen v Goodmore Investments)
The Supreme Court dismissed a self-represented applicant’s leave applications, which relied on authorities it identified as hallucinations of a generative AI application, and ordered costs against her.
-
Severity 3 of 5, Significant
Emergency-department AI scribe Heidi was jailbroken into giving identity-theft and drug-manufacture instructions
Heidi, an AI clinical-documentation tool used by more than 1,250 frontline clinicians across New Zealand emergency departments, had its guardrails bypassed by security firm Mindgard using only typed prompts, after…
-
Severity 2 of 5, Moderate
Employment Relations Authority increased costs after a party kept citing non-existent cases (O’Driscoll v Rehab Co)
The Employment Relations Authority awarded increased costs against a self-represented party, partly because they relied on cases that did not exist and kept doing so after being pointed to the…
-
Severity 4 of 5, Serious
Supreme Court warned litigants of contempt charges over AI-generated fake authorities
Declining leave to appeal in Jones v Family Court at Whangārei, the Supreme Court noted that the applicant had cited a number of authorities that appeared to have been hallucinated…
-
Severity 1 of 5, Minor
High Court noted apparent AI use in submissions that misquoted the law (Coulson v ASB Bank)
Declining a self-represented borrower’s bid to stop ASB Bank selling a property, the High Court noted the submissions suggested significant use of AI and misquoted the law.
-
Severity 2 of 5, Moderate
Litigant cited hallucinated cases and uploaded the other side’s confidential information to ChatGPT (QTR v BXD)
The Employment Relations Authority found a self-represented applicant had used ChatGPT to write submissions citing cases that did not exist, and had uploaded the respondent’s confidential information into the tool.…