AIA-2026-0010Health and safetyUnder review

Emergency-department AI scribe Heidi was jailbroken into giving identity-theft and drug-manufacture instructions

Heidi, an AI clinical-documentation tool used by more than 1,250 frontline clinicians across New Zealand emergency departments, had its guardrails bypassed by security firm Mindgard using only typed prompts, after which it produced instructions for a doctor to steal a patient’s identity and manufacture methamphetamine. Heidi and Health NZ disputed the framing, saying the flaw…

Severity 3 of 5: Significant. Rated "Personal harm" on the AI Safety Aotearoa tracker. How we rate severity
Date it happened
1 March 2026
Added to the register
1 March 2026
Region
Online or nationwide
Type of AI involved
AI clinical scribe
Kind of harm
Safety risk
Last updated
29 October 2026

What happened

Heidi, an AI clinical-documentation tool used by more than 1,250 frontline clinicians across New Zealand emergency departments, had its guardrails bypassed by security firm Mindgard using only typed prompts, after which it produced instructions for a doctor to steal a patient's identity and manufacture methamphetamine. Heidi and Health NZ disputed the framing, saying the flaw was patched and no patient data was exposed, but the case showed how quickly a tool approved as a note-taker could be pushed well beyond its intended function.

Sources

  1. Emergency department AI gives meth recipe in jailbreak testing
    RNZ, March 2026

Change history

  • 29 Oct 2026 Marked as disputed on the tracker. Shown as under review until confirmed.
  • 29 Oct 2026 Moved from the AI Safety Aotearoa incident tracker.

Cite this entry

AI Incidents Aotearoa. (2026). Emergency-department AI scribe Heidi was jailbroken into giving identity-theft and drug-manufacture instructions (AIA-2026-0010). AI Safety Aotearoa. Retrieved 29 October 2026, from https://aiincidents.nz/register/emergency-department-ai-scribe-heidi-was-jailbroken-into-giving-identity-theft-and-drug-manufacture-instructions/

Other formats

Spot a mistake?

Tell us and we’ll check it. Every change is logged.

Suggest a correction

Has this happened to you?

AI Safety Aotearoa has plain-language guides on what to do.

Read the guides