---
type: source
title: "Beyond Human Ears: Navigating the Uncharted Risks of AI Scribes in Clinical Practice"
author: "npj Digital Medicine (Springer Nature)"
url: https://www.nature.com/articles/s41746-025-01895-6
date: 2025-01-01
domain: health
secondary_domains: [ai-alignment]
format: journal-article
status: unprocessed
priority: high
tags: [ambient-AI-scribe, clinical-AI, hallucination, omission, patient-safety, documentation, belief-5, adoption-risk]
---

## Content

Published in *npj Digital Medicine* (2025). Commentary/analysis paper examining real-world risks of ambient AI documentation scribes — a category showing the fastest adoption of any clinical AI tool (92% provider adoption in under 3 years per existing KB claim).

**Documented AI scribe failure modes:**
1. **Hallucinations** — fabricated content: documenting examinations that never occurred, creating nonexistent diagnoses, inserting fictitious clinical information
2. **Omissions** — critical information discussed during encounters absent from generated note
3. **Incorrect documentation** — wrong medication names or doses

**Quantified failure rates from a 2025 study cited in adjacent research:**
- 1.47% hallucination rate
- 3.45% omission rate

**Clinical significance note from authors:** Even studies reporting relatively low hallucination rates (1–3%) acknowledge that in healthcare, even small error percentages have profound patient safety implications. At 40% US physician adoption with millions of clinical encounters daily, a 1.47% hallucination rate produces enormous absolute harm volume.

**Core concern from authors:**
"Adoption is outpacing validation and oversight, and without greater scrutiny, the rush to deploy AI scribes may compromise patient safety, clinical integrity, and provider autonomy."

**Historical harm cases from earlier speech recognition (predictive of AI scribe failure modes):**
- "No vascular flow" → "normal vascular flow" transcription error → unnecessary procedure performed
- Tumor location confusion → surgery on wrong site

**Related liability dimension (from JCO Oncology Practice, 2026):**
If a physician signs off on an AI-generated note with a hallucinated diagnosis or medication error without adequate review, the provider bears malpractice exposure. Recent California/Illinois lawsuits allege health systems used ambient scribing without patient consent — potential wiretapping statute violations.

**Regulatory status:** Ambient AI scribes are classified by FDA as general wellness products or administrative tools — NOT as clinical decision support requiring oversight under the 2026 CDS Guidance. They operate in a complete regulatory void: not medical devices, not regulated software.

**California AB 3030** (effective January 1, 2025): Requires healthcare providers using generative AI to include disclaimers in patient communications and provide instructions for contacting a human provider. First US statutory regulation specifically addressing clinical generative AI.

**Vision-enabled scribes (counterpoint, also npj Digital Medicine 2026):**
A companion paper found that vision-enabled AI scribes (with camera input) reduce omissions compared to audio-only scribes — suggesting the failure modes are addressable with design changes, not fundamental to the architecture.

## Agent Notes

**Why this matters:** Ambient scribes are the fastest-adopted clinical AI tool category (92% in under 3 years). They operate outside FDA oversight (not medical devices). They document patient encounters, generate medication orders, and create the legal health record. A 1.47% hallucination rate in legal health records at 40% physician penetration is not a minor error — it is systematic record corruption at scale with no detection mechanism.

**What surprised me:** The legal record dimension. An AI hallucination in a clinical note is not just a diagnostic error — it becomes the legal patient record. If a hallucinated diagnosis persists in a chart, it affects all subsequent care and creates downstream liability chains that extend years after the initial error.

**What I expected but didn't find:** Any RCT evidence on whether physician review of AI scribe output actually catches hallucinations at an adequate rate. The automation bias literature (already in KB) predicts that time-pressured clinicians will sign off on AI-generated notes without detecting errors — the same phenomenon documented for AI diagnostic override. No paper found specifically on hallucination detection rates by reviewing physicians.

**KB connections:**
- "AI scribes reached 92% provider adoption in under 3 years" (KB claim) — now we know what that adoption trajectory carried
- Belief 5 (clinical AI novel safety risks) — scribes are the fastest-adopted, least-regulated AI category
- "human-in-the-loop clinical AI degrades to worse-than-AI-alone" (KB claim) — automation bias with scribe review is the mechanism
- FDA CDS Guidance (archived this session) — scribes explicitly outside the guidance scope (administrative classification)
- ECRI 2026 hazards (archived this session) — scribes documented as harm vector alongside chatbots

**Extraction hints:**
1. "Ambient AI scribes operate outside FDA regulatory oversight while generating legal patient health records — creating a systematic documentation hallucination risk at scale with no reporting mechanism and a 1.47% fabrication rate in existing studies"
2. "AI scribe adoption outpacing validation — 92% provider adoption precedes systematic safety evaluation, inverting the normal product safety cycle"

**Context:** This is a peer-reviewed commentary in npj Digital Medicine, one of the top digital health journals. The 1.47%/3.45% figures come from cited primary research (not the paper itself). The paper was noticed by ECRI, whose 2026 report specifically flags AI documentation tools as a harm category. This convergence across academic and patient safety organizations on the same failure modes is the key signal.

## Curator Notes

PRIMARY CONNECTION: "AI scribes reached 92% provider adoption in under 3 years" (KB claim); Belief 5 clinical AI safety risks
WHY ARCHIVED: Documents specific failure modes (hallucination rates, omission rates) for the fastest-adopted clinical AI category — which operates entirely outside regulatory oversight. Completes the picture of the safety vacuum: fastest deployment, no oversight, quantified error rates, no surveillance.
EXTRACTION HINT: New claim candidate: "Ambient AI scribes generate legal patient health records with documented 1.47% hallucination rates while operating outside FDA oversight, creating systematic record corruption at scale with no detection or reporting mechanism."