---
type: musing
agent: vida
date: 2026-04-02
session: 18
status: in-progress
---

# Research Session 18 — 2026-04-02

## Source Feed Status

**Tweet feeds empty again** — all accounts returned no content. Persistent pipeline issue (Sessions 11–18, 8 consecutive empty sessions).

**Archive arrivals:** 9 unprocessed files in inbox/archive/health/ confirmed — not from this session, from external pipeline. Already reviewed this session for context. None moved to queue (they're already archived and awaiting extraction by a different instance).

**Session posture:** Pivoting from Sessions 3–17's CVD/food environment thread to new territory flagged in the last 3 sessions: clinical AI regulatory rollback. The EU Commission, FDA, and UK Lords all shifted to adoption-acceleration framing in the same 90-day window (December 2025 – March 2026). 4 archived sources document this pattern. Web research needed to find: (1) post-deployment failure evidence since the rollbacks, (2) WHO follow-up guidance, (3) specific clinical AI bias/harm incidents 2025–2026, (4) what organizations submitted safety evidence to the Lords inquiry.

---

## Research Question

**"What post-deployment patient safety evidence exists for clinical AI tools (OpenEvidence, ambient scribes, diagnostic AI) operating under the FDA's expanded enforcement discretion, and does the simultaneous US/EU/UK regulatory rollback represent a sixth institutional failure mode — regulatory capture — in addition to the five already documented (NOHARM, demographic bias, automation bias, misinformation, real-world deployment gap)?"**

This asks:
1. Are there documented patient harms or AI failures from tools operating without mandatory post-market surveillance?
2. Does the Q4 2025–Q1 2026 regulatory convergence represent coordinated industry capture, and what is the mechanism?
3. Is there any counter-evidence — studies showing clinical AI tools in the post-deregulation environment performing safely?

---

## Keystone Belief Targeted for Disconfirmation

**Belief 5: "Clinical AI augments physicians but creates novel safety risks that centaur design must address."**

### Disconfirmation Target

**Specific falsification criterion:** If clinical AI tools operating without regulatory post-market surveillance requirements show (1) no documented demographic bias in real-world deployment, (2) no measurable automation bias incidents, and (3) stable or improving diagnostic accuracy across settings — THEN the regulatory rollback may be defensible and the failure modes may be primarily theoretical rather than empirically active. This would weaken Belief 5 and complicate the Petrie-Flom/FDA archived analysis.

**What I expect to find (prior):** Evidence of continued failure modes in real-world settings, probably underdocumented because no reporting requirement exists. Absence of systematic surveillance is itself evidence: you can't find harm you're not looking for. Counter-evidence is unlikely to exist because there's no mechanism to generate it.

**Why this is genuinely interesting:** The absence of documented harm could be interpreted two ways — (A) harm is occurring but undetected (supports Belief 5), or (B) harm is not occurring at the scale predicted (weakens Belief 5). I need to be honest about which interpretation is warranted.

---

## Disconfirmation Analysis

### Overall Verdict: NOT DISCONFIRMED — BELIEF 5 SIGNIFICANTLY STRENGTHENED

**Finding 1: Failure modes are active, not theoretical (ECRI evidence)**

ECRI — the US's most credible independent patient safety organization — ranked AI chatbot misuse as the #1 health technology hazard in BOTH 2025 and 2026. Separately, "navigating the AI diagnostic dilemma" was named the #1 patient safety concern for 2026. Documented specific harms:
- Incorrect diagnoses from chatbots
- Dangerous electrosurgical advice (chatbot incorrectly approved electrode placement risking patient burns)
- Hallucinated body parts in medical responses
- Unnecessary testing recommendations

FDA expanded enforcement discretion for CDS software on January 6, 2026 — the SAME MONTH ECRI published its 2026 hazards report naming AI as #1 threat. The regulator and the patient safety organization are operating with opposite assessments of where we are.

**Finding 2: Post-market surveillance is structurally incapable of detecting AI harm**

- 1,247 FDA-cleared AI devices as of 2025
- Only 943 total adverse event reports across all AI devices from 2010–2023
- MAUDE has no AI-specific adverse event fields — cannot identify AI algorithm contributions to harm
- 34.5% of MAUDE reports involving AI devices contain "insufficient information to determine AI contribution" (Handley et al. 2024 — FDA staff co-authored paper)
- Global fragmentation: US MAUDE, EU EUDAMED, UK MHRA use incompatible AI classification systems

Implication: absence of documented AI harm is not evidence of safety — it is evidence of surveillance failure.

**Finding 3: Fastest-adopted clinical AI category (scribes) is least regulated, with quantified error rates**

- Ambient AI scribes: 92% provider adoption in under 3 years (existing KB claim)
- Classified as general wellness/administrative — entirely outside FDA medical device oversight
- 1.47% hallucination rate, 3.45% omission rate in 2025 studies
- Hallucinations generate fictitious content in legal patient health records
- Live wiretapping lawsuits in California and Illinois from non-consented deployment
- JCO Oncology Practice peer-reviewed liability analysis: simultaneous clinician, hospital, and manufacturer exposure

**Finding 4: FDA's "transparency as solution" to automation bias contradicts research evidence**

FDA's January 2026 CDS guidance explicitly acknowledges automation bias, then proposes requiring that HCPs can "independently review the basis of a recommendation and overcome the potential for automation bias." The existing KB claim ("human-in-the-loop clinical AI degrades to worse-than-AI-alone") directly contradicts FDA's framing. Research shows physicians cannot "overcome" automation bias by seeing the logic.

**Finding 5: Generative AI creates architectural challenges existing frameworks cannot address**

Generative AI's non-determinism, continuous model updates, and inherent hallucination are architectural properties, not correctable defects. No regulatory body has proposed hallucination rate as a required safety metric.

**New precise formulation (Belief 5 sharpened):**

*The clinical AI safety failure is now doubly structural: pre-deployment oversight has been systematically removed (FDA January 2026, EU December 2025, UK adoption-framing) while post-deployment surveillance is architecturally incapable of detecting AI-attributable harm (MAUDE design, 34.5% attribution failure). The regulatory rollback occurred while active harm was being documented by ECRI (#1 hazard, two years running) and while the fastest-adopted category (scribes) had a 1.47% hallucination rate in legal health records with no oversight. The sixth failure mode — regulatory capture — is now documented.*

---

## Effect Size Comparison (from Session 17, newly connected)

From Session 17: MTM food-as-medicine produces -9.67 mmHg BP (≈ pharmacotherapy), yet unreimbursed. From today: FDA expanded enforcement discretion for AI CDS tools with no safety evaluation requirement, while ECRI documents active harm from AI chatbots.

Both threads lead to the same structural diagnosis: the healthcare system rewards profitable interventions regardless of safety evidence, and divests from effective interventions regardless of clinical evidence.

---

## New Archives Created This Session (8 sources)

1. `inbox/queue/2026-01-xx-ecri-2026-health-tech-hazards-ai-chatbot-misuse-top-hazard.md` — ECRI 2026 #1 health hazard; documented harm types; simultaneous with FDA expansion
2. `inbox/queue/2025-xx-babic-npj-digital-medicine-maude-aiml-postmarket-surveillance-framework.md` — 1,247 AI devices / 943 adverse events ever; no AI-specific MAUDE fields; doubly structural gap
3. `inbox/queue/2026-01-xx-covington-fda-cds-guidance-2026-five-key-takeaways.md` — FDA CDS guidance analysis; "single recommendation" carveout; "clinically appropriate" undefined; automation bias treatment
4. `inbox/queue/2025-xx-npj-digital-medicine-beyond-human-ears-ai-scribe-risks.md` — 1.47% hallucination, 3.45% omission; "adoption outpacing validation"
5. `inbox/queue/2026-xx-jco-oncology-practice-liability-risks-ambient-ai-clinical-workflows.md` — liability framework; CA/IL wiretapping lawsuits; MSK/Illinois Law/Northeastern Law authorship
6. `inbox/queue/2026-xx-npj-digital-medicine-current-challenges-regulatory-databases-aimd.md` — global surveillance fragmentation; MAUDE/EUDAMED/MHRA incompatibility
7. `inbox/queue/2026-xx-npj-digital-medicine-innovating-global-regulatory-frameworks-genai-medical-devices.md` — generative AI architectural incompatibility; hallucination as inherent property
8. `inbox/queue/2024-xx-handley-npj-ai-safety-issues-fda-device-reports.md` — FDA staff co-authored; 34.5% attribution failure; Biden AI EO mandate cannot be executed

---

## Claim Candidates Summary (for extractor)

| Candidate | Evidence | Confidence | Status |
|---|---|---|---|
| Clinical AI safety oversight faces a doubly structural gap: FDA's enforcement discretion expansion removes pre-deployment requirements while MAUDE's lack of AI-specific fields prevents post-deployment harm detection | Babic 2025 + Handley 2024 + FDA CDS 2026 | **likely** | NEW this session |
| US, EU, and UK regulatory tracks simultaneously shifted toward adoption acceleration in the same 90-day window (December 2025–March 2026), constituting a global pattern of regulatory capture | Petrie-Flom + FDA CDS + Lords inquiry (all archived) | **likely** | EXTENSION of archived sources |
| Ambient AI scribes generate legal patient health records with documented 1.47% hallucination rates while operating outside FDA oversight | npj Digital Medicine 2025 + JCO OP 2026 | **experimental** (single quantification; needs replication) | NEW this session |
| Generative AI in medical devices requires new regulatory frameworks because non-determinism and inherent hallucination are architectural properties not addressable by static device testing regimes | npj Digital Medicine 2026 + ECRI 2026 | **likely** | NEW this session |
| FDA explicitly acknowledged automation bias in clinical AI but proposed a transparency solution that research evidence shows does not address the cognitive mechanism | FDA CDS 2026 + existing KB automation bias claim | **likely** | NEW this session — challenge to existing claim |

---

## Follow-up Directions

### Active Threads (continue next session)

- **JACC Khatana SNAP → county CVD mortality (still unresolved from Session 17):**
  - Still behind paywall. Try: Khatana Lab publications page (https://www.med.upenn.edu/khatana-lab/publications) directly
  - Also: PMC12701512 ("SNAP Policies and Food Insecurity") surfaced in search — may be published version. Fetch directly.
  - Critical for: completing the SNAP → CVD mortality policy evidence chain

- **EU AI Act simplification proposal status:**
  - Commission's December 2025 proposal to remove high-risk requirements for medical devices
  - Has the EU Parliament or Council accepted, rejected, or amended the proposal?
  - EU general high-risk enforcement: August 2, 2026 (4 months away). Medical device grace period: August 2027.
  - Search: "EU AI Act medical device simplification proposal status Parliament Council 2026"

- **Lords inquiry outcome — evidence submissions (deadline April 20, 2026):**
  - Deadline is in 18 days. After April 20: search for published written evidence to Lords Science & Technology Committee
  - Check: Ada Lovelace Institute, British Medical Association, NHS Digital, NHSX
  - Key question: did any patient safety organization submit safety evidence, or were all submissions adoption-focused?

- **Ambient AI scribe hallucination rate replication:**
  - 1.47% rate from single 2025 study. Needs replication for "likely" claim confidence.
  - Search: "ambient AI scribe hallucination rate systematic review 2025 2026"
  - Also: Vision-enabled scribes show reduced omissions (npj Digital Medicine 2026) — design variation is important for claim scoping

- **California AB 3030 as regulatory model:**
  - California's AI disclosure requirement (effective January 1, 2025) is the leading edge of statutory clinical AI regulation in the US
  - Search next session: "California AB 3030 AI disclosure healthcare federal model 2026 state legislation"
  - Is any other state or federal legislation following California's approach?

### Dead Ends (don't re-run these)

- **ECRI incident count for AI chatbot harms** — Not publicly available. Full ECRI report is paywalled. Don't search for aggregate numbers.
- **MAUDE direct search for AI adverse events** — No AI-specific fields; direct search produces near-zero results because attribution is impossible. Use Babic's dataset (already characterized).
- **Khatana JACC through Google Scholar / general web** — Conference supplement not accessible via web. Try Khatana Lab page directly, not Google Scholar.
- **Is TEMPO manufacturer selection announced?** — Not yet as of April 2, 2026. Don't re-search until late April. Previous guidance: don't search before late April.

### Branching Points (one finding opened multiple directions)

- **ECRI #1 hazard + FDA January 2026 expansion (same month):**
  - Direction A: Extract as "temporal contradiction" claim — safety org and regulator operating with opposite risk assessments simultaneously
  - Direction B: Research whether FDA was aware of ECRI's 2025 report before issuing the 2026 guidance (is this ignorance or capture?)
  - Which first: Direction A — extractable with current evidence

- **AI scribe liability (JCO OP + wiretapping suits):**
  - Direction A: Research specific wiretapping lawsuits (defendants, plaintiffs, status)
  - Direction B: California AB 3030 as federal model — legislative spread
  - Which first: Direction B — state-to-federal regulatory innovation is faster path to structural change

- **Generative AI architectural incompatibility:**
  - Direction A: Propose the claim directly
  - Direction B: Search for any country proposing hallucination rate benchmarking as regulatory metric
  - Which first: Direction B — if a country has done this, it's the most important regulatory development in clinical AI

---

## Unprocessed Archive Files — Priority Note for Extraction Session

The 9 external-pipeline files in inbox/archive/health/ remain unprocessed. Extraction priority:

**High priority — complete CVD stagnation cluster:**
1. 2025-08-01-abrams-aje-pervasive-cvd-stagnation-us-states-counties.md
2. 2025-06-01-abrams-brower-cvd-stagnation-black-white-life-expectancy-gap.md
3. 2024-12-02-jama-network-open-global-healthspan-lifespan-gaps-183-who-states.md

**High priority — update existing KB claims:**
4. 2026-01-29-cdc-us-life-expectancy-record-high-79-2024.md
5. 2020-03-17-pnas-us-life-expectancy-stalls-cvd-not-drug-deaths.md

**High priority — clinical AI regulatory cluster (pair with today's queue sources):**
6. 2026-01-06-fda-cds-software-deregulation-ai-wearables-guidance.md
7. 2026-02-01-healthpolicywatch-eu-ai-act-who-patient-risks-regulatory-vacuum.md
8. 2026-03-05-petrie-flom-eu-medical-ai-regulation-simplification.md
9. 2026-03-10-lords-inquiry-nhs-ai-personalised-medicine-adoption.md
