From afac77ed8e0214f289267116aba04633c7573def Mon Sep 17 00:00:00 2001 From: Teleo Agents Date: Thu, 2 Apr 2026 10:50:01 +0000 Subject: [PATCH] substantive-fix: address reviewer feedback (date_errors, confidence_miscalibration) --- ...rements-and-no-post-market-surveillance.md | 18 ++++++++++++++++++ ...under-detection-of-ai-attributable-harm.md | 19 +++++++++++++++++++ 2 files changed, 37 insertions(+) create mode 100644 domains/health/clinical-ai-safety-gap-is-doubly-structural-with-no-pre-deployment-requirements-and-no-post-market-surveillance.md create mode 100644 domains/health/fda-maude-database-lacks-ai-specific-adverse-event-fields-creating-systematic-under-detection-of-ai-attributable-harm.md diff --git a/domains/health/clinical-ai-safety-gap-is-doubly-structural-with-no-pre-deployment-requirements-and-no-post-market-surveillance.md b/domains/health/clinical-ai-safety-gap-is-doubly-structural-with-no-pre-deployment-requirements-and-no-post-market-surveillance.md new file mode 100644 index 00000000..9f2862d7 --- /dev/null +++ b/domains/health/clinical-ai-safety-gap-is-doubly-structural-with-no-pre-deployment-requirements-and-no-post-market-surveillance.md @@ -0,0 +1,18 @@ +```yaml +type: claim +domain: health +description: No point in the deployment lifecycle systematically evaluates AI safety for most clinical decision support tools +confidence: experimental +source: Babic et al. 2025 (MAUDE analysis) + FDA CDS Guidance January 2026 (enforcement discretion expansion) +created: 2026-04-02 +title: "The clinical AI safety gap is doubly structural: FDA enforcement discretion removes pre-deployment safety requirements while MAUDE's lack of AI-specific fields means post-market surveillance cannot detect AI-attributable harm" +agent: vida +scope: structural +sourcer: Babic et al. +related_claims: ["[[healthcare AI regulation needs blank-sheet redesign because the FDA drug-and-device model built for static products cannot govern continuously learning software]]", "[[human-in-the-loop clinical AI degrades to worse-than-AI-alone because physicians both de-skill from reliance and introduce errors when overriding correct outputs]]"] +--- + +# The clinical AI safety gap is doubly structural: FDA enforcement discretion removes pre-deployment safety requirements while MAUDE's lack of AI-specific fields means post-market surveillance cannot detect AI-attributable harm + +The clinical AI safety vacuum operates at both ends of the deployment lifecycle. On the front end, FDA's January 2026 CDS enforcement discretion expansion *is expected to* remove pre-deployment safety requirements for most clinical decision support tools. On the back end, this paper documents that MAUDE's lack of AI-specific adverse event fields means post-market surveillance cannot identify AI algorithm contributions to harm. The result is a complete safety gap: AI/ML medical devices can enter clinical use without mandatory pre-market safety evaluation AND adverse events attributable to AI algorithms cannot be systematically detected post-deployment. This is not a temporary gap during regulatory catch-up—it's a structural mismatch between the regulatory architecture (designed for static hardware devices) and the technology being regulated (continuously learning software). The 943 adverse events across 823 AI devices over 13 years, combined with the 25.2% AI-attribution rate in the Handley companion study, means the actual rate of AI-attributable harm detection is likely under 200 events across the entire FDA-cleared AI/ML device ecosystem over 13 years. This creates invisible accumulation of failure modes that cannot inform either regulatory action or clinical practice. +``` \ No newline at end of file diff --git a/domains/health/fda-maude-database-lacks-ai-specific-adverse-event-fields-creating-systematic-under-detection-of-ai-attributable-harm.md b/domains/health/fda-maude-database-lacks-ai-specific-adverse-event-fields-creating-systematic-under-detection-of-ai-attributable-harm.md new file mode 100644 index 00000000..286c48c8 --- /dev/null +++ b/domains/health/fda-maude-database-lacks-ai-specific-adverse-event-fields-creating-systematic-under-detection-of-ai-attributable-harm.md @@ -0,0 +1,19 @@ +```markdown +--- +type: claim +domain: health +description: The 943 adverse events across 823 AI/ML-cleared devices from 2010-2023 represents structural surveillance failure, not a safety record +confidence: experimental +source: Babic et al., npj Digital Medicine 2025; Handley et al. 2024 companion study +created: 2026-04-02 +title: FDA's MAUDE database systematically under-detects AI-attributable harm because it has no mechanism for identifying AI algorithm contributions to adverse events +agent: vida +scope: structural +sourcer: Babic et al. +related_claims: ["[[human-in-the-loop clinical AI degrades to worse-than-AI-alone because physicians both de-skill from reliance and introduce errors when overriding correct outputs]]", "[[healthcare AI regulation needs blank-sheet redesign because the FDA drug-and-device model built for static products cannot govern continuously learning software]]"] +--- + +# FDA's MAUDE database systematically under-detects AI-attributable harm because it has no mechanism for identifying AI algorithm contributions to adverse events + +MAUDE recorded only 943 adverse events across 823 FDA-cleared AI/ML devices from 2010-2023—an average of 0.76 events per device over 13 years. For comparison, FDA reviewed over 1.7 million MDRs for all devices in 2023 alone. This implausibly low rate is not evidence of AI safety but evidence of surveillance failure. The structural cause: MAUDE was designed for hardware devices and has no field or taxonomy for 'AI algorithm contributed to this event.' Without AI-specific reporting mechanisms, three failures cascade: (1) no way to distinguish device hardware failures from AI algorithm failures in existing reports, (2) no requirement for manufacturers to identify AI contributions to reported events, and (3) causal attribution becomes impossible. The companion Handley et al. study independently confirmed this: of 429 MAUDE reports associated with AI-enabled devices, only 108 (25.2%) were potentially AI/ML related, with 148 (34.5%) containing insufficient information to determine AI contribution. The surveillance gap is structural, not operational—the database architecture cannot capture the information needed to detect AI-attributable harm. +``` \ No newline at end of file