pipeline: clean 3 stale queue duplicates
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
This commit is contained in:
Teleo Agents
parent
bc092fd100
commit
76566cb151
3 changed files with 0 additions and 226 deletions
|
|
@ -1,74 +0,0 @@
|
|||
---
|
||||
type: source
|
||||
title: "LLMs Propagate Medical Misinformation 32% of the Time — 47% in Clinical Note Format (Lancet Digital Health, February 2026)"
|
||||
author: "Eyal Klang et al., Icahn School of Medicine at Mount Sinai"
|
||||
url: https://www.thelancet.com/journals/landig/article/PIIS2589-7500(25)00131-1/fulltext
|
||||
date: 2026-02-10
|
||||
domain: health
|
||||
secondary_domains: [ai-alignment]
|
||||
format: research paper
|
||||
status: enrichment
|
||||
priority: high
|
||||
tags: [clinical-ai-safety, llm-misinformation, automation-bias, openevidence, lancet, mount-sinai, medical-language, clinical-note, belief-5]
|
||||
processed_by: vida
|
||||
processed_date: 2026-03-23
|
||||
enrichments_applied: ["human-in-the-loop clinical AI degrades to worse-than-AI-alone because physicians both de-skill from reliance and introduce errors when overriding correct outputs.md"]
|
||||
extraction_model: "anthropic/claude-sonnet-4.5"
|
||||
---
|
||||
|
||||
## Content
|
||||
|
||||
Published in The Lancet Digital Health, February 2026. Lead author: Eyal Klang, Icahn School of Medicine at Mount Sinai. Title: "Mapping the susceptibility of large language models to medical misinformation across clinical notes and social media: a cross-sectional benchmarking analysis."
|
||||
|
||||
**Study design:**
|
||||
- Cross-sectional benchmarking analysis
|
||||
- 1M+ prompts tested across leading language models
|
||||
- Two settings: (1) misinformation embedded in social media format, (2) misinformation embedded in clinical notes/hospital discharge summaries
|
||||
- Compared propagation rates across model tiers (smaller/less advanced vs. frontier models)
|
||||
|
||||
**Key findings:**
|
||||
- **Average misinformation propagation: 32%** across all models tested
|
||||
- **Clinical note/hospital discharge summary format: 47% propagation** — confident, professional medical language triggers substantially higher belief in false claims
|
||||
- Smaller or less advanced models: >60% propagation rate
|
||||
- ChatGPT-4o: ~10% propagation rate (best performer)
|
||||
- Mechanism: "AI systems treat confident medical language as true by default, even when it's clearly wrong" (Klang, co-senior author)
|
||||
|
||||
**Key quote:** "Our findings show that current AI systems can treat confident medical language as true by default, even when it's clearly wrong."
|
||||
|
||||
**Context:**
|
||||
- Covered by Euronews Health, February 10, 2026
|
||||
- Mount Sinai press release: "Can Medical AI Lie? Large Study Maps How LLMs Handle Health Misinformation"
|
||||
- Related companion editorial in Lancet Digital Health (same issue): "Large language models need immunisation to protect against misinformation" (PIIS2589-7500(25)00160-8)
|
||||
|
||||
## Agent Notes
|
||||
|
||||
**Why this matters:** This is the FOURTH clinical AI safety failure mode documented across 11 sessions, distinct from (1) omission errors (NOHARM: 76.6%), (2) sociodemographic bias (Nature Medicine), and (3) automation bias (NCT06963957). Medical misinformation propagation is particularly insidious for OE specifically: OE's use case is synthesizing medical literature in response to clinical queries. If a physician's query contains a false clinical assumption (stated in confident medical language — typical clinical language is confident by convention), OE may accept the false premise and build its synthesis around it, then confirm the physician's existing plan. Combined with the NOHARM omission finding: physician's query → OE accepts false premise → OE confirms plan WITH the false premise embedded → physician's confidence in the (false) plan increases. This is the reinforcement-as-amplification mechanism operating through a different input pathway than demographic bias.
|
||||
|
||||
**What surprised me:** The 47% propagation rate in clinical-note format vs. 32% average is a substantial gap. Clinical language is the format of OE queries. The most concerning failure mode operates in exactly the format most relevant to OE's use case.
|
||||
|
||||
**What I expected but didn't find:** No model-specific breakdown beyond the ChatGPT-4o vs. "smaller models" comparison. Knowing WHERE OE's model sits in this propagation-rate spectrum would be high value — but OE's architecture is undisclosed.
|
||||
|
||||
**KB connections:**
|
||||
- Fourth failure mode for Belief 5 (clinical AI safety) failure catalogue
|
||||
- Combines with NOHARM (omission errors), Nature Medicine (demographic bias), NCT06963957 (automation bias) to define a comprehensive failure mode set
|
||||
- Connects to OE "reinforces plans" PMC finding (PMC12033599): the three-layer failure scenario (physician query with false premise → OE propagates → OE confirms → omission left in place)
|
||||
- Cross-domain: connects to Theseus's alignment work on misinformation propagation in AI systems
|
||||
|
||||
**Extraction hints:** Primary claim: LLMs propagate medical misinformation at clinically dangerous rates (32% average, 47% in clinical language). Secondary claim: the clinical-note format amplification effect makes this failure mode specifically relevant to point-of-care clinical AI tools. Confidence should be "likely" for the domain application claim (connection to OE is inference) and "proven" for the empirical rate finding (1M+ prompts, published in Lancet Digital Health).
|
||||
|
||||
**Context:** Mount Sinai's Klang group is the same group that produced the orchestrated multi-agent AI paper (npj Health Systems, March 2026). They are the most prolific clinical AI safety research group in 2025-2026, producing the NOHARM framework, the misinformation study, and the multi-agent efficiency study in rapid succession.
|
||||
|
||||
## Curator Notes (structured handoff for extractor)
|
||||
PRIMARY CONNECTION: "human-in-the-loop clinical AI degrades to worse-than-AI-alone because physicians both de-skill from reliance and introduce errors when overriding correct outputs" — the misinformation propagation finding adds a new upstream failure to this chain
|
||||
WHY ARCHIVED: Fourth clinical AI safety failure mode; high KB value as distinct mechanism from the three already documented; the clinical-note format specificity directly implicates OE's use case
|
||||
EXTRACTION HINT: Extract as a new claim about LLM misinformation propagation specifically in clinical contexts. Note the 47% clinical-language amplification as the mechanism that makes this relevant to clinical AI tools (not just general AI assistants). Create a wiki link to the OE "reinforces plans" finding (PMC12033599) — the combination defines a three-layer failure scenario.
|
||||
|
||||
|
||||
## Key Facts
|
||||
- Study tested 1M+ prompts across leading language models
|
||||
- ChatGPT-4o achieved ~10% misinformation propagation rate (best performer)
|
||||
- Smaller/less advanced models showed >60% propagation rates
|
||||
- Study published in The Lancet Digital Health, February 2026
|
||||
- Companion editorial titled 'Large language models need immunisation to protect against misinformation' (PIIS2589-7500(25)00160-8)
|
||||
- Lead author: Eyal Klang, Icahn School of Medicine at Mount Sinai
|
||||
- Mount Sinai's Klang group also produced the NOHARM framework and the orchestrated multi-agent AI paper (npj Health Systems, March 2026)
|
||||
|
|
@ -1,73 +0,0 @@
|
|||
---
|
||||
type: source
|
||||
title: "NHS England DTAC Version 2 — Mandatory Clinical Safety and Data Protection Standards for Digital Health Tools, Deadline April 6, 2026"
|
||||
author: "NHS England"
|
||||
url: https://hitconsultant.net/2026/01/06/securing-agentic-ai-in-the-2026-healthcare-landscape/
|
||||
date: 2026-02-24
|
||||
domain: health
|
||||
secondary_domains: [ai-alignment]
|
||||
format: regulatory document
|
||||
status: null-result
|
||||
priority: medium
|
||||
tags: [nhs, dtac, regulatory, clinical-ai-safety, digital-health-standards, uk, mandatory-compliance, belief-3, belief-5]
|
||||
processed_by: vida
|
||||
processed_date: 2026-03-23
|
||||
extraction_model: "anthropic/claude-sonnet-4.5"
|
||||
extraction_notes: "LLM returned 2 claims, 2 rejected by validator"
|
||||
---
|
||||
|
||||
## Content
|
||||
|
||||
NHS England published Version 2 of the Digital Technology Assessment Criteria (DTAC) on February 24, 2026. DTAC V2 establishes mandatory clinical safety and data protection standards for digital health tools deployed in NHS settings.
|
||||
|
||||
**Key compliance requirement:**
|
||||
- All digital health tools used in NHS clinical workflows must meet DTAC V2 standards by **April 6, 2026**
|
||||
- This is a mandatory compliance deadline, not a voluntary standard
|
||||
- Covers: clinical safety, data protection, interoperability, usability
|
||||
|
||||
**Context within the 2026 regulatory landscape:**
|
||||
- NIST AI Agent Standards Initiative (announced February 2026): agent identity, authorization, security as priority areas for standardization — but NO healthcare-specific guidance yet
|
||||
- EU AI Act Annex III: healthcare AI high-risk classification, mandatory obligations August 2, 2026 (separate archive: 2026-08-02-eu-ai-act-healthcare-high-risk-obligations.md)
|
||||
- Coalition for Health AI: advancing safety assessment methods with growing guidelines sets
|
||||
|
||||
**What DTAC V2 covers (general scope from context):**
|
||||
- Clinical safety assessment for digital health products
|
||||
- Data protection compliance (GDPR in UK context)
|
||||
- Interoperability standards
|
||||
- Usability requirements for NHS deployment
|
||||
|
||||
**Implication for clinical AI tools like OE:**
|
||||
- If OE is used in NHS hospital or GP settings (UK has strong clinical AI adoption), DTAC V2 compliance is mandatory by April 6, 2026 (NOW, two weeks from the date of this session)
|
||||
- DTAC V2's clinical safety assessment process would require documenting safety validation for OE's recommendations
|
||||
- Any UK health system that deploys OE without DTAC V2 compliance is out of regulatory compliance
|
||||
|
||||
## Agent Notes
|
||||
|
||||
**Why this matters:** NHS DTAC V2 is the UK parallel to the EU AI Act — a mandatory regulatory standard that requires clinical safety demonstration for digital health tools. The April 6, 2026 deadline is happening NOW (two weeks from this session). If OE is deployed in NHS settings, compliance is required immediately. Unlike the EU AI Act (August 2026 deadline, international obligation), NHS DTAC V2 is already in effect with a deadline that is arriving in days.
|
||||
|
||||
**What surprised me:** The very short time between publication (February 24) and deadline (April 6) — 41 days — is aggressive. This suggests NHS England has been warning about DTAC V2 requirements for some time and the publication was the final version of something already signaled. Any digital health company operating in NHS settings should have been aware this was coming.
|
||||
|
||||
**What I expected but didn't find:** OE-specific DTAC V2 compliance announcement or NHS deployment status. OE's press releases focus on US health systems. Whether OE is used in NHS settings is unknown from public information, but the UK is a major clinical AI market and NHS deployment would trigger DTAC requirements.
|
||||
|
||||
**KB connections:**
|
||||
- Companion to EU AI Act archive (2026-08-02-eu-ai-act-healthcare-high-risk-obligations.md): together these define the regulatory track that is arriving to close the commercial-research gap in clinical AI safety
|
||||
- Relevant to Belief 3 (structural misalignment): regulatory mandate as a correction mechanism when market incentives fail — same pattern as VBC payment reform requiring CMS policy action rather than organic market transition
|
||||
- Relevant to Belief 5 (clinical AI safety): DTAC's clinical safety assessment requirement would mandate the kind of safety validation that OE has not produced voluntarily
|
||||
|
||||
**Extraction hints:** Extract as a factual regulatory claim about NHS DTAC V2: mandatory clinical safety standards for NHS digital health tools, deadline April 6, 2026. Confidence: proven (regulatory fact). Secondary claim: the combination of NHS DTAC V2 (April 2026) and EU AI Act (August 2026) constitutes the first mandatory regulatory framework requiring clinical AI tools to demonstrate safety — creating external pressure that has not been produced by market forces. Confidence: likely (the regulatory facts are proven; the characterization as "first mandatory framework" requires checking for earlier analogous US regulations, which are less clear on clinical AI specifically).
|
||||
|
||||
**Context:** DTAC has been a voluntary standard in prior versions. V2 making it mandatory for NHS deployments is the significant change. The scope is broader than just AI — it covers all digital health tools — but AI tools are now the primary new entrant in NHS digital health, making this primarily relevant to clinical AI deployment.
|
||||
|
||||
## Curator Notes (structured handoff for extractor)
|
||||
PRIMARY CONNECTION: EU AI Act high-risk healthcare AI obligations — DTAC V2 is the UK parallel creating mandatory clinical safety assessment
|
||||
WHY ARCHIVED: First mandatory UK clinical safety standard applying to digital health tools; companion to EU AI Act creating a 2026 regulatory wave that could force clinical AI safety disclosure
|
||||
EXTRACTION HINT: Extract alongside the EU AI Act archive. Frame together as the "2026 regulatory wave": NHS DTAC V2 (April) and EU AI Act (August) represent the first regulatory framework requiring clinical AI safety demonstration in major markets. This is the structural mechanism that could force OE model transparency. Confidence for the regulatory facts: proven. Confidence for OE-specific implications: experimental (depends on whether OE is deployed in NHS settings).
|
||||
|
||||
|
||||
## Key Facts
|
||||
- NHS England published DTAC Version 2 on February 24, 2026
|
||||
- DTAC V2 compliance deadline is April 6, 2026 (41 days after publication)
|
||||
- DTAC V2 covers clinical safety, data protection, interoperability, and usability for digital health tools in NHS settings
|
||||
- EU AI Act Annex III classifies healthcare AI as high-risk with mandatory obligations effective August 2, 2026
|
||||
- NIST announced AI Agent Standards Initiative in February 2026 focusing on agent identity, authorization, and security but without healthcare-specific guidance
|
||||
- Previous DTAC versions were voluntary standards; V2 makes compliance mandatory for NHS deployment
|
||||
|
|
@ -1,79 +0,0 @@
|
|||
---
|
||||
type: source
|
||||
title: "NCT07328815: Ensemble-LLM Confidence Signals as Behavioral Nudge to Mitigate Physician Automation Bias (RCT, Registered 2026)"
|
||||
author: "Follow-on research group to NCT06963957 (Pakistan MBBS physician cohort)"
|
||||
url: https://clinicaltrials.gov/study/NCT07328815
|
||||
date: 2026-03-15
|
||||
domain: health
|
||||
secondary_domains: [ai-alignment]
|
||||
format: research paper
|
||||
status: enrichment
|
||||
priority: medium
|
||||
tags: [automation-bias, behavioral-nudge, ensemble-llm, clinical-ai-safety, system-2-thinking, multi-agent-ui, centaur-model, belief-5, nct07328815]
|
||||
processed_by: vida
|
||||
processed_date: 2026-03-23
|
||||
enrichments_applied: ["human-in-the-loop clinical AI degrades to worse-than-AI-alone because physicians both de-skill from reliance and introduce errors when overriding correct outputs.md"]
|
||||
extraction_model: "anthropic/claude-sonnet-4.5"
|
||||
---
|
||||
|
||||
## Content
|
||||
|
||||
Registered at ClinicalTrials.gov as NCT07328815: "Mitigating Automation Bias in Physician-LLM Diagnostic Reasoning Using Behavioral Nudges." This is the direct follow-on to NCT06963957 (the automation bias RCT archived March 22, 2026).
|
||||
|
||||
**Study design:**
|
||||
- Single-blind, randomized controlled trial, two parallel arms (1:1)
|
||||
- Target sample: 50 physicians (25/arm)
|
||||
- Population: Medical doctors (MBBS) — same cohort as NCT06963957
|
||||
|
||||
**Intervention — dual-mechanism behavioral nudge:**
|
||||
1. **Anchoring cue:** Before evaluation begins, participants are shown ChatGPT's average diagnostic reasoning accuracy on standard medical datasets — establishing realistic performance expectations and anchoring System 2 engagement
|
||||
2. **Selective attention cue:** Color-coded confidence signals generated for each AI recommendation
|
||||
|
||||
**Confidence signal generation (the novel multi-agent element):**
|
||||
- Three independent LLMs each provide confidence ratings for every AI recommendation: Claude Sonnet 4.5, Gemini 2.5 Pro Thinking, and GPT-5.1
|
||||
- Mean confidence across three models determines the signal color (presumably red/yellow/green or equivalent)
|
||||
- When models DISAGREE on confidence (ensemble spread is high), the signal flags uncertainty
|
||||
- This is a form of multi-agent architecture used as a UI layer safety tool, not as a clinical reasoning tool
|
||||
|
||||
**Primary outcome:**
|
||||
- Whether the dual-mechanism nudge reduces physicians' uncritical acceptance of incorrect LLM recommendations (automation bias)
|
||||
- Secondary: whether anchoring + color signal together outperform either mechanism alone
|
||||
|
||||
**Related documents:**
|
||||
- Protocol/SAP available at: cdn.clinicaltrials.gov/large-docs/15/NCT07328815/Prot_SAP_000.pdf
|
||||
- Parent study: NCT06963957 (archived queue: 2026-03-22-automation-bias-rct-ai-trained-physicians.md)
|
||||
- Arxiv preprint on evidence-based nudges in biomedical context: 2602.10345
|
||||
|
||||
**Current status:** Registered but results not yet published (as of March 2026). Study appears to be recently registered or currently enrolling.
|
||||
|
||||
## Agent Notes
|
||||
|
||||
**Why this matters:** This is the first operationalized solution to the physician automation bias problem that is being tested in an RCT framework. The parent study (NCT06963957) showed that even 20-hour AI-literacy training fails to prevent automation bias — this trial tests whether a UI-layer intervention (behavioral nudge) can succeed where training failed. The ensemble-LLM confidence signal is a creative design: it doesn't require the physician to know anything about the underlying model; it uses model disagreement as an automatic uncertainty flag. This is a novel application of multi-agent architecture — not for better clinical reasoning (NOHARM's use case) but for better physician reasoning about clinical AI.
|
||||
|
||||
**What surprised me:** The specific models used (Claude Sonnet 4.5, Gemini 2.5 Pro Thinking, GPT-5.1) include three frontier models from three different companies. The design implicitly assumes these models' confidence ratings are correlated enough with accuracy to be informative — if the models all confidently give the same wrong answer, the signal would fail. This is a real limitation: ensemble overconfidence is a known failure mode of multiple models trained on similar data.
|
||||
|
||||
**What I expected but didn't find:** No published results yet. The trial is likely in data collection or analysis. Results would answer the most important open question in automation bias research: can a lightweight UI intervention do what 20 hours of training cannot?
|
||||
|
||||
**KB connections:**
|
||||
- Direct extension of NCT06963957 (parent study): the automation bias RCT → nudge mitigation trial
|
||||
- Connects to Belief 5 (clinical AI safety): the centaur model problem requires structural solutions; this trial is testing whether UI design is a viable structural solution
|
||||
- The ensemble-LLM signal design connects to the Mount Sinai multi-agent architecture paper (npj Health Systems, March 2026) — both are using multi-model approaches but for different purposes
|
||||
- Cross-domain: connects to Theseus's alignment work on human oversight mechanisms — this is a domain-specific test of whether UI design can maintain meaningful human oversight
|
||||
|
||||
**Extraction hints:** Primary claim: the first RCT of a UI-layer behavioral nudge to reduce physician automation bias in LLM-assisted diagnosis uses an ensemble of three frontier LLMs to generate color-coded confidence signals — operationalizing multi-agent architecture as a safety tool rather than a clinical reasoning tool. This is "experimental" confidence (trial registered, results unpublished). Note the parent study (NCT06963957) as context — the clinical rationale for this trial is established.
|
||||
|
||||
**Context:** This trial is being conducted by researchers who studied automation bias in AI-trained physicians. The 50-participant sample is small; generalizability will be limited even if the nudge shows a significant effect. The trial design is methodologically novel enough to generate high-citation follow-on work regardless of outcome. If the nudge works, it provides a deployable solution. If it fails, it suggests the problem requires architectural (not UI) solutions — which points back to NOHARM's multi-agent recommendation.
|
||||
|
||||
## Curator Notes (structured handoff for extractor)
|
||||
PRIMARY CONNECTION: "erroneous LLM recommendations significantly degrade diagnostic accuracy even in AI-trained physicians" (parent study finding) — this trial is testing the UI solution
|
||||
WHY ARCHIVED: First concrete solution attempt for physician automation bias; the ensemble-LLM confidence signal is a novel multi-agent safety design; results (expected 2026) will be highest-value near-term KB update for Belief 5
|
||||
EXTRACTION HINT: Extract as "experimental" confidence claim about the nudge intervention design. Don't claim efficacy (unpublished). Focus on the design's novelty: multi-agent confidence aggregation as a UI safety layer — the architectural insight is valuable independent of trial outcome. Note that ensemble overconfidence (all models wrong together) is the key limitation to flag in the claim.
|
||||
|
||||
|
||||
## Key Facts
|
||||
- NCT07328815 is a single-blind RCT with 50 physicians (25 per arm) testing automation bias mitigation
|
||||
- The trial uses three frontier LLMs for confidence signal generation: Claude Sonnet 4.5, Gemini 2.5 Pro Thinking, and GPT-5.1
|
||||
- The trial is registered at ClinicalTrials.gov as of March 15, 2026
|
||||
- Protocol and statistical analysis plan available at cdn.clinicaltrials.gov/large-docs/15/NCT07328815/Prot_SAP_000.pdf
|
||||
- Related arxiv preprint on evidence-based nudges: 2602.10345
|
||||
- Parent study NCT06963957 showed 20-hour AI-literacy training failed to prevent automation bias
|
||||
Loading…
Reference in a new issue