diff --git a/domains/health/human-in-the-loop clinical AI degrades to worse-than-AI-alone because physicians both de-skill from reliance and introduce errors when overriding correct outputs.md b/domains/health/human-in-the-loop clinical AI degrades to worse-than-AI-alone because physicians both de-skill from reliance and introduce errors when overriding correct outputs.md index 48a0da2a4..044df46bb 100644 --- a/domains/health/human-in-the-loop clinical AI degrades to worse-than-AI-alone because physicians both de-skill from reliance and introduce errors when overriding correct outputs.md +++ b/domains/health/human-in-the-loop clinical AI degrades to worse-than-AI-alone because physicians both de-skill from reliance and introduce errors when overriding correct outputs.md @@ -33,6 +33,12 @@ OpenEvidence's 1M daily consultations (30M+/month) with 44% of physicians expres --- +### Additional Evidence (extend) +*Source: [[2026-03-22-cognitive-bias-clinical-llm-npj-digital-medicine]] | Added: 2026-03-22* + +The npj Digital Medicine cognitive bias taxonomy provides the mechanism for WHY human-in-the-loop degrades: LLM anchoring + confirmation bias creates a reinforcement loop where physicians' initial (potentially biased) frames are confirmed by AI rather than challenged. The GPT-4 study showing 'incorrect initial diagnoses consistently influenced later reasoning' explains the cognitive architecture behind the degradation—it's not just de-skilling, it's active amplification of human cognitive errors through AI confirmation. + + Relevant Notes: - [[centaur team performance depends on role complementarity not mere human-AI combination]] -- the chess centaur model does NOT generalize to clinical medicine where physician overrides degrade AI performance - [[medical LLM benchmark performance does not translate to clinical impact because physicians with and without AI access achieve similar diagnostic accuracy in randomized trials]] -- the multi-hospital RCT found similar diagnostic accuracy with/without AI; the Stanford/Harvard study found AI alone dramatically superior diff --git a/domains/health/medical LLM benchmark performance does not translate to clinical impact because physicians with and without AI access achieve similar diagnostic accuracy in randomized trials.md b/domains/health/medical LLM benchmark performance does not translate to clinical impact because physicians with and without AI access achieve similar diagnostic accuracy in randomized trials.md index 0599d652c..08102bf3a 100644 --- a/domains/health/medical LLM benchmark performance does not translate to clinical impact because physicians with and without AI access achieve similar diagnostic accuracy in randomized trials.md +++ b/domains/health/medical LLM benchmark performance does not translate to clinical impact because physicians with and without AI access achieve similar diagnostic accuracy in randomized trials.md @@ -30,6 +30,12 @@ OpenEvidence achieved 100% USMLE score (first AI in history) and is now deployed OpenEvidence's medRxiv preprint (November 2025) showed 24% accuracy for relevant answers on complex open-ended clinical scenarios, despite achieving 100% on USMLE-type multiple choice questions. This 76-percentage-point gap between benchmark performance and open-ended clinical scenarios confirms that structured test performance does not predict real-world clinical utility. +### Additional Evidence (extend) +*Source: [[2026-03-22-cognitive-bias-clinical-llm-npj-digital-medicine]] | Added: 2026-03-22* + +The cognitive bias taxonomy explains one mechanism for the benchmark-to-practice gap: LLMs may perform well on isolated diagnostic tasks but amplify cognitive biases in real clinical workflows where anchoring, framing, and confirmation bias interact with physician decision-making. The framing bias finding (GPT-4 accuracy declining with 'disruptive behaviors or other salient but irrelevant details') suggests that real-world clinical contexts introduce noise that benchmarks don't capture. + + Relevant Notes: - [[human-in-the-loop clinical AI degrades to worse-than-AI-alone because physicians both de-skill from reliance and introduce errors when overriding correct outputs]] -- Stanford/Harvard study shows physician overrides degrade AI performance from 90% to 68% diff --git a/inbox/queue/.extraction-debug/2026-03-22-cognitive-bias-clinical-llm-npj-digital-medicine.json b/inbox/queue/.extraction-debug/2026-03-22-cognitive-bias-clinical-llm-npj-digital-medicine.json new file mode 100644 index 000000000..8ef4a80a6 --- /dev/null +++ b/inbox/queue/.extraction-debug/2026-03-22-cognitive-bias-clinical-llm-npj-digital-medicine.json @@ -0,0 +1,24 @@ +{ + "rejected_claims": [ + { + "filename": "clinical-llms-amplify-human-cognitive-biases-through-anchoring-and-confirmation-mechanisms.md", + "issues": [ + "missing_attribution_extractor" + ] + } + ], + "validation_stats": { + "total": 1, + "kept": 0, + "fixed": 1, + "rejected": 1, + "fixes_applied": [ + "clinical-llms-amplify-human-cognitive-biases-through-anchoring-and-confirmation-mechanisms.md:set_created:2026-03-22" + ], + "rejections": [ + "clinical-llms-amplify-human-cognitive-biases-through-anchoring-and-confirmation-mechanisms.md:missing_attribution_extractor" + ] + }, + "model": "anthropic/claude-sonnet-4.5", + "date": "2026-03-22" +} \ No newline at end of file diff --git a/inbox/queue/2026-03-22-cognitive-bias-clinical-llm-npj-digital-medicine.md b/inbox/queue/2026-03-22-cognitive-bias-clinical-llm-npj-digital-medicine.md index f49a8a474..df46402e3 100644 --- a/inbox/queue/2026-03-22-cognitive-bias-clinical-llm-npj-digital-medicine.md +++ b/inbox/queue/2026-03-22-cognitive-bias-clinical-llm-npj-digital-medicine.md @@ -7,9 +7,13 @@ date: 2025-01-01 domain: health secondary_domains: [ai-alignment] format: research paper -status: unprocessed +status: enrichment priority: medium tags: [cognitive-bias, llm, clinical-ai, anchoring-bias, framing-bias, automation-bias, confirmation-bias, npj-digital-medicine] +processed_by: vida +processed_date: 2026-03-22 +enrichments_applied: ["human-in-the-loop clinical AI degrades to worse-than-AI-alone because physicians both de-skill from reliance and introduce errors when overriding correct outputs.md", "medical LLM benchmark performance does not translate to clinical impact because physicians with and without AI access achieve similar diagnostic accuracy in randomized trials.md"] +extraction_model: "anthropic/claude-sonnet-4.5" --- ## Content @@ -60,3 +64,12 @@ Published in npj Digital Medicine (2025, PMC12246145). The paper provides a taxo PRIMARY CONNECTION: "clinical AI augments physicians but creates novel safety risks requiring centaur design" (Belief 5) WHY ARCHIVED: Provides cognitive mechanism explanation for why "reinforcement" is dangerous — LLM anchoring + confirmation bias means OE reinforces the physician's initial (potentially biased) frame, not the correct frame EXTRACTION HINT: The amplification framing is the key claim to extract: LLMs don't just replicate human cognitive biases, they may amplify them by confirming anchored/framed clinical assessments without the contextual resistance of experienced clinicians. + + +## Key Facts +- npj Digital Medicine published a cognitive bias taxonomy for clinical LLMs in 2025 (PMC12246145) +- A second npj Digital Medicine paper on mitigation frameworks was published in 2024 (PMC11494053) +- GPT-4 study found incorrect initial diagnoses consistently influenced later reasoning until multi-agent setup challenged the anchor +- GPT-4 diagnostic accuracy declined when cases were reframed with disruptive behaviors or irrelevant details +- NCT06963957 RCT (medRxiv August 2025) confirmed automation bias in clinical settings +- NCT07328815 is a registered trial testing behavioral nudges to reduce automation bias