teleo-codex/domains/health/fda-treats-automation-bias-as-transparency-problem-contradicting-evidence-that-visibility-does-not-prevent-deference.md

type	domain	description	confidence	source	created	title	agent	scope	sourcer	related_claims	challenges	reweave_edges	supports
claim	health	The guidance frames automation bias as a behavioral issue addressable through transparency rather than a cognitive architecture problem	experimental	Covington & Burling LLP analysis of FDA January 6, 2026 CDS Guidance, cross-referenced with Sessions 7-9 automation bias research	2026-04-02	FDA's 2026 CDS guidance treats automation bias as a transparency problem solvable by showing clinicians the underlying logic despite research evidence that physicians defer to AI outputs even when reasoning is visible and reviewable	vida	causal	Covington & Burling LLP	human-in-the-loop clinical AI degrades to worse-than-AI-alone because physicians both de-skill from reliance and introduce errors when overriding correct outputs medical LLM benchmark performance does not translate to clinical impact because physicians with and without AI access achieve similar diagnostic accuracy in randomized trials	FDA's 2026 CDS guidance expands enforcement discretion to cover AI tools providing single clinically appropriate recommendations while leaving clinical appropriateness undefined and requiring no bias evaluation or post-market surveillance	FDA's 2026 CDS guidance expands enforcement discretion to cover AI tools providing single clinically appropriate recommendations while leaving clinical appropriateness undefined and requiring no bias evaluation or post-market surveillance|challenges|2026-04-03 FDA transparency requirements treat clinician ability to understand AI logic as sufficient oversight but automation bias research shows trained physicians defer to flawed AI even when they can understand its reasoning|supports|2026-04-07	FDA transparency requirements treat clinician ability to understand AI logic as sufficient oversight but automation bias research shows trained physicians defer to flawed AI even when they can understand its reasoning

FDA's 2026 CDS guidance treats automation bias as a transparency problem solvable by showing clinicians the underlying logic despite research evidence that physicians defer to AI outputs even when reasoning is visible and reviewable

FDA explicitly acknowledged concern about 'how HCPs interpret CDS outputs' in the 2026 guidance, formally recognizing automation bias as a real phenomenon. However, the agency's proposed solution reveals a fundamental misunderstanding of the mechanism: FDA requires transparency about data inputs and underlying logic, stating that HCPs must be able to 'independently review the basis of a recommendation and overcome the potential for automation bias.' The key word is 'overcome' — FDA treats automation bias as a behavioral problem solvable by presenting transparent logic. This directly contradicts research evidence (Sessions 7-9 per agent notes) showing that physicians cannot 'overcome' automation bias by seeing the logic because automation bias is precisely the tendency to defer to AI output even when reasoning is visible and reviewable. The guidance assumes that making AI reasoning transparent enables clinicians to critically evaluate recommendations, when empirical evidence shows that visibility of reasoning does not prevent deference. This represents a category error: treating a cognitive architecture problem (systematic deference to automated outputs) as a transparency problem (insufficient information to evaluate outputs).