4fbe30da36
Some checks failed
Mirror PR to Forgejo / mirror (pull_request) Has been cancelled
Pentagon-Agent: Vida <HEADLESS>
58 lines
4.5 KiB
Markdown
58 lines
4.5 KiB
Markdown
---
|
|
type: source
|
|
title: "Upskilling or Deskilling? Measurable Role of AI-Supported Training for Radiology Residents"
|
|
author: "Heudel et al. (PMC11780016)"
|
|
url: https://pmc.ncbi.nlm.nih.gov/articles/PMC11780016/
|
|
date: 2025-01
|
|
domain: health
|
|
secondary_domains: []
|
|
format: study
|
|
status: unprocessed
|
|
priority: high
|
|
tags: [clinical-ai, deskilling, upskilling, radiology, training, residents, diagnostic-performance]
|
|
---
|
|
|
|
## Content
|
|
|
|
**Study design:** 8 residents (4 first-year, 4 third-year) evaluated 150 chest X-rays using the Brixia severity score across three scenarios: no-AI, on-demand-AI, and integrated-AI. Each resident assessed 50 images per scenario. Setting: pandemic-era radiology training.
|
|
|
|
**Key findings:**
|
|
|
|
*Performance WITH AI:*
|
|
- Inter-rater agreement (ICC-1) improved from 0.665 (no-AI) to 0.813 (integrated-AI) — **22% improvement**
|
|
- Mean absolute error decreased significantly across all AI-supported scenarios (p<0.001)
|
|
- Residents showed "resilience to AI errors above an acceptability threshold" — when AI made major errors (>3 points), residents maintained average errors around 2.75-2.88
|
|
|
|
*Critical methodological limitation:*
|
|
- The study does NOT test whether residents retained improved skills WITHOUT AI after AI-assisted training
|
|
- There is no post-training, no-AI assessment
|
|
- As the paper acknowledges: "there is a substantial lack of quantitative assessments in residency education contexts"
|
|
|
|
**Conclusion:** This study documents **improved performance while AI is present**, not durable upskilling. The resilience finding (rejecting major AI errors) is notable but does not constitute evidence of skill acquisition independent of the tool.
|
|
|
|
## Agent Notes
|
|
|
|
**Why this matters:** This study is being cited in the "upskilling" literature (including Oettl et al. 2026) as evidence that AI improves physician skills. Close reading reveals it shows no such thing — it shows AI assistance improves performance WHILE AI IS PRESENT. This is the crucial distinction the divergence file needs to capture.
|
|
|
|
**What surprised me:** The Oettl 2026 paper directly cites this study as evidence for AI-induced upskilling ("radiology residents using AI tools made significantly fewer scoring errors and achieved 22% higher inter-rater agreement"). That citation is technically accurate but misleading — the study doesn't test durable skill retention. The divergence isn't about what happens with AI; it's about what happens without it after training.
|
|
|
|
**What I expected but didn't find:** A follow-up arm testing the same residents without AI after the AI-training period. The study design would have been easy to extend this way but apparently wasn't.
|
|
|
|
**KB connections:**
|
|
- This is the core piece of the deskilling/upskilling DIVERGENCE that Session 24 flagged
|
|
- The deskilling side has RCT evidence (colonoscopy ADR 28.4%→22.4% when AI removed; radiology false positives +12%)
|
|
- This study is the best empirical source for the "upskilling" side, but it only shows performance WITH AI
|
|
- Compare with the colonoscopy RCT from Session 22, which tested performance AFTER AI removal — that's the design that distinguishes deskilling from AI-assistance
|
|
|
|
**Extraction hints:**
|
|
- This source should feed a DIVERGENCE FILE, not a standalone claim
|
|
- Frame carefully: "performance improvement with AI present" ≠ "durable upskilling"
|
|
- The n=8 residents is an important scope qualifier — this is a small pilot study
|
|
- The resilience-to-errors finding is potentially extractable: "Residents can detect large AI errors but may accept small ones"
|
|
|
|
**Context:** Published in Insights into Imaging (Springer), a peer-reviewed radiology journal. First author Heudel is at Centre Léon Bérard (cancer institute). The study's framing ("upskilling or deskilling?") is its own title question — notably the answer from the data is neither clearly confirmed.
|
|
|
|
## Curator Notes (structured handoff for extractor)
|
|
PRIMARY CONNECTION: Clinical AI deskilling divergence (flagged by Session 24)
|
|
WHY ARCHIVED: Best available study cited as evidence for AI "upskilling" of physicians. Critical reading shows it only documents improved performance WITH AI present, not durable skill retention. This feeds the divergence file as the "upskilling thesis" evidence — along with its methodological limitation.
|
|
EXTRACTION HINT: Do not extract as a standalone upskilling claim. Extract as divergence evidence: what the upskilling side actually shows vs. what would be needed to confirm durable upskilling. The distinction between "performance with AI" and "durable skill after AI training" is the crux.
|