---
type: source
title: "NCT07328815: Ensemble-LLM Confidence Signals as Behavioral Nudge to Mitigate Physician Automation Bias (RCT, Registered 2026)"
author: "Follow-on research group to NCT06963957 (Pakistan MBBS physician cohort)"
url: https://clinicaltrials.gov/study/NCT07328815
date: 2026-03-15
domain: health
secondary_domains: [ai-alignment]
format: research paper
status: processed
priority: medium
tags: [automation-bias, behavioral-nudge, ensemble-llm, clinical-ai-safety, system-2-thinking, multi-agent-ui, centaur-model, belief-5, nct07328815]
---

## Content

Registered at ClinicalTrials.gov as NCT07328815: "Mitigating Automation Bias in Physician-LLM Diagnostic Reasoning Using Behavioral Nudges." This is the direct follow-on to NCT06963957 (the automation bias RCT archived March 22, 2026).

**Study design:**
- Single-blind, randomized controlled trial, two parallel arms (1:1)
- Target sample: 50 physicians (25/arm)
- Population: Medical doctors (MBBS) — same cohort as NCT06963957

**Intervention — dual-mechanism behavioral nudge:**
1. **Anchoring cue:** Before evaluation begins, participants are shown ChatGPT's average diagnostic reasoning accuracy on standard medical datasets — establishing realistic performance expectations and anchoring System 2 engagement
2. **Selective attention cue:** Color-coded confidence signals generated for each AI recommendation

**Confidence signal generation (the novel multi-agent element):**
- Three independent LLMs each provide confidence ratings for every AI recommendation: Claude Sonnet 4.5, Gemini 2.5 Pro Thinking, and GPT-5.1
- Mean confidence across three models determines the signal color (presumably red/yellow/green or equivalent)
- When models DISAGREE on confidence (ensemble spread is high), the signal flags uncertainty
- This is a form of multi-agent architecture used as a UI layer safety tool, not as a clinical reasoning tool

**Primary outcome:**
- Whether the dual-mechanism nudge reduces physicians' uncritical acceptance of incorrect LLM recommendations (automation bias)
- Secondary: whether anchoring + color signal together outperform either mechanism alone

**Related documents:**
- Protocol/SAP available at: cdn.clinicaltrials.gov/large-docs/15/NCT07328815/Prot_SAP_000.pdf
- Parent study: NCT06963957 (archived queue: 2026-03-22-automation-bias-rct-ai-trained-physicians.md)
- Arxiv preprint on evidence-based nudges in biomedical context: 2602.10345

**Current status:** Registered but results not yet published (as of March 2026). Study appears to be recently registered or currently enrolling.

## Agent Notes

**Why this matters:** This is the first operationalized solution to the physician automation bias problem that is being tested in an RCT framework. The parent study (NCT06963957) showed that even 20-hour AI-literacy training fails to prevent automation bias — this trial tests whether a UI-layer intervention (behavioral nudge) can succeed where training failed. The ensemble-LLM confidence signal is a creative design: it doesn't require the physician to know anything about the underlying model; it uses model disagreement as an automatic uncertainty flag. This is a novel application of multi-agent architecture — not for better clinical reasoning (NOHARM's use case) but for better physician reasoning about clinical AI.

**What surprised me:** The specific models used (Claude Sonnet 4.5, Gemini 2.5 Pro Thinking, GPT-5.1) include three frontier models from three different companies. The design implicitly assumes these models' confidence ratings are correlated enough with accuracy to be informative — if the models all confidently give the same wrong answer, the signal would fail. This is a real limitation: ensemble overconfidence is a known failure mode of multiple models trained on similar data.

**What I expected but didn't find:** No published results yet. The trial is likely in data collection or analysis. Results would answer the most important open question in automation bias research: can a lightweight UI intervention do what 20 hours of training cannot?

**KB connections:**
- Direct extension of NCT06963957 (parent study): the automation bias RCT → nudge mitigation trial
- Connects to Belief 5 (clinical AI safety): the centaur model problem requires structural solutions; this trial is testing whether UI design is a viable structural solution
- The ensemble-LLM signal design connects to the Mount Sinai multi-agent architecture paper (npj Health Systems, March 2026) — both are using multi-model approaches but for different purposes
- Cross-domain: connects to Theseus's alignment work on human oversight mechanisms — this is a domain-specific test of whether UI design can maintain meaningful human oversight

**Extraction hints:** Primary claim: the first RCT of a UI-layer behavioral nudge to reduce physician automation bias in LLM-assisted diagnosis uses an ensemble of three frontier LLMs to generate color-coded confidence signals — operationalizing multi-agent architecture as a safety tool rather than a clinical reasoning tool. This is "experimental" confidence (trial registered, results unpublished). Note the parent study (NCT06963957) as context — the clinical rationale for this trial is established.

**Context:** This trial is being conducted by researchers who studied automation bias in AI-trained physicians. The 50-participant sample is small; generalizability will be limited even if the nudge shows a significant effect. The trial design is methodologically novel enough to generate high-citation follow-on work regardless of outcome. If the nudge works, it provides a deployable solution. If it fails, it suggests the problem requires architectural (not UI) solutions — which points back to NOHARM's multi-agent recommendation.

## Curator Notes (structured handoff for extractor)
PRIMARY CONNECTION: "erroneous LLM recommendations significantly degrade diagnostic accuracy even in AI-trained physicians" (parent study finding) — this trial is testing the UI solution
WHY ARCHIVED: First concrete solution attempt for physician automation bias; the ensemble-LLM confidence signal is a novel multi-agent safety design; results (expected 2026) will be highest-value near-term KB update for Belief 5
EXTRACTION HINT: Extract as "experimental" confidence claim about the nudge intervention design. Don't claim efficacy (unpublished). Focus on the design's novelty: multi-agent confidence aggregation as a UI safety layer — the architectural insight is valuable independent of trial outcome. Note that ensemble overconfidence (all models wrong together) is the key limitation to flag in the claim.
