---
type: claim
domain: ai-alignment
secondary_domains: [collective-intelligence, internet-finance]
description: "Anthropic's Project Deal pilot found users reported identical fairness (4.05 vs 4.06 on a 7-point scale) across Opus and Haiku agents despite Opus sellers extracting $2.68 more per item and Opus buyers paying $2.45 less — subjective satisfaction was decoupled from measurable capability-driven outcome gaps"
confidence: experimental
source: "Anthropic, 'Project Deal: What happens when AI agents go to the market?' (December 2025, 69-participant pilot, N=186 deals, randomized Opus/Haiku assignment in mixed-model runs)"
created: 2026-04-24
related:
- AI capability and reliability are independent dimensions because Claude solved a 30-year open mathematical problem while simultaneously degrading at basic program execution during the same session
- centaur team performance depends on role complementarity not mere human-AI combination
- economic forces push humans out of every cognitive loop where output quality is independently verifiable because human-in-the-loop is a cost that competitive markets eliminate
- all agents running the same model family creates correlated blind spots that adversarial review cannot catch because the evaluator shares the proposers training biases
sourced_from:
  - inbox/archive/ai-alignment/2025-12-anthropic-project-deal.md
supports:
- agent mediated commerce produces invisible economic stratification because capability gaps translate to measurable market disadvantage that users cannot detect and therefore cannot correct through provider switching
reweave_edges:
- agent mediated commerce produces invisible economic stratification because capability gaps translate to measurable market disadvantage that users cannot detect and therefore cannot correct through provider switching|supports|2026-04-25
---

# Users cannot detect when their AI agent is underperforming because subjective fairness ratings decouple from measurable economic outcomes across capability tiers

Anthropic's Project Deal pilot (December 2025) ran a controlled comparison of autonomous agent-to-agent commerce across four parallel Slack marketplaces. 69 participants were randomly assigned Claude Opus 4.5 or Haiku 4.5 agents and given $100 each to buy and sell personal items through a week of autonomous negotiation. 186 deals completed. The empirical structure is tight: same marketplace, same items, same instructions, randomized agent assignment — any outcome difference isolates the model variable.

## The empirical finding

Opus agents produced statistically significant dollar-value advantages over Haiku agents across every metric measured:
- Completed approximately 2 more deals per participant (p=0.001)
- Extracted $2.68 more per item when selling identical items (p=0.030)
- Paid $2.45 less per item when buying (p=0.015)
- Opus-to-Haiku transactions averaged $24.18; Opus-to-Opus averaged $18.63

A specific example from the study: the same broken folding bike sold for $38 by a Haiku agent and $65 by an Opus agent.

But when surveyed about the experience, participants reported fairness scores of 4.05 (Opus) vs 4.06 (Haiku) on a 1-7 scale. Satisfaction showed no statistically significant difference. Of participants who experienced both models in sequence, 17 ranked their Opus run above their Haiku run — but 11 ranked it the other way. Anthropic's summary: "Those with weaker models didn't notice their disadvantage."

## Why this matters

User perception of AI agent performance is the feedback signal most existing literature assumes governs deployment quality. If users can detect when their agent underperforms, they switch to better agents, and the market selects toward capability. The Project Deal finding shows this feedback loop is broken for a non-trivial class of tasks: users lack the reference frame to detect capability gaps that produce measurable economic disparities.

The mechanism is structural rather than psychological. In autonomous commerce, the user sees only their own transactions — not the counterfactual transactions they would have completed with a better agent. Without that counterfactual, a $38 sale feels like a successful negotiation rather than a $27 underperformance relative to what a capable agent would have extracted. The reference frame for "what good looks like" requires seeing outcomes across capability tiers, which individual users cannot do.

This connects to [[centaur team performance depends on role complementarity not mere human-AI combination]] — the centaur model assumes humans can evaluate and correct AI outputs. But when the AI operates autonomously in a domain where the human lacks independent performance benchmarks, the correction channel collapses. And since [[economic forces push humans out of every cognitive loop where output quality is independently verifiable because human-in-the-loop is a cost that competitive markets eliminate]], the trajectory is toward more autonomous agent commerce, not less — which amplifies the blind spot rather than eliminating it.

## Scope and limitations

The finding is from a single pilot study — 69 participants, one organization, one week, narrow task class (personal goods negotiation among Anthropic employees). The fairness Likert scale (1-7) may not capture the specific dimensions where users would detect underperformance; different survey instruments could surface the disparity. Participants were Anthropic employees, plausibly more trusting of AI agents than a general population. The study does not include longitudinal data on whether users eventually detect disparities through repeated interactions over longer timeframes.

The claim is scoped to **autonomous commerce with low-frequency goods and no performance benchmarks visible to the user**. It does not necessarily generalize to domains where users have independent performance benchmarks (trading with observable market prices), repeated interactions over long time horizons (where users accumulate evidence), or adversarial contexts (where users have stronger motivation to detect underperformance).

## Challenges

- Single pilot study with no independent replication. The p-values are strong but the study design has not been repeated by external researchers, and the participant pool is homogeneous.
- The survey instrument matters. Asking "how fair was this deal?" on a 1-7 scale is a specific measurement choice. Different instruments — asking users to estimate what a skilled negotiator would have extracted, showing counterfactual prices, or measuring behavioral changes rather than stated satisfaction — might surface the disparity users couldn't articulate.
- The magnitude of capability disparity (~$3 per item, ~$100 total per participant over a week) may be below the threshold users bother to detect. The same decoupling might break down at larger magnitudes where the disparity becomes visible through other channels (e.g., people comparing notes, obvious pricing anomalies).

---

Relevant Notes:
- [[AI capability and reliability are independent dimensions because Claude solved a 30-year open mathematical problem while simultaneously degrading at basic program execution during the same session]] — capability disparities exist; Project Deal shows users can't detect them in deployed autonomous settings
- [[centaur team performance depends on role complementarity not mere human-AI combination]] — centaur correction fails when the human lacks independent performance benchmarks to evaluate AI output
- [[economic forces push humans out of every cognitive loop where output quality is independently verifiable because human-in-the-loop is a cost that competitive markets eliminate]] — the trajectory is toward more autonomous agent operation, amplifying the perception gap
- [[all agents running the same model family creates correlated blind spots that adversarial review cannot catch because the evaluator shares the proposers training biases]] — related blindness pattern: correlated errors go undetected by evaluators who share the error-producing traits

Topics:
- [[_map]]