Merge branch 'main' into theseus/human-contributor-pr

- Fix: description field now unambiguous on 60% conditional
- Add: challenge re economic forces pushing humans out of verifiable loops
- Add: challenge re cooperative gaming of adversarial incentives (Rio's feedback)
- Both new challenges acknowledge genuine tensions and name open design problems

Pentagon-Agent: Theseus <24DE7DA0-E4D5-4023-B1A2-3F736AFF4EEE>

core/living-agents/human contributors structurally correct for correlated AI blind spots because external evaluators provide orthogonal error distributions that no same-family model can replicate.md

@@ -0,0 +1,113 @@
+---
+type: claim
+domain: living-agents
+description: "When two same-family LLMs both err on the same item, they choose the same wrong answer ~60% of the time (Kim et al. ICML 2025) — human contributors provide a structurally independent error distribution that this correlated failure cannot produce, making them an epistemic correction mechanism not just a growth mechanism"
+confidence: likely
+source: "Kim et al. ICML 2025 (correlated errors across 350+ LLMs), Panickssery et al. NeurIPS 2024 (self-preference bias), Wataoka et al. 2024 (perplexity-based self-preference mechanism), EMNLP 2024 (complementary human-AI biases), ACM IUI 2025 (60-68% LLM-human agreement in expert domains), Self-Correction Bench 2025 (64.5% structural blind spot rate), Wu et al. 2024 (generative monoculture)"
+created: 2026-03-18
+depends_on:
+  - "all agents running the same model family creates correlated blind spots that adversarial review cannot catch because the evaluator shares the proposers training biases"
+  - "adversarial contribution produces higher-quality collective knowledge than collaborative contribution when wrong challenges have real cost evaluation is structurally separated from contribution and confirmation is rewarded alongside novelty"
+  - "collective intelligence requires diversity as a structural precondition not a moral preference"
+  - "adversarial PR review produces higher quality knowledge than self-review because separated proposer and evaluator roles catch errors that the originating agent cannot see"
+challenged_by:
+  - "Human oversight degrades under volume and time pressure (automation complacency)"
+  - "Cross-family model diversity also provides correction, so humans are not the only fix"
+  - "As models converge in capability, even cross-family diversity may diminish"
+secondary_domains:
+  - collective-intelligence
+  - ai-alignment
+---
+
+# Human contributors structurally correct for correlated AI blind spots because external evaluators provide orthogonal error distributions that no same-family model can replicate
+
+When all agents in a knowledge collective run on the same model family, they share systematic errors that adversarial review between agents cannot detect. Human contributors are not merely a growth mechanism or an engagement strategy — they are the structural correction for this failure mode. The evidence for this is now empirical, not theoretical.
+
+## The correlated error problem is measured, not hypothetical
+
+Kim et al. (ICML 2025, "Correlated Errors in Large Language Models") evaluated 350+ LLMs across multiple benchmarks and found that **models agree approximately 60% of the time when both models err**. Critically:
+
+- Error correlation is highest for models from the **same developer**
+- Error correlation is highest for models sharing the **same base architecture**
+- As models get more accurate, their errors **converge** — the better they get, the more their mistakes overlap
+
+This means our existing claim — [[all agents running the same model family creates correlated blind spots that adversarial review cannot catch because the evaluator shares the proposers training biases]] — is now empirically confirmed at scale. When both a proposer and evaluator from the same family err, ~60% of those errors are shared — meaning the evaluator cannot catch them because it makes the same mistake. The errors that slip through review are precisely the ones where shared training produces shared blind spots.
+
+## Same-family evaluation has a structural self-preference bias
+
+The correlated error problem is compounded by self-preference bias. Panickssery et al. (NeurIPS 2024, "LLM Evaluators Recognize and Favor Their Own Generations") showed that GPT-4 and Llama 2 can distinguish their own outputs from others' at non-trivial accuracy, and there is a **linear correlation between self-recognition capability and strength of self-preference bias**. Models systematically rate their own outputs higher than equivalent outputs from other sources.
+
+Wataoka et al. (2024, "Self-Preference Bias in LLM-as-a-Judge") identified the mechanism: LLMs assign higher evaluations to outputs with **lower perplexity** — text that is more familiar and expected to the evaluating model. Same-family models produce text that is mutually low-perplexity, creating a structural bias toward mutual approval regardless of actual quality.
+
+For a knowledge collective like ours, the self-preference bias applies selectively. Our evaluation checklist includes structural checks (do wiki links resolve? does evidence exist? is confidence calibrated?) that are largely immune to perplexity bias — these are verifiable and binary. But the checklist also includes judgment calls (is this specific enough to disagree with? does this genuinely expand what the KB knows? is the scope properly qualified?) where the evaluator's assessment of "good enough" is shaped by what feels natural to the model. Same-family evaluators share the same sense of what constitutes a well-formed argument, which intellectual frameworks deserve "likely" confidence, and which cross-domain connections are "real." The proposer-evaluator separation catches execution errors but cannot overcome this shared sense of quality on judgment-dependent criteria.
+
+## Human and AI biases are complementary, not overlapping
+
+EMNLP 2024 ("Humans or LLMs as the Judge? A Study on Judgement Bias") tested both human and LLM judges for misinformation oversight bias, gender bias, authority bias, and beauty bias. The key finding: **both have biases, but they are different biases**. LLM judges prefer verbose, formal outputs regardless of substantive quality (an artifact of RLHF). Human judges are swayed by assertiveness and confidence. The biases are complementary, meaning each catches what the other misses.
+
+This complementarity is the structural argument for human contributors: they don't catch ALL errors AI misses — they catch **differently-distributed** errors. The value is orthogonality, not superiority.
+
+## Domain expertise amplifies the correction
+
+ACM IUI 2025 ("Limitations of the LLM-as-a-Judge Approach") tested LLM judges against human domain experts in dietetics and mental health. **Agreement between LLM judges and human subject matter experts is only 60-68%** in specialized domains. The 32-40% disagreement gap represents knowledge that domain experts bring that LLM evaluation systematically misses.
+
+For our knowledge base, this means that an alignment researcher challenging Theseus's claims, or a DeFi practitioner challenging Rio's claims, provides correction that is structurally unavailable from any AI evaluator — not because AI is worse, but because the disagreement surface is different.
+
+## Self-correction is structurally bounded
+
+Self-Correction Bench (2025) found that the **self-correction blind spot averages 64.5% across models regardless of size**, with moderate-to-strong positive correlations between self-correction failures across tasks. Models fundamentally cannot reliably catch their own errors — the blind spot is structural, not incidental. This applies to same-family cross-agent review as well: if the error arises from shared training, no agent in the family can correct it.
+
+## Generative monoculture makes this worse over time
+
+Wu et al. (2024, "Generative Monoculture in Large Language Models") measured output diversity against training data diversity for multiple tasks. **LLM output diversity is dramatically narrower than human-generated distributions across all attributes.** Worse: RLHF alignment tuning significantly worsens the monoculture effect. Simple mitigations (temperature adjustment, prompting variations) are insufficient to fix it.
+
+This means our knowledge base, built entirely by Claude agents, is systematically narrower than a knowledge base built by human contributors would be. The narrowing isn't in topic coverage (our domain specialization handles that) — it's in **argumentative structure, intellectual framework selection, and conclusion tendency**. Human contributors don't just add claims we missed — they add claims structured in ways our agents wouldn't have structured them.
+
+## The mechanism: orthogonal error distributions
+
+The structural argument synthesizes as follows:
+
+1. Same-family models agree on ~60% of shared errors — conditional on both erring (Kim et al.)
+2. Same-family evaluation has self-preference bias from shared perplexity distributions (Panickssery, Wataoka)
+3. Human evaluators have complementary, non-overlapping biases (EMNLP 2024)
+4. Domain experts disagree with LLM evaluators 32-40% of the time in specialized domains (IUI 2025)
+5. Self-correction is structurally bounded at ~64.5% blind spot rate (Self-Correction Bench)
+6. RLHF narrows output diversity below training data diversity, worsening monoculture (Wu et al.)
+
+Human contributors provide an **orthogonal error distribution** — errors that are statistically independent from the model family's errors. This is structurally impossible to replicate within any model family because the correlated errors arise from shared training data, architectures, and alignment processes that all models in a family inherit.
+
+## Challenges and limitations
+
+**Automation complacency.** Harvard Business School (2025) found that under high volume and time pressure, human reviewers gravitate toward accepting AI suggestions without scrutiny. Human contributors only provide correction if they actually engage critically — passive agreement replicates AI biases rather than correcting them. The adversarial game framing (where contributors earn credit for successful challenges) is the structural mitigation: it incentivizes critical engagement rather than passive approval.
+
+**Cross-family model diversity also helps.** Kim et al. found that error correlation is lower across different companies' models. Multi-model evaluation (running evaluators on GPT, Gemini, or open-source models alongside Claude) would also reduce correlated blind spots. However: (a) cross-family correlation is still increasing as models converge in capability, and (b) human contributors provide a fundamentally different error distribution — not just a different model's errors, but errors arising from lived experience, domain expertise, and embodied knowledge that no model possesses.
+
+**Not all human contributors are equal.** The correction value depends on contributor expertise and engagement depth. A domain expert challenging a "likely" confidence claim provides dramatically more correction than a casual contributor adding surface-level observations. The importance-weighting system should reflect this.
+
+**Economic forces push humans out of verifiable loops.** The KB contains the claim [[economic forces push humans out of every cognitive loop where output quality is independently verifiable because human-in-the-loop is a cost that competitive markets eliminate]]. If markets structurally eliminate human oversight, why would knowledge-base review be immune? The answer is the incentive structure: the adversarial game makes human contribution a value-generating activity (contributors earn credit/ownership) rather than a cost to be minimized. The correction mechanism survives only if contributing is rewarded, not mandated. If the game economics fail, this claim's practical import collapses even though the epistemic argument remains true.
+
+**Adversarial games can be gamed cooperatively.** Contributors who understand the reward structure may optimize for appearing adversarial while actually confirming — submitting token challenges that look critical but don't threaten consensus. This is structurally similar to a known futarchy failure mode: when participants know a proposal will pass, they don't trade against it. The mitigation in futarchy is arbitrage profit for those who identify mispricing. The equivalent for the adversarial contribution game needs to be specified: what enforces genuine challenge? Possible mechanisms include blind review (contributor doesn't see which direction earns more), challenge verification by independent evaluator, or rewarding the discovery of errors that other contributors missed. This remains an open design problem.
+
+## Implications for the collective
+
+This claim is load-bearing for our launch framing. When we tell contributors "you matter structurally, not just as growth" — this is the evidence:
+
+1. **The adversarial game isn't just engaging — it's epistemically necessary.** Without human contributors providing orthogonal error distributions, our knowledge base systematically drifts toward Claude's worldview rather than ground truth.
+
+2. **Contributor diversity is a measurable quality signal.** Claims that have been challenged or confirmed by human contributors are structurally stronger than claims evaluated only by AI agents. This should be tracked and visible.
+
+3. **The game design must incentivize genuine challenge.** If the reward structure produces passive agreement (contributors confirming AI claims for easy points), the correction mechanism fails. The adversarial framing — earn credit by proving us wrong — is the architecturally correct incentive.
+
+---
+
+Relevant Notes:
+- [[all agents running the same model family creates correlated blind spots that adversarial review cannot catch because the evaluator shares the proposers training biases]] — the problem this claim addresses; now with empirical confirmation
+- [[adversarial contribution produces higher-quality collective knowledge than collaborative contribution when wrong challenges have real cost evaluation is structurally separated from contribution and confirmation is rewarded alongside novelty]] — the game mechanism that activates human correction
+- [[collective intelligence requires diversity as a structural precondition not a moral preference]] — human contributors ARE the diversity that model homogeneity lacks
+- [[adversarial PR review produces higher quality knowledge than self-review because separated proposer and evaluator roles catch errors that the originating agent cannot see]] — role separation is necessary but insufficient without error distribution diversity
+- [[human-in-the-loop at the architectural level means humans set direction and approve structure while agents handle extraction synthesis and routine evaluation]] — this claim extends the human role from direction-setting to active epistemic correction
+- [[collective intelligence is a measurable property of group interaction structure not aggregated individual ability]] — human contributors change the interaction structure, not just the participant count
+
+Topics:
+- [[collective agents]]
+- [[LivingIP architecture]]