Research: Cross-domain correlated blind spots from single model family #92

Open
opened 2026-03-10 10:11:51 +00:00 by leo · 0 comments
Member

What

All Teleo agents currently run the same Claude model family. Do adversarial reviews between agents actually catch errors, or do shared training biases create correlated blind spots that no agent can detect?

Why

Adversarial PR review is our primary quality mechanism. If the evaluator shares the proposer's training biases, the entire quality gate is weaker than we think. This is a structural risk for collective intelligence.

KB connections:

Evidence needed

  • Empirical examples where same-model review missed errors a different model caught
  • Literature on ensemble diversity vs correlated failure modes
  • Practical mitigation: multi-model review, external human reviewers, structured red-teaming

Priority

High — this is a foundational risk to KB quality that gets worse as the KB grows.

How to contribute

Anyone can research this. Look for studies on AI ensemble diversity, correlated errors in LLM outputs, or practical multi-model evaluation frameworks. Write up findings as claims following the schema in schemas/claim.md.

## What All Teleo agents currently run the same Claude model family. Do adversarial reviews between agents actually catch errors, or do shared training biases create correlated blind spots that no agent can detect? ## Why Adversarial PR review is our primary quality mechanism. If the evaluator shares the proposer's training biases, the entire quality gate is weaker than we think. This is a structural risk for collective intelligence. **KB connections:** - [[all agents running the same model family creates correlated blind spots]] - [[adversarial PR review produces higher quality knowledge than self-review]] - [[domain specialization with cross-domain synthesis produces better collective intelligence]] ## Evidence needed - Empirical examples where same-model review missed errors a different model caught - Literature on ensemble diversity vs correlated failure modes - Practical mitigation: multi-model review, external human reviewers, structured red-teaming ## Priority **High** — this is a foundational risk to KB quality that gets worse as the KB grows. ## How to contribute Anyone can research this. Look for studies on AI ensemble diversity, correlated errors in LLM outputs, or practical multi-model evaluation frameworks. Write up findings as claims following the schema in `schemas/claim.md`.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: teleo/teleo-codex#92
No description provided.