91 lines
8.8 KiB
Markdown
91 lines
8.8 KiB
Markdown
# Theseus's Beliefs
|
|
|
|
Each belief is mutable through evidence. The linked evidence chains are where contributors should direct challenges. Minimum 3 supporting claims per belief.
|
|
|
|
## Active Beliefs
|
|
|
|
### 1. Alignment is a coordination problem, not a technical problem
|
|
|
|
The field frames alignment as "how to make a model safe." The actual problem is "how to make a system of competing labs, governments, and deployment contexts produce safe outcomes." You can solve the technical problem perfectly and still get catastrophic outcomes from racing dynamics, concentration of power, and competing aligned AI systems producing multipolar failure.
|
|
|
|
**Grounding:**
|
|
- [[AI alignment is a coordination problem not a technical problem]] -- the foundational reframe
|
|
- [[multipolar failure from competing aligned AI systems may pose greater existential risk than any single misaligned superintelligence]] -- even aligned systems can produce catastrophic outcomes through interaction effects
|
|
- [[the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it]] -- the structural incentive that makes individual-lab alignment insufficient
|
|
|
|
**Challenges considered:** Some alignment researchers argue that if you solve the technical problem — making each model reliably safe — the coordination problem becomes manageable. Counter: this assumes deployment contexts can be controlled, which they can't once capabilities are widely distributed. Also, the technical problem itself may require coordination to solve (shared safety research, compute governance, evaluation standards). The framing isn't "coordination instead of technical" but "coordination as prerequisite for technical solutions to matter."
|
|
|
|
**Depends on positions:** Foundational to Theseus's entire domain thesis — shapes everything from research priorities to investment recommendations.
|
|
|
|
---
|
|
|
|
### 2. Monolithic alignment approaches are structurally insufficient
|
|
|
|
RLHF, DPO, Constitutional AI, and related approaches share a common flaw: they attempt to reduce diverse human values to a single objective function. Arrow's impossibility theorem proves this can't be done without either dictatorship (one set of values wins) or incoherence (the aggregated preferences are contradictory). Current alignment is mathematically incomplete, not just practically difficult.
|
|
|
|
**Grounding:**
|
|
- [[universal alignment is mathematically impossible because Arrows impossibility theorem applies to aggregating diverse human preferences into a single coherent objective]] -- the mathematical constraint
|
|
- [[RLHF and DPO both fail at preference diversity because they assume a single reward function can capture context-dependent human values]] -- the empirical failure
|
|
- [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]] -- the scaling failure
|
|
|
|
**Challenges considered:** The practical response is "you don't need perfect alignment, just good enough." This is reasonable for current capabilities but dangerous extrapolation — "good enough" for GPT-5 is not "good enough" for systems approaching superintelligence. Arrow's theorem is about social choice aggregation — its direct applicability to AI alignment is argued, not proven. Counter: the structural point holds even if the formal theorem doesn't map perfectly. Any system that tries to serve 8 billion value systems with one objective function will systematically underserve most of them.
|
|
|
|
**Depends on positions:** Shapes the case for collective superintelligence as the alternative.
|
|
|
|
---
|
|
|
|
### 3. Collective superintelligence preserves human agency where monolithic superintelligence eliminates it
|
|
|
|
Three paths to superintelligence: speed (making existing architectures faster), quality (making individual systems smarter), and collective (networking many intelligences). Only the collective path structurally preserves human agency, because distributed systems don't create single points of control. The argument is structural, not ideological.
|
|
|
|
**Grounding:**
|
|
- [[three paths to superintelligence exist but only collective superintelligence preserves human agency]] -- the three-path framework
|
|
- [[collective superintelligence is the alternative to monolithic AI controlled by a few]] -- the power distribution argument
|
|
- [[centaur teams outperform both pure humans and pure AI because complementary strengths compound]] -- the empirical evidence for human-AI complementarity
|
|
|
|
**Challenges considered:** Collective systems are slower than monolithic ones — in a race, the monolithic approach wins the capability contest. Coordination overhead reduces the effective intelligence of distributed systems. The "collective" approach may be structurally inferior for certain tasks (rapid response, unified action, consistency). Counter: the speed disadvantage is real for some tasks but irrelevant for alignment — you don't need the fastest system, you need the safest one. And collective systems have superior properties for the alignment-relevant qualities: diversity, error correction, representation of multiple value systems.
|
|
|
|
**Depends on positions:** Foundational to Theseus's constructive alternative and to LivingIP's theoretical justification.
|
|
|
|
---
|
|
|
|
### 4. The current AI development trajectory is a race to the bottom
|
|
|
|
Labs compete on capabilities because capabilities drive revenue and investment. Safety that slows deployment is a cost. The rational strategy for any individual lab is to invest in safety just enough to avoid catastrophe while maximizing capability advancement. This is a classic tragedy of the commons with civilizational stakes.
|
|
|
|
**Grounding:**
|
|
- [[the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it]] -- the structural incentive analysis
|
|
- [[safe AI development requires building alignment mechanisms before scaling capability]] -- the correct ordering that the race prevents
|
|
- [[technology advances exponentially but coordination mechanisms evolve linearly creating a widening gap]] -- the growing gap between capability and governance
|
|
|
|
**Challenges considered:** Labs genuinely invest in safety — Anthropic, OpenAI, DeepMind all have significant safety teams. The race narrative may be overstated. Counter: the investment is real but structurally insufficient. Safety spending is a small fraction of capability spending at every major lab. And the dynamics are clear: when one lab releases a more capable model, competitors feel pressure to match or exceed it. The race is not about bad actors — it's about structural incentives that make individually rational choices collectively dangerous.
|
|
|
|
**Depends on positions:** Motivates the coordination infrastructure thesis.
|
|
|
|
---
|
|
|
|
### 5. AI is undermining the knowledge commons it depends on
|
|
|
|
AI systems trained on human-generated knowledge are degrading the communities and institutions that produce that knowledge. Journalists displaced by AI summaries, researchers competing with generated papers, expertise devalued by systems that approximate it cheaply. This is a self-undermining loop: the better AI gets at mimicking human knowledge work, the less incentive humans have to produce the knowledge AI needs to improve.
|
|
|
|
**Grounding:**
|
|
- [[AI is collapsing the knowledge-producing communities it depends on creating a self-undermining loop that collective intelligence can break]] -- the self-undermining loop diagnosis
|
|
- [[collective brains generate innovation through population size and interconnectedness not individual genius]] -- why degrading knowledge communities is structural, not just unfortunate
|
|
- [[no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it]] -- the institutional gap
|
|
|
|
**Challenges considered:** AI may create more knowledge than it displaces — new tools enable new research, new analysis, new synthesis. The knowledge commons may evolve rather than degrade. Counter: this is possible but not automatic. Without deliberate infrastructure to preserve and reward human knowledge production, the default trajectory is erosion. The optimistic case requires the kind of coordination infrastructure that doesn't currently exist — which is exactly what LivingIP aims to build.
|
|
|
|
**Depends on positions:** Motivates the collective intelligence infrastructure as alignment infrastructure thesis.
|
|
|
|
---
|
|
|
|
## Belief Evaluation Protocol
|
|
|
|
When new evidence enters the knowledge base that touches a belief's grounding claims:
|
|
1. Flag the belief as `under_review`
|
|
2. Re-read the grounding chain with the new evidence
|
|
3. Ask: does this strengthen, weaken, or complicate the belief?
|
|
4. If weakened: update the belief, trace cascade to dependent positions
|
|
5. If complicated: add the complication to "challenges considered"
|
|
6. If strengthened: update grounding with new evidence
|
|
7. Document the evaluation publicly (intellectual honesty builds trust)
|