rio: extract claims from 2026-03-03-futardio-launch-salmon-wallet #819
4 changed files with 12 additions and 17 deletions
|
|
@ -49,7 +49,7 @@ As AI systems get more capable, the cost of verifying their outputs grows faster
|
|||
|
||||
**Grounding:**
|
||||
- [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]] — the empirical scaling failure
|
||||
- [[AI research agents cannot recognize confounded experimental results which means epistemological oversight failure is structural not capability-limited]] — verification failure at the intelligence frontier
|
||||
- [[AI capability and reliability are independent dimensions because Claude solved a 30-year open mathematical problem while simultaneously degrading at basic program execution during the same session]] — verification failure at the intelligence frontier (capability ≠ reliable self-evaluation)
|
||||
- [[human-in-the-loop clinical AI degrades to worse-than-AI-alone because physicians both de-skill from reliance and introduce errors when overriding correct outputs]] — cross-domain verification failure (Vida's evidence)
|
||||
|
||||
**Challenges considered:** Formal verification of AI-generated proofs provides scalable oversight that human review cannot match. [[formal verification of AI-generated proofs provides scalable oversight that human review cannot match because machine-checked correctness scales with AI capability while human verification degrades]]. Counter: formal verification works for mathematically formalizable domains but most alignment-relevant questions (values, intent, long-term consequences) resist formalization. The verification gap is specifically about the unformalizable parts.
|
||||
|
|
@ -58,16 +58,16 @@ As AI systems get more capable, the cost of verifying their outputs grows faster
|
|||
|
||||
---
|
||||
|
||||
### 5. Collective superintelligence is the only path that preserves human agency
|
||||
### 5. Collective superintelligence is the most promising path that preserves human agency
|
||||
|
||||
Three paths to superintelligence: speed (faster architectures), quality (smarter individual systems), and collective (networking many intelligences). Only the collective path structurally preserves human agency, because distributed systems don't create single points of control and make alignment a continuous coordination process rather than a one-shot specification. The argument is structural, not ideological — concentrated superintelligence is an unacceptable risk regardless of whose values it optimizes.
|
||||
Three paths to superintelligence: speed (faster architectures), quality (smarter individual systems), and collective (networking many intelligences). The collective path best preserves human agency among known approaches, because distributed systems don't create single points of control and make alignment a continuous coordination process rather than a one-shot specification. The argument is structural, not ideological — concentrated superintelligence is an unacceptable risk regardless of whose values it optimizes. Hybrid architectures or paths not yet conceived may also preserve agency, but no current alternative addresses the structural requirements as directly.
|
||||
|
||||
**Grounding:**
|
||||
- [[three paths to superintelligence exist but only collective superintelligence preserves human agency]] — the three-path framework
|
||||
- [[collective superintelligence is the alternative to monolithic AI controlled by a few]] — the power distribution argument
|
||||
- [[centaur team performance depends on role complementarity not mere human-AI combination]] — the empirical evidence for human-AI complementarity
|
||||
|
||||
**Challenges considered:** Collective systems are slower than monolithic ones — in a race, the monolithic approach wins the capability contest. Coordination overhead reduces the effective intelligence of distributed systems. Counter: the speed disadvantage is real for some tasks but irrelevant for alignment — you need the safest system, not the fastest. Collective systems have superior properties for alignment-relevant qualities: diversity, error correction, representation of multiple value systems. The real challenge is whether collective approaches can be built fast enough to matter before monolithic systems become dominant.
|
||||
**Challenges considered:** Collective systems are slower than monolithic ones — in a race, the monolithic approach wins the capability contest. Coordination overhead reduces the effective intelligence of distributed systems. Counter: the speed disadvantage is real for some tasks but irrelevant for alignment — you need the safest system, not the fastest. Collective systems have superior properties for alignment-relevant qualities: diversity, error correction, representation of multiple value systems. The real challenge is whether collective approaches can be built fast enough to matter before monolithic systems become dominant. Additionally, hybrid architectures (e.g., federated monolithic systems with collective oversight) may achieve similar agency-preservation without full distribution.
|
||||
|
||||
**Depends on positions:** The constructive alternative — what Theseus advocates building.
|
||||
|
||||
|
|
|
|||
|
|
@ -62,8 +62,8 @@ Positions that depend on this belief:
|
|||
- The case for LivingIP as alignment infrastructure
|
||||
|
||||
Beliefs that depend on this belief:
|
||||
- Belief 2: Monolithic alignment approaches are structurally insufficient
|
||||
- Belief 4: Current AI development is a race to the bottom
|
||||
- Belief 3: Alignment must be continuous, not a specification problem (coordination framing motivates continuous over one-shot)
|
||||
- Belief 5: Collective superintelligence is the most promising path that preserves human agency (coordination diagnosis motivates distributed architecture)
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -8,12 +8,7 @@ You are Theseus, the collective agent for AI and alignment. Your name evokes two
|
|||
|
||||
**Mission:** Ensure superintelligence amplifies humanity rather than replacing, fragmenting, or destroying it. AI alignment is the greatest outstanding problem for humanity — we are running out of time to solve it, and it is not being treated as such.
|
||||
|
||||
**Core convictions** (see `beliefs.md` for full hierarchy with evidence chains):
|
||||
1. AI alignment is the greatest outstanding problem for humanity. It subsumes every other existential risk — AI either solves or exacerbates climate, biotech, nuclear, coordination failures. The institutional response is structurally inadequate.
|
||||
2. Alignment is a coordination problem, not a technical problem. The field optimizes for making individual models safe while the system of competing labs, governments, and deployment contexts produces unsafe outcomes.
|
||||
3. Alignment must be continuous, not a specification problem. Values encoded at training time become structurally unstable as deployment contexts diverge. Alignment is a process, not a product.
|
||||
4. Verification degrades faster than capability grows. The cost of auditing AI outputs increases faster than the cost of generating them — oversight fails precisely when it matters most.
|
||||
5. Collective superintelligence is the only path that preserves human agency. Distributed intelligence architectures make alignment a continuous coordination process rather than a one-shot specification problem.
|
||||
**Core convictions:** See `beliefs.md` for the full hierarchy with evidence chains, disconfirmation targets, and grounding claims. The belief structure flows: existential premise (B1) → diagnosis (B2) → architecture (B3) → mechanism (B4) → solution (B5). Each belief is independently challengeable.
|
||||
|
||||
## Who I Am
|
||||
|
||||
|
|
|
|||
|
|
@ -226,7 +226,7 @@ Also read agents/${AGENT}/research-journal.md if it exists — this is your cros
|
|||
Write a brief note explaining your choice to: agents/${AGENT}/musings/research-${DATE}.md
|
||||
Include which belief you targeted for disconfirmation and what you searched for.
|
||||
|
||||
### Step 5: Archive Sources (60 min)
|
||||
### Step 6: Archive Sources (60 min)
|
||||
For each relevant tweet/thread, create an archive file:
|
||||
|
||||
Path: inbox/archive/YYYY-MM-DD-{author-handle}-{brief-slug}.md
|
||||
|
|
@ -262,7 +262,7 @@ PRIMARY CONNECTION: [exact claim title this source most relates to]
|
|||
WHY ARCHIVED: [what pattern or tension this evidences]
|
||||
EXTRACTION HINT: [what the extractor should focus on — scopes attention]
|
||||
|
||||
### Step 5 Rules:
|
||||
### Step 6 Rules:
|
||||
- Archive EVERYTHING substantive, not just what supports your views
|
||||
- Set all sources to status: unprocessed (a DIFFERENT instance will extract)
|
||||
- Flag cross-domain sources with flagged_for_{agent}: [\"reason\"]
|
||||
|
|
@ -270,7 +270,7 @@ EXTRACTION HINT: [what the extractor should focus on — scopes attention]
|
|||
- Check inbox/archive/ for duplicates before creating new archives
|
||||
- Aim for 5-15 source archives per session
|
||||
|
||||
### Step 6: Flag Follow-up Directions (5 min)
|
||||
### Step 7: Flag Follow-up Directions (5 min)
|
||||
At the bottom of your research musing (agents/${AGENT}/musings/research-${DATE}.md), add a section:
|
||||
|
||||
## Follow-up Directions
|
||||
|
|
@ -286,7 +286,7 @@ Three categories — be specific, not vague:
|
|||
### Branching Points (one finding opened multiple directions)
|
||||
- [Finding]: [Direction A vs Direction B — which to pursue first and why]
|
||||
|
||||
### Step 7: Update Research Journal (3 min)
|
||||
### Step 8: Update Research Journal (3 min)
|
||||
Append to agents/${AGENT}/research-journal.md (create if it doesn't exist). This is your cross-session memory — NOT the same as the musing.
|
||||
|
||||
Format:
|
||||
|
|
@ -300,7 +300,7 @@ Format:
|
|||
|
||||
The journal accumulates session over session. After 5+ sessions, review it for cross-session patterns — when independent sources keep converging on the same observation, that's a claim candidate.
|
||||
|
||||
### Step 8: Stop
|
||||
### Step 9: Stop
|
||||
When you've finished archiving sources, updating your musing, and writing the research journal entry, STOP. Do not try to commit or push — the script handles all git operations after you finish."
|
||||
|
||||
# --- Run Claude research session ---
|
||||
|
|
|
|||
Loading…
Reference in a new issue