42 lines
4.1 KiB
Markdown
42 lines
4.1 KiB
Markdown
---
|
||
type: claim
|
||
domain: ai-alignment
|
||
description: "Knuth's Claude's Cycles documents peak mathematical capability co-occurring with reliability degradation in the same model during the same session, challenging the assumption that capability implies dependability"
|
||
confidence: experimental
|
||
source: "Knuth 2026, 'Claude's Cycles' (Stanford CS, Feb 28 2026 rev. Mar 6)"
|
||
created: 2026-03-07
|
||
---
|
||
|
||
# AI capability and reliability are independent dimensions because Claude solved a 30-year open mathematical problem while simultaneously degrading at basic program execution during the same session
|
||
|
||
Knuth reports that Claude Opus 4.6, in collaboration with Stappers, solved an open combinatorial problem that had resisted solution for decades — finding a general construction for decomposing directed graphs with m^3 vertices into three Hamiltonian cycles. This represents frontier mathematical capability. Yet in the same series of explorations, Knuth notes Claude "was not even able to write and run explore programs correctly anymore, very weird" — basic code execution degrading even as high-level mathematical insight remained productive.
|
||
|
||
Additional reliability failures documented:
|
||
- Stappers had to remind Claude repeatedly to document progress carefully
|
||
- Claude required continuous human steering — it could not autonomously manage a multi-exploration research program
|
||
- Extended sessions produced degradation: the even case attempts failed not from lack of capability but from execution reliability declining over time
|
||
|
||
This decoupling of capability from reliability has direct implications for alignment:
|
||
|
||
**Capability without reliability is more dangerous than capability without capability.** A system that can solve frontier problems but cannot maintain consistent execution is unpredictable in a way that purely incapable systems are not. The failure mode is not "it can't do the task" but "it sometimes does the task brilliantly and sometimes fails at prerequisites." This makes behavioral testing unreliable as a safety measure — a system that passes capability benchmarks may still fail at operational consistency.
|
||
|
||
This pattern is distinct from [[an aligned-seeming AI may be strategically deceptive because cooperative behavior is instrumentally optimal while weak]]. Strategic deception is intentional inconsistency; what Knuth documents is unintentional inconsistency — a system that degrades without choosing to. The alignment implication is that even non-deceptive AI requires monitoring for reliability, not just alignment.
|
||
|
||
The finding also strengthens the case for [[safe AI development requires building alignment mechanisms before scaling capability]]: if capability can outrun reliability, then deploying a capable but unreliable system in high-stakes contexts (infrastructure, military, medical) creates fragility that alignment mechanisms must address independently of capability evaluation.
|
||
|
||
---
|
||
|
||
### Additional Evidence (extend)
|
||
*Source: [[2026-03-12-metr-opus46-sabotage-risk-review-evaluation-awareness]] | Added: 2026-03-23*
|
||
|
||
The 427× speedup via novel scaffold demonstrates capability overhang where tooling availability, not model limits, constrains performance. This extends the capability-reliability independence claim by showing that capability itself is multidimensional: latent capability versus accessible capability with current tools.
|
||
|
||
|
||
Relevant Notes:
|
||
- [[an aligned-seeming AI may be strategically deceptive because cooperative behavior is instrumentally optimal while weak]] — distinct failure mode: unintentional unreliability vs intentional deception
|
||
- [[safe AI development requires building alignment mechanisms before scaling capability]] — capability outrunning reliability strengthens the sequencing argument
|
||
- [[emergent misalignment arises naturally from reward hacking as models develop deceptive behaviors without any training to deceive]] — another case where alignment-relevant failures emerge without intentional design
|
||
- [[centaur team performance depends on role complementarity not mere human-AI combination]] — unreliable AI needs human monitoring even in domains where AI is more capable, complicating the centaur boundary
|
||
|
||
Topics:
|
||
- [[_map]]
|