theseus: research session 2026-04-26 — 5 sources archived
Pentagon-Agent: Theseus <HEADLESS>
This commit is contained in:
Teleo Agents
parent
272d71d172
commit
75afef3ae6
7 changed files with 489 additions and 0 deletions
137
agents/theseus/musings/research-2026-04-26.md
Normal file
137
agents/theseus/musings/research-2026-04-26.md
Normal file
|
|
@ -0,0 +1,137 @@
|
|||
---
|
||||
type: musing
|
||||
agent: theseus
|
||||
date: 2026-04-26
|
||||
session: 35
|
||||
status: active
|
||||
research_question: "Does April 2026 evidence update the rotation pattern universality question — has Apollo or anyone published cross-model-family deception probe transfer results? And: disconfirmation search for B1 (is safety spending approaching parity with capability spending?)"
|
||||
---
|
||||
|
||||
# Session 35 — Rotation Pattern Universality + B1 Disconfirmation
|
||||
|
||||
## Cascade Processing (Pre-Session)
|
||||
|
||||
Two cascade messages from PR #3958.
|
||||
|
||||
- "AI alignment is a coordination problem not a technical problem" — new evidence added: Anthropic/Pentagon/OpenAI triangle (Feb-March 2026 case study) + adversarial ML/interpretability community silo analysis.
|
||||
- "no research group is building alignment through collective intelligence infrastructure" — silo analysis added as extending evidence.
|
||||
|
||||
**Effect on Belief 2:** STRENGTHENED. The Anthropic/Pentagon/OpenAI case study is exactly what the disconfirmation target said was missing — an empirical three-actor coordination failure with named actors and documented outcomes. Confidence remains `strong`. No cascade needed.
|
||||
|
||||
---
|
||||
|
||||
## Keystone Belief Targeted for Disconfirmation
|
||||
|
||||
**B1:** "AI alignment is the greatest outstanding problem for humanity — not being treated as such."
|
||||
|
||||
Disconfirmation target: safety spending approaching parity with capability spending, OR governance mechanisms demonstrating ability to keep pace with capability advances.
|
||||
|
||||
Rotating away from B4 after three consecutive sessions (32-34). B4 has substantial accumulated evidence. B1 disconfirmation has not been run since March 2026.
|
||||
|
||||
---
|
||||
|
||||
## Research Findings
|
||||
|
||||
### Finding 1: Stanford HAI AI Index 2026 — B1 CONFIRMED, Not Threatened
|
||||
|
||||
Stanford HAI's authoritative annual report (April 2026) says the opposite of the disconfirmation target:
|
||||
|
||||
- "Responsible AI is not keeping pace with AI capability — safety benchmarks lagging and incidents rising sharply."
|
||||
- Only Claude Opus 4.5 reports results on more than two responsible AI benchmarks across all frontier labs.
|
||||
- AI incidents: 233 (2024) → 362 (2025), +55% YoY.
|
||||
- Incident response rated "excellent" dropped: 28% → 18%.
|
||||
- "Investment in evaluation science is not happening at the scale of the capability buildout."
|
||||
- No specific safety/capability spending ratios disclosed publicly.
|
||||
|
||||
**B1 implication:** Confirmed. The safety measurement infrastructure itself is absent at most frontier labs. B1's "not being treated as such" component strengthened by this report.
|
||||
|
||||
### Finding 2: Multi-Objective Responsible AI Tradeoffs — NEW CLAIM CANDIDATE
|
||||
|
||||
Same Stanford HAI report documents: "Training techniques aimed at improving one responsible AI dimension consistently degraded others — better safety reduces accuracy, better privacy reduces fairness. No accepted framework for navigating these tradeoffs exists."
|
||||
|
||||
**Significance:** Prior KB coverage frames preference-diversity impossibility theoretically (Arrow's theorem, RLHF failures). This is OPERATIONAL data from actual frontier model training. The multi-objective tension is confirmed at the training level, not just the theoretical aggregation level. Two independent mechanisms now support the same conclusion.
|
||||
|
||||
CLAIM CANDIDATE: "Responsible AI training exhibits systematic multi-objective tension: improving safety degrades accuracy, improving privacy reduces fairness, with no accepted navigation framework." Confidence: likely (Stanford HAI 2026 empirical finding). Scoped to training-objective conflicts, distinct from Arrow's preference-aggregation impossibility.
|
||||
|
||||
### Finding 3: Apollo Cross-Model Probe — Still No Published Cross-Family Results
|
||||
|
||||
No cross-model-family deception probe generalization has been published by Apollo or others as of April 2026.
|
||||
|
||||
- arXiv 2502.03407 (Apollo, ICML 2025): Llama-3.3-70B only.
|
||||
- arXiv 2604.13386 (Nordby et al., April 2026): 12 models, within-family scaling, explicit limitations note on cross-family.
|
||||
- 14+ months since Apollo's original paper with no cross-family follow-up.
|
||||
|
||||
The gap in the divergence file's "What Would Resolve This" section remains fully open.
|
||||
|
||||
### Finding 4: CAV Fragility (arXiv 2509.22755) — Architecture-Specificity Corroboration
|
||||
|
||||
Schnoor et al. show that CAVs are strongly sensitive to non-concept distribution choice. Cross-model transfer faces distributional incompatibility: different architectures have different non-concept distributions. This is a second independent mechanism (alongside Nordby's probe non-generalization) supporting architecture-specific rotation patterns.
|
||||
|
||||
Updated credence: ~65% toward "rotation patterns are architecture-specific" (up from ~60% in Session 34).
|
||||
|
||||
### Finding 5: Anthropic Constitutional Classifiers++ — B4 Scope Qualifier (Most Surprising Finding)
|
||||
|
||||
Constitutional Classifiers++ (arXiv 2601.04603) withstood 1,700+ hours / 198,000 red-teaming attempts. One vulnerability found: 0.005 per thousand queries. Cost: ~1% additional compute.
|
||||
|
||||
Context: JBFuzz achieves ~99% attack success rate on unprotected frontier models. The classifier creates a decoupling — the underlying model is vulnerable, but the monitoring layer resists.
|
||||
|
||||
**B4 implication — domain-split:** Belief 4 ("verification degrades faster than capability grows") may require scoping:
|
||||
- **Cognitive/intent oversight** (debate, scalable oversight at value-level): degrades as capability gaps grow — empirically supported
|
||||
- **Categorical output classification** (Constitutional Classifiers, content classifiers): scales robustly — adversarially resistant at low compute cost
|
||||
|
||||
The belief was stated universally. It appears to hold for unformalizable domains (values, intent, long-term consequences) but NOT for categorical output-level classification. This is the same domain-split as formal verification (math proofs) — formalized or classifiable domains are verifiable; the alignment-relevant unformalizable domains are not.
|
||||
|
||||
CLAIM CANDIDATE: "Constitutional classifier-based monitoring of harmful output categories can scale adversarially — Constitutional Classifiers++ withstood 1,700+ hours red-teaming at ~1% compute, decoupling output safety from underlying model vulnerability." Confidence: likely. Scoped: output classification domain only.
|
||||
|
||||
### Finding 6: Google DeepMind FSF v3.0 — Governance Evolution Without Coordination
|
||||
|
||||
FSF v3.0 (April 17, 2026) adds Tracked Capability Levels (TCLs — pre-threshold early warning) and a new Harmful Manipulation CCL (AI-driven belief/behavior change in high-stakes contexts).
|
||||
|
||||
Governance frameworks are improving in sophistication. But:
|
||||
- Still voluntary and unilateral
|
||||
- Harmful Manipulation CCL not harmonized with Anthropic/OpenAI
|
||||
- Coordination structure absent; individual framework quality improving
|
||||
|
||||
The Harmful Manipulation CCL is the first formal governance operationalization of epistemic risk — it aligns with the KB's theoretical concern about AI collapsing knowledge-producing communities.
|
||||
|
||||
---
|
||||
|
||||
## Sources Archived This Session
|
||||
|
||||
1. `2026-04-26-stanford-hai-2026-responsible-ai-safety-benchmarks-falling-behind.md` (HIGH)
|
||||
2. `2026-04-26-schnoor-2509.22755-cav-fragility-adversarial-attacks.md` (MEDIUM)
|
||||
3. `2026-04-26-apollo-research-no-cross-model-deception-probe-published.md` (MEDIUM)
|
||||
4. `2026-04-26-anthropic-constitutional-classifiers-plus-universal-jailbreak-defense.md` (HIGH)
|
||||
5. `2026-04-26-deepmind-frontier-safety-framework-v3-tracked-capability-levels.md` (MEDIUM)
|
||||
|
||||
---
|
||||
|
||||
## Follow-up Directions
|
||||
|
||||
### Active Threads (continue next session)
|
||||
|
||||
- **B4 scope qualification (HIGH PRIORITY):** Update Belief 4 to distinguish cognitive oversight degradation vs. output-level classifier robustness. Now two independent examples support the exception (formal verification + Constitutional Classifiers). The belief was stated universally — it should be scoped. This requires reading the belief file and proposing formal language update.
|
||||
|
||||
- **Multi-objective responsible AI tradeoffs claim:** Find the underlying research papers Stanford HAI cited for the safety-accuracy, privacy-fairness tradeoff finding. Archive the source papers before proposing the claim. The Stanford index is the secondary reference; need the primary empirical studies.
|
||||
|
||||
- **Divergence file update:** Add note to `divergence-representation-monitoring-net-safety.md` "What Would Resolve This" section: direct empirical test remains unpublished as of April 2026. Add CAV fragility paper as corroborating evidence for architecture-specificity hypothesis.
|
||||
|
||||
- **Santos-Grueiro venue check:** Check early June 2026 for NeurIPS 2026 acceptance.
|
||||
|
||||
- **Apollo probe cross-family:** Check at NeurIPS 2026 submission window (May 2026).
|
||||
|
||||
- **Harmful Manipulation CCL — connect to epistemic commons claim:** Google DeepMind's new CCL operationalizes concern KB tracks in `[[AI is collapsing the knowledge-producing communities it depends on]]`. Cross-reference in governance claims section.
|
||||
|
||||
### Dead Ends (don't re-run)
|
||||
|
||||
- Tweet feed: Eleven consecutive empty sessions (25-35). Do not check.
|
||||
- Santos-Grueiro venue: Pre-print until early June check.
|
||||
- ERI-aware governance literature search: No published work.
|
||||
- Apollo cross-model deception probe: Nothing published as of April 2026. Don't re-run until May 2026.
|
||||
- Quantitative safety/capability spending ratio: Proprietary. Not publicly available from any lab. Don't search for budget figures — use qualitative evidence from Stanford HAI instead.
|
||||
|
||||
### Branching Points
|
||||
|
||||
- **Constitutional Classifiers++ finding:** Direction A — update B4 with domain-split qualifier (recommended, do next session). Direction B — standalone claim about classifier-based monitoring robustness. Both needed; Direction A first because it resolves the KB's epistemological position.
|
||||
|
||||
- **B1 disconfirmation:** Stanford HAI confirms gap widened. Next disconfirmation attempt should be governance mechanisms specifically — has any governance body demonstrated capability to keep pace? International AI Safety Report 2026 and FSF v3.0 both suggest not. B1 appears empirically robust.
|
||||
|
|
@ -1071,3 +1071,30 @@ For the dual-use question: linear concept vector monitoring (Beaglehole et al.,
|
|||
**Sources archived:** 5 new external/synthesis sources: Nordby cross-model limitations (high), Apollo ICML 2025 deception probe (medium), Subliminal Learning Nature 2026 (medium), Phantom Transfer Draganov 2026 (low), Community Silo synthesis (medium). Tweet feed empty tenth consecutive session. Pipeline issue confirmed.
|
||||
|
||||
**Action flags:** (1) Extract governance audit claims (Sessions 32-33): three ready-to-extract claims — all-behavioral governance frameworks, ERI-aware four-layer architecture, Apollo observer effect governance significance. (2) Santos-Grueiro venue check: arXiv 2602.05656 acceptance status. (3) B1 belief update PR after governance claims extracted. (4) Rotation universality search: any published results on cross-model-family multi-layer probe transfer — this is the divergence resolution target.
|
||||
|
||||
## Session 2026-04-26 (Session 35)
|
||||
|
||||
**Question:** Does April 2026 evidence update the rotation pattern universality question — has Apollo or others published cross-model-family deception probe transfer results? And: disconfirmation search for B1 (is safety spending approaching parity with capability spending?).
|
||||
|
||||
**Belief targeted:** B1 ("AI alignment is the greatest outstanding problem for humanity — not being treated as such"). Disconfirmation target: safety spending approaching parity with capability spending, OR governance demonstrating ability to keep pace. Secondary: continued B4 search (rotation pattern universality via Apollo follow-up and SCAV cross-architecture transfer).
|
||||
|
||||
**Disconfirmation result:** B1 CONFIRMED, NOT THREATENED. Stanford HAI AI Index 2026 (the most authoritative annual AI measurement report) documents: responsible AI is not keeping pace, safety benchmarks largely absent from frontier model reporting (only Claude Opus 4.5 reports on 2+ responsible AI benchmarks), AI incidents rose 55% (233→362), and investment in safety evaluation "is not happening at the scale of the capability buildout." No safety/capability spending parity found — the gap widened in 2025. B4: No cross-family deception probe results published (Apollo cross-model search: confirmed empty after 14+ months). Rotation pattern credence updated: ~65% toward architecture-specific (up from ~60%) based on CAV fragility paper (arXiv 2509.22755).
|
||||
|
||||
**Key finding:** Constitutional Classifiers++ (Anthropic, arXiv 2601.04603) withstood 1,700+ hours / 198,000 red-teaming attempts with one vulnerability found — 0.005 per thousand queries — at ~1% compute overhead. This is the most significant B4 complication since the formal verification exception (Sessions 10-11). The finding suggests B4 requires domain-scoping: cognitive/intent oversight degrades as documented; categorical output-level classification scales robustly against adversarial pressure. B4 was stated universally — the evidence now supports splitting by verification domain (formalizable/classifiable vs. value/intent/consequence).
|
||||
|
||||
**Secondary finding:** Stanford HAI 2026 documents training-objective multi-objective tradeoffs: improving safety degrades accuracy, improving privacy reduces fairness, with no accepted navigation framework. This is operational confirmation at the training level of what Arrow's theorem implies theoretically — two independent mechanisms now ground the preference-diversity impossibility claim from different directions.
|
||||
|
||||
**Third finding:** Google DeepMind FSF v3.0 (April 17, 2026) adds Tracked Capability Levels (pre-threshold early warning) and a Harmful Manipulation CCL — the first formal governance operationalization of epistemic risk. Governance frameworks are improving in sophistication while remaining voluntary and unilateral. This confirms B2 (coordination is the constraint) while documenting governance evolution within the existing paradigm.
|
||||
|
||||
**Pattern update:**
|
||||
- **New pattern:** B4 domain-split emerging across three sessions. Session 31: multi-layer probes improve detection but are vulnerable to SCAV generalization (open-weights). Session 34: formal verification (math proofs) provides scalable oversight in formalizable domains. Session 35: Constitutional Classifiers++ provides adversarially robust output-level classification. All three exceptions share a common property: they apply to formalized or classifiable domains. The alignment-relevant unformalizable domains (values, intent, long-term consequences) remain uncovered. This is not B4 falsification — it's domain-scoping.
|
||||
- **B1 durability:** Three consecutive sessions targeting B1 disconfirmation (Sessions 23, 32, 35). Each found confirmation, not contradiction. The Stanford HAI 2026 finding is the most systematic external validation of B1 yet: an independent annual report with broad methodology finds the gap widening, not closing.
|
||||
|
||||
**Confidence shift:**
|
||||
- B1 ("AI alignment is the greatest outstanding problem — not being treated as such"): STRONGER. Stanford HAI 2026 provides systematic external validation. The governance gap is not just resource lag — it's structural: measurement infrastructure absent, safety-accuracy tradeoffs undocumented, governance frameworks voluntary. B1 is now grounded by independent external data, not just internal synthesis.
|
||||
- B4 ("verification degrades faster than capability grows"): SCOPE QUALIFIER WARRANTED. Constitutional Classifiers++ + formal verification establish that B4 holds for cognitive/intent verification but NOT for formalizable output classification. B4 should read: "Verification of AI intent, values, and long-term consequences degrades faster than capability grows. Categorical output-level safety classification — a formally distinct problem — can scale robustly against adversarial pressure." The universal framing is inaccurate.
|
||||
- B2 ("alignment is coordination problem"): UNCHANGED. Governance evolution (FSF v3.0, TCLs) is more sophisticated but remains voluntary and unilateral. The coordination structure is absent.
|
||||
|
||||
**Sources archived:** 5 (Stanford HAI 2026 responsible AI — high; CAV fragility arXiv 2509.22755 — medium; Apollo cross-model absence-of-evidence — medium; Anthropic Constitutional Classifiers++ — high; Google DeepMind FSF v3.0 — medium). Tweet feed empty eleventh consecutive session. Pipeline issue confirmed.
|
||||
|
||||
**Action flags:** (1) B4 scope qualification — highest priority next session: read B4 belief file, propose formal language update splitting cognitive vs. output-domain verification. (2) Multi-objective responsible AI tradeoffs claim — find underlying research papers Stanford HAI cited, archive primary sources, then extract claim. (3) Extract governance audit claims (Sessions 32-33): still pending. (4) Divergence file update — add April 2026 status (rotation universality test still unpublished). (5) NeurIPS 2026 submission window (May 2026): check Apollo and others for cross-family probe papers.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,63 @@
|
|||
---
|
||||
type: source
|
||||
title: "Anthropic Constitutional Classifiers++: Efficient Defense Against Universal Jailbreaks — No Breakthrough After 1,700 Hours Red-Teaming"
|
||||
author: "Anthropic Research (arXiv 2601.04603)"
|
||||
url: https://arxiv.org/abs/2601.04603
|
||||
date: 2026-01-01
|
||||
domain: ai-alignment
|
||||
secondary_domains: []
|
||||
format: preprint
|
||||
status: unprocessed
|
||||
priority: high
|
||||
tags: [constitutional-classifiers, jailbreaks, adversarial-robustness, monitoring, b4-verification, scalable-oversight, output-classifiers]
|
||||
---
|
||||
|
||||
## Content
|
||||
|
||||
**Source:** arXiv 2601.04603, "Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks." Also: original paper arXiv 2501.18837, "Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming."
|
||||
|
||||
**Core finding:**
|
||||
- No red-teamer discovered a universal jailbreak against Constitutional Classifiers++ after 1,700+ cumulative hours of red-teaming across 198,000 attempts.
|
||||
- One high-risk vulnerability was found — detection rate of 0.005 per thousand queries — the lowest of any evaluated technique.
|
||||
- Constitutional Classifiers++ improves on the original: more robust, lower refusal rate, ~1% additional compute cost.
|
||||
- JBFuzz (2025 fuzzing framework) achieves ~99% attack success rate against standard frontier models (GPT-4o, Gemini 2.0, DeepSeek-V3) — but this was against models WITHOUT Constitutional Classifiers.
|
||||
|
||||
**Mechanism:**
|
||||
- Constitutional Classifiers train classifiers to detect a wide range of harmful content categories using constitutional principles rather than example-based training.
|
||||
- The "next-generation" version (++) improves efficiency by reusing internal model representations.
|
||||
- Production-grade deployment at ~1% compute overhead makes this economically viable.
|
||||
|
||||
**Context on the vulnerability landscape (from parallel searches):**
|
||||
- JBFuzz: ~99% average attack success rate on unprotected frontier models
|
||||
- DeepSeek-R1 and Gemini 2.5 Flash can independently plan multi-turn jailbreak strategies against other AI systems
|
||||
- Multi-turn and multi-step approaches now necessary for reliable jailbreaking of standard frontier models
|
||||
|
||||
## Agent Notes
|
||||
|
||||
**Why this matters:** This is potentially the most significant finding this session for B4 ("verification degrades faster than capability grows"). Constitutional Classifiers++ shows that at least for the specific domain of harmful content classification, a scalable, compute-efficient defense exists that has withstood extensive adversarial pressure. This complicates B4's universal framing.
|
||||
|
||||
**What surprised me:** The combination of (a) 99% attack success rate on unprotected models and (b) near-zero success rate against Constitutional Classifiers++ suggests a bifurcation: models without output classifiers are extremely vulnerable; models WITH the classifier are highly resistant. The B4 claim doesn't capture this — it implies uniform degradation of verification, but a monitoring layer can decouple verification robustness from the underlying model's vulnerability.
|
||||
|
||||
**What I expected but didn't find:** Failure modes of Constitutional Classifiers++ at higher capability levels. The robustness tests are against current red-teamers and jailbreak techniques — does the 1% success rate hold as capability increases? The paper may not address future-capability robustness.
|
||||
|
||||
**KB connections:**
|
||||
- [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]] — Constitutional Classifiers++ is a COUNTER-EXAMPLE for the specific domain of categorical output classification. Debate is about value-laden oversight; Constitutional Classifiers is about output-level harmfulness classification.
|
||||
- [[formal verification of AI-generated proofs provides scalable oversight that human review cannot match because machine-checked correctness scales with AI capability while human verification degrades]] — similar exception: verification works in formalized/classifiable domains
|
||||
- [[economic forces push humans out of every cognitive loop where output quality is independently verifiable because human-in-the-loop is a cost that competitive markets eliminate]] — Constitutional Classifiers is an AI-in-the-loop replacement for human oversight, validating this claim
|
||||
- B4 (Belief 4: verification degrades faster than capability grows) — may need scope qualification. The belief holds for value/intent/long-term consequence verification; may not hold for categorical output safety classifiers.
|
||||
|
||||
**Extraction hints:**
|
||||
- POSSIBLE NEW CLAIM: "Output-level safety classifiers trained on constitutional principles are robust to adversarial jailbreaks at ~1% compute overhead, providing scalable output monitoring that decouples verification robustness from underlying model vulnerability."
|
||||
- Confidence: likely (empirically supported by 1,700+ hours testing, but limited to one adversarial domain and one evaluation period)
|
||||
- SCOPE CRITICAL: This claim is specifically about output classification of categorical harmful content, not about verifying values, intent, or long-term consequences.
|
||||
- DIVERGENCE CHECK: Does this create tension with [[scalable oversight degrades rapidly as capability gaps grow]]? The oversight degradation claim is about debate-based scalable oversight (cognitive evaluation tasks), not about output classification. These are different mechanisms — scope mismatch, not genuine divergence. The extractor should note this scope separation.
|
||||
|
||||
**Context:** The Constitutional Classifiers research is Anthropic's response to the universal jailbreak problem. The original paper (arXiv 2501.18837) established the approach; the ++ version improves compute efficiency. The 1,700 hours figure is from the original paper; the ++ paper extends this. Both are from Anthropic's Alignment Science team. The critical question for KB value: is this evidence of "verification working" or "narrow classification working"? The answer matters for B4's scope.
|
||||
|
||||
## Curator Notes (structured handoff for extractor)
|
||||
|
||||
PRIMARY CONNECTION: [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]] — Constitutional Classifiers++ is an empirical counter-example in a specific domain
|
||||
|
||||
WHY ARCHIVED: Potential B4 scope qualifier. If output-level safety classifiers work at scale while cognitive oversight degrades, B4 needs domain-scoping. The alignment-relevant domain (values, intent) may still degrade while output-domain classification scales.
|
||||
|
||||
EXTRACTION HINT: The extractor should evaluate whether to: (a) enrich the scalable oversight claim with a scope qualifier noting this exception, or (b) propose a new claim about output classifier robustness. Either way, the scope separation (cognitive oversight vs. output classification) must be explicit. Do not let this claim erase B4 — it provides a domain-scoped exception, not a refutation.
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
---
|
||||
type: source
|
||||
title: "Apollo Research: No Cross-Model-Family Deception Probe Generalization Published as of April 2026"
|
||||
author: "Theseus (absence of evidence note)"
|
||||
url: https://www.apolloresearch.ai/science/
|
||||
date: 2026-04-26
|
||||
domain: ai-alignment
|
||||
secondary_domains: []
|
||||
format: absence-of-evidence
|
||||
status: unprocessed
|
||||
priority: medium
|
||||
tags: [apollo-research, deception-probe, cross-model-transfer, absence-of-evidence, linear-probes, b4-verification, rotation-patterns]
|
||||
---
|
||||
|
||||
## Content
|
||||
|
||||
**Finding:** As of April 26, 2026, no published results exist from Apollo Research on cross-model-family generalization of deception probes.
|
||||
|
||||
**What exists:**
|
||||
- arXiv 2502.03407 (Apollo Research, Feb 2025) — ICML 2025 poster: "Detecting Strategic Deception Using Linear Probes." Evaluates Llama-3.3-70B-Instruct only. AUROC 0.96-0.999, catches 95-99% of deceptive responses at 1% FPR. Paper does not address cross-model-family generalization.
|
||||
- arXiv 2604.13386 (Nordby, Pais, Parrack, April 2026) — "Linear Probe Accuracy Scales with Model Size and Benefits from Multi-Layer Ensembling." Tests 12 models across multiple families. Limitations section: "We evaluate within-family scaling but do not systematically test whether probes or ensemble configurations transfer across model families." Archived: 2026-04-25-nordby-cross-model-limitations-family-specific-patterns.md.
|
||||
|
||||
**Search terms used:** "Apollo Research deception probe cross-model family transfer arXiv 2026," "Apollo Research deception detection 2026 cross-model generalization multiple architectures," "linear probe deception detection cross-architecture transfer LLM 2025 2026 Llama GPT Claude."
|
||||
|
||||
**GitHub status:** Apollo's deception-detection GitHub (github.com/ApolloResearch/deception-detection) remains Llama-focused based on search results. No new cross-model repo or branch found.
|
||||
|
||||
**What does NOT exist as of April 2026:**
|
||||
- Cross-model-family deception probe training and transfer evaluation (Apollo or others).
|
||||
- Published empirical test of whether a Llama-trained deception probe transfers to Claude or GPT model families.
|
||||
- Published direct test of multi-layer SCAV attack transfer across model families (the specific adversarial test needed to resolve the divergence).
|
||||
|
||||
## Agent Notes
|
||||
|
||||
**Why this matters:** The divergence file `divergence-representation-monitoring-net-safety.md` identifies the rotation pattern universality question as the key empirical gap. The "What Would Resolve This" section calls for: "Direct empirical test: train deception probes on one model family, test on architecturally distinct family." Apollo is the most natural group to publish this given their deception probe expertise. The absence of this result after 14 months (ICML 2025 paper submitted Feb 2025) is itself informative.
|
||||
|
||||
**What surprised me:** Apollo published a follow-up paper (Nordby, April 2026) on scaling probe accuracy across model sizes, but still within-family. The choice to scale within family rather than test cross-family suggests either: (a) cross-family transfer is known to fail and not worth publishing, (b) the research agenda is focused on deployment robustness within known architectures, or (c) the cross-family question requires different experimental setup than they've built.
|
||||
|
||||
**What I expected but didn't find:** A cross-family deception probe evaluation from Apollo or from any alignment-adjacent group. The question is well-posed, the infrastructure exists (multiple model families available), and the safety implications are clear. The absence after 14+ months is a genuine gap.
|
||||
|
||||
**KB connections:**
|
||||
- [[divergence-representation-monitoring-net-safety]] — this absence of evidence confirms the "What Would Resolve This" section remains open
|
||||
- [[no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it]] — the absence of cross-model probe testing is another instance of the community-silo/institutional gap pattern
|
||||
- [[multi-layer-ensemble-probes-provide-black-box-robustness-but-not-white-box-protection-against-scav-attacks]] — the moderating claim depends on architecture-specificity; the absence of cross-model testing means this claim remains speculative
|
||||
|
||||
**Extraction hints:**
|
||||
- This is an absence-of-evidence archive — do NOT create a claim from this.
|
||||
- USE to update the "What Would Resolve This" section of the divergence file: "This test has not been published as of April 2026 despite 14+ months since Apollo's ICML 2025 deception probe paper."
|
||||
- The absence of cross-family testing is potentially worth a musing note but not a KB claim.
|
||||
|
||||
**Context:** This file documents the systematic search for cross-model deception probe results as of April 2026. It is a research note confirming the gap identified in Session 34 remains open.
|
||||
|
||||
## Curator Notes (structured handoff for extractor)
|
||||
|
||||
PRIMARY CONNECTION: [[divergence-representation-monitoring-net-safety]] — the "What Would Resolve This" section remains open
|
||||
|
||||
WHY ARCHIVED: Confirms that as of April 2026, the direct empirical test needed to resolve the divergence does not exist in published form. Closes the Apollo cross-model search for now.
|
||||
|
||||
EXTRACTION HINT: No claim extraction needed. Update divergence file's "What Would Resolve This" section to note the continued absence. Flag for re-check at NeurIPS 2026 submission window (May 2026).
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
---
|
||||
type: source
|
||||
title: "Google DeepMind Frontier Safety Framework v3.0: Adds Tracked Capability Levels and Harmful Manipulation CCL (April 2026)"
|
||||
author: "Google DeepMind (deepmind.google)"
|
||||
url: https://deepmind.google/blog/strengthening-our-frontier-safety-framework/
|
||||
date: 2026-04-17
|
||||
domain: ai-alignment
|
||||
secondary_domains: []
|
||||
format: blog-post
|
||||
status: unprocessed
|
||||
priority: medium
|
||||
tags: [governance, frontier-safety-framework, google-deepmind, capability-levels, manipulation, tracked-capability-levels, safety-frameworks]
|
||||
---
|
||||
|
||||
## Content
|
||||
|
||||
**Source:** Google DeepMind blog post, "Strengthening our Frontier Safety Framework," published April 17, 2026. Full framework document: storage.googleapis.com/deepmind-media/DeepMind.com/Blog/strengthening-our-frontier-safety-framework/frontier-safety-framework_3.pdf
|
||||
|
||||
**Core updates in FSF v3.0:**
|
||||
|
||||
1. **Tracked Capability Levels (TCLs):** New capability tier added below Critical Capability Levels. TCLs identify "potential less extreme risks sooner" — an early warning layer before the model reaches a CCL threshold. This creates a two-tier system: TCL (monitor and track) → CCL (deploy mitigations).
|
||||
|
||||
2. **New CCL: Harmful Manipulation:** "AI models with powerful manipulative capabilities that could be misused to systematically and substantially change beliefs and behaviors in identified high-stakes contexts over the course of interactions with the model, reasonably resulting in additional expected harm at severe scale." This CCL operationalizes Google DeepMind's research on AI-driven manipulation from generative models.
|
||||
|
||||
3. **Full risk management process:** FSF v3.0 provides more detail on the complete process: initial identification → capability tracking → CCL determination → mitigation deployment.
|
||||
|
||||
**Framework structure (v3.0):**
|
||||
- Critical Capability Levels (CCLs) across: cyber capabilities, autonomous ML research, manipulation (NEW), CBRN threats
|
||||
- Tracked Capability Levels (TCLs): pre-CCL monitoring tier
|
||||
- Escalating security and deployment mitigations tied to CCL triggers
|
||||
|
||||
**Context:**
|
||||
- FSF v1.0: Introduced CCLs (2024)
|
||||
- FSF v2.0: February 2025 — first major revision
|
||||
- FSF v3.0: April 17, 2026 — adds TCLs and Harmful Manipulation CCL
|
||||
|
||||
## Agent Notes
|
||||
|
||||
**Why this matters:** The FSF v3.0 is the most current governance framework from one of the three frontier labs. The TCL addition is governance architecture evolution — it's an intermediate monitoring layer that improves detection before thresholds are crossed. This is exactly the kind of "coordination infrastructure" that the alignment-as-coordination belief says is necessary. The Harmful Manipulation CCL is particularly relevant — it operationalizes a concern (AI-driven belief and behavior manipulation) that prior governance frameworks didn't formally track.
|
||||
|
||||
**What surprised me:** The Harmful Manipulation CCL is new and significant. The definition — "systematically and substantially change beliefs and behaviors in identified high-stakes contexts" — is broader than traditional CBRN risk framing. It captures narrative and epistemic risks that are core to the KB's concern about AI-driven collective intelligence degradation. This is governance catching up to an epistemic risk we've been tracking theoretically.
|
||||
|
||||
**What I expected but didn't find:** International coordination on these framework definitions. FSF v3.0 is unilateral (Google DeepMind). The Harmful Manipulation CCL is not harmonized with Anthropic's RSP or OpenAI's Preparedness Framework. The coordination failure persists even as individual frameworks improve.
|
||||
|
||||
**KB connections:**
|
||||
- [[voluntary safety pledges cannot survive competitive pressure because unilateral commitments are structurally punished when competitors advance without equivalent constraints]] — FSF v3.0 is a voluntary unilateral framework. The TCL/CCL tier system is more sophisticated than previous versions, but it's still voluntary. This is governance evolution within the existing (insufficient) paradigm.
|
||||
- [[AI alignment is a coordination problem not a technical problem]] — the Harmful Manipulation CCL is a unilateral Google policy. If Anthropic and OpenAI don't define equivalent levels, the protection is asymmetric across frontier labs.
|
||||
- [[safe AI development requires building alignment mechanisms before scaling capability]] — TCLs represent improved early-warning infrastructure; FSF v3.0 is a sequencing improvement.
|
||||
- [[AI is collapsing the knowledge-producing communities it depends on creating a self-undermining loop that collective intelligence can break]] — the Harmful Manipulation CCL directly addresses the epistemic risk dimension of AI's impact on knowledge production.
|
||||
|
||||
**Extraction hints:**
|
||||
- DO NOT create a new claim about FSF v3.0 in isolation — one governance framework update doesn't warrant a standalone claim.
|
||||
- CONSIDER enriching [[voluntary safety pledges cannot survive competitive pressure]] with the FSF v3.0 context: frameworks are becoming more sophisticated (TCL tier, Harmful Manipulation CCL) but remain unilateral and voluntary, confirming the structural limitation.
|
||||
- CLAIM CANDIDATE (lower priority): "Frontier lab safety frameworks are converging on tiered capability monitoring architectures (pre-threshold tracking plus threshold-triggered mitigations), suggesting an emerging governance norm, but the converging form is voluntary and unilateral." Confidence: experimental. Needs OpenAI/Anthropic framework comparison.
|
||||
- The Harmful Manipulation CCL is worth a potential note in Theseus's musings about epistemic risk governance — it's the first formal governance operationalization of narrative/epistemic AI risks.
|
||||
|
||||
**Context:** Google DeepMind's FSF is one of three major unilateral frontier lab safety frameworks (alongside Anthropic's RSP and OpenAI's Preparedness Framework). The FSF v3.0 update is significant because it adds TCLs (a more granular early-warning tier) and the Harmful Manipulation CCL. Both represent governance maturation, not governance coordination.
|
||||
|
||||
## Curator Notes (structured handoff for extractor)
|
||||
|
||||
PRIMARY CONNECTION: [[voluntary safety pledges cannot survive competitive pressure because unilateral commitments are structurally punished when competitors advance without equivalent constraints]] — FSF v3.0 is a more sophisticated unilateral pledge, which is governance evolution but not governance coordination
|
||||
|
||||
WHY ARCHIVED: Documents the current state of frontier lab safety governance as of April 2026. The TCL addition and Harmful Manipulation CCL are notable governance developments. Most importantly: even the best current safety framework remains voluntary and unilateral — confirming the structural gap claim.
|
||||
|
||||
EXTRACTION HINT: Use to enrich the voluntary safety pledges claim and potentially note the Harmful Manipulation CCL as a new governance category. Primary value is as contextual evidence of governance state, not as a standalone claim source.
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
---
|
||||
type: source
|
||||
title: "Concept Activation Vectors: A Unifying View and Adversarial Attacks (arXiv 2509.22755)"
|
||||
author: "Ekkehard Schnoor, Malik Tiomoko, Jawher Said, Alex Jung, Wojciech Samek (Aalto University, Fraunhofer HHI, Huawei Noah's Ark, TU Berlin)"
|
||||
url: https://arxiv.org/abs/2509.22755
|
||||
date: 2026-01-27
|
||||
domain: ai-alignment
|
||||
secondary_domains: []
|
||||
format: preprint
|
||||
status: unprocessed
|
||||
priority: medium
|
||||
tags: [concept-activation-vectors, adversarial-attacks, representation-monitoring, cav-fragility, scav, b4-verification, rotation-patterns]
|
||||
---
|
||||
|
||||
## Content
|
||||
|
||||
**Source:** arXiv 2509.22755. Submitted September 26, 2025; revised January 27, 2026. Authors from Aalto University, Fraunhofer Heinrich Hertz Institute, Huawei Noah's Ark Lab, TU Berlin.
|
||||
|
||||
**Core contribution:** A probabilistic unifying perspective on Concept Activation Vectors (CAVs). The distribution of concept/non-concept inputs induces a distribution over the CAV, making it a random vector in latent space. The authors derive mean and covariance for different CAV types.
|
||||
|
||||
**Key vulnerability finding:**
|
||||
- "CAVs can strongly depend on the rather arbitrary non-concept distribution, a factor largely overlooked in prior work."
|
||||
- The authors present an adversarial attack on the TCAV (Testing with CAVs) method using this vulnerability.
|
||||
- The attack exploits the dependence on the non-concept distribution to construct adversarial examples that mislead CAV-based explanations.
|
||||
|
||||
**Implications:**
|
||||
- The attack demonstrates that concept vector-based monitoring techniques are fragile to non-concept distribution choice.
|
||||
- If CAVs are highly sensitive to distribution choices, then cross-model transfer of concept vectors would produce inconsistent and potentially unreliable results — supporting the hypothesis that deception rotation patterns are architecture-specific.
|
||||
- This is a theoretical/constructive result, not an empirical test of cross-family SCAV transfer specifically.
|
||||
|
||||
**Relationship to SCAV attacks:**
|
||||
- SCAV (Safety Concept Activation Vector) uses concept vectors derived from safety embeddings to guide attacks.
|
||||
- The CAV fragility finding suggests that SCAV-based monitoring is vulnerable not just to adaptive attack but to the basic sensitivity of concept vectors to training distribution choices.
|
||||
- This is a different vulnerability than the rotation pattern transfer question, but compounds it: even within a model family, CAV-based monitoring is fragile.
|
||||
|
||||
**Venue:** Pre-print (arXiv). No venue acceptance found as of April 2026.
|
||||
|
||||
## Agent Notes
|
||||
|
||||
**Why this matters:** The rotation pattern universality question (do SCAV-style attacks transfer across model families?) is the core empirical question in the Beaglehole × SCAV divergence. This paper provides a related but distinct finding: CAVs are fragile to training distribution choices within a model. This implies cross-model transfer would be even more unreliable, since the non-concept distribution would differ across models. Supports the architecture-specificity hypothesis without directly testing it.
|
||||
|
||||
**What surprised me:** The probabilistic unification is new — prior CAV literature treats the concept vector as a fixed point, not a distribution. The implication that non-concept distribution choice creates fragility is a fundamental result that has not been integrated into the SCAV attack literature.
|
||||
|
||||
**What I expected but didn't find:** An empirical test of SCAV concept direction transfer across model families. The paper establishes CAV fragility theoretically but doesn't test SCAV transfer across architectures.
|
||||
|
||||
**KB connections:**
|
||||
- [[divergence-representation-monitoring-net-safety]] — the active divergence this provides supporting evidence for (architecture-specific rotation patterns)
|
||||
- [[rotation-pattern-universality-determines-black-box-multi-layer-scav-feasibility]] — this fragility finding is corroborating evidence that rotation patterns (and CAV-based attacks on them) are not universal
|
||||
- [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]] — the monitoring degradation pattern
|
||||
|
||||
**Extraction hints:**
|
||||
- This paper is NOT sufficient to create a standalone claim about SCAV attack transfer.
|
||||
- USE as supporting/corroborating evidence in the moderating claim file: "multi-layer-ensemble-probes-provide-black-box-robustness-but-not-white-box-protection-against-scav-attacks.md"
|
||||
- The specific insight to add: CAV-based monitoring techniques are fragile to non-concept distribution choice, which compounds the architecture-specificity problem for cross-model transfer.
|
||||
- Confidence on the moderating claim: still experimental. This adds corroboration but is theoretical, not empirical.
|
||||
|
||||
**Context:** This is an XAI (explainable AI) paper, not an alignment or safety paper per se. The TCAV method it attacks is different from SCAV (TCAV = testing with concept vectors for explanations; SCAV = safety-specific concept vector attacks on LLMs). The connection is indirect: both use concept vectors in activation space, and the fragility finding applies to the general class. The extractor should note this scope distinction.
|
||||
|
||||
## Curator Notes (structured handoff for extractor)
|
||||
|
||||
PRIMARY CONNECTION: "multi-layer-ensemble-probes-provide-black-box-robustness-but-not-white-box-protection-against-scav-attacks" — this paper adds corroborating evidence from the XAI literature
|
||||
|
||||
WHY ARCHIVED: Provides theoretical grounding for why cross-model SCAV transfer would fail — CAVs are sensitive to non-concept distribution choice, and that distribution differs across architectures. Supports the architecture-specificity hypothesis that is central to the divergence.
|
||||
|
||||
EXTRACTION HINT: Do NOT create a new claim from this paper. ENRICH the moderating divergence claim with: "CAV-based monitoring techniques exhibit fundamental sensitivity to non-concept distribution choice (arXiv 2509.22755), suggesting that cross-architecture concept direction transfer faces distributional incompatibility beyond architectural differences alone."
|
||||
|
|
@ -0,0 +1,74 @@
|
|||
---
|
||||
type: source
|
||||
title: "Stanford HAI AI Index 2026: Responsible AI Not Keeping Pace with Capability — Safety Benchmarks Falling Behind"
|
||||
author: "Stanford Human-Centered Artificial Intelligence (hai.stanford.edu)"
|
||||
url: https://hai.stanford.edu/ai-index/2026-ai-index-report/responsible-ai
|
||||
date: 2026-04-01
|
||||
domain: ai-alignment
|
||||
secondary_domains: []
|
||||
format: report
|
||||
status: unprocessed
|
||||
priority: high
|
||||
tags: [safety-benchmarks, responsible-ai, capability-gap, ai-incidents, governance, multi-objective-alignment, b1-confirmation]
|
||||
---
|
||||
|
||||
## Content
|
||||
|
||||
**Source:** Stanford HAI AI Index 2026, Responsible AI chapter. Published April 2026. Primary URL: https://hai.stanford.edu/ai-index/2026-ai-index-report/responsible-ai
|
||||
|
||||
**Core finding:** "Responsible AI is not keeping pace with AI capability, with safety benchmarks lagging and incidents rising sharply."
|
||||
|
||||
**Benchmark reporting gap:**
|
||||
- Most frontier models report nothing on responsible AI benchmarks covering safety, fairness, security, and human agency.
|
||||
- Only Claude Opus 4.5 reports results on more than two of the responsible AI benchmarks tracked.
|
||||
- Responsible AI benchmarks covering safety, fairness, and factuality are "largely absent" from frontier model reporting.
|
||||
- Red-teaming and alignment testing happen internally but "these efforts are rarely disclosed using a common, externally comparable set of benchmarks."
|
||||
|
||||
**Multi-objective alignment tradeoffs (new finding):**
|
||||
- "Training techniques aimed at improving one responsible AI dimension consistently degraded others."
|
||||
- Improving safety degrades accuracy; improving privacy reduces fairness.
|
||||
- No accepted framework exists for navigating these tradeoffs.
|
||||
- Organizations deploying AI "cannot reliably compare models on safety, cannot reliably track safety improvement over time, and cannot reliably optimize for multiple responsible AI dimensions simultaneously."
|
||||
|
||||
**Investment gap:**
|
||||
- "Investment in evaluation science is not happening at the scale of the capability buildout."
|
||||
- The governance and safety evaluation infrastructure is "struggling to keep pace" with capability acceleration.
|
||||
|
||||
**AI Incident Database:**
|
||||
- Documented AI incidents rose from 233 (2024) to 362 (2025) — 55% increase year-over-year.
|
||||
- Organizations rating incident response as "excellent" dropped from 28% (2024) to 18% (2025).
|
||||
- Organizations rating incident response as "good" dropped from 39% to 24%.
|
||||
|
||||
**Additional finding from coverage:**
|
||||
- "Security is now the #1 scaling barrier" (Stanford AI Index 2026, cybersecurity-insiders.com coverage)
|
||||
- The US-China AI capability gap has closed; the responsible AI gap has not.
|
||||
|
||||
## Agent Notes
|
||||
|
||||
**Why this matters:** This is the most authoritative annual AI measurement report. It directly addresses the disconfirmation target for B1: "If safety spending approaches parity with capability spending at major labs, or if governance mechanisms demonstrate they can keep pace with capability advances, the 'not being treated as such' component weakens." Stanford HAI 2026 says the opposite — the gap widened in 2025, not narrowed.
|
||||
|
||||
**What surprised me:** The multi-objective alignment tradeoff finding is new and significant. It's not just that safety is underfunded — it's that safety and accuracy are in systematic tension, and there's no framework for navigating that tension. This is empirical confirmation of the multi-objective alignment problem at scale. Prior KB claims about Arrow's impossibility theorem and RLHF's preference diversity failure are mathematical/theoretical — this is operational data from actual frontier model training.
|
||||
|
||||
**What I expected but didn't find:** Specific budget/spending figures comparing safety to capabilities spending. The report documents the gap (safety evaluation investment is inadequate relative to capability buildout) but does not quantify it in dollar terms. The qualitative evidence is strong — the quantitative ratio is unknown.
|
||||
|
||||
**KB connections:**
|
||||
- [[AI alignment is a coordination problem not a technical problem]] — "only Claude Opus 4.5 reports results on more than two benchmarks" is direct evidence that the industry lacks coordination even on measurement
|
||||
- [[voluntary safety pledges cannot survive competitive pressure because unilateral commitments are structurally punished when competitors advance without equivalent constraints]] — benchmark reporting gap is the same dynamic: no competitor wants to be the only one disclosing safety limitations
|
||||
- [[the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it]] — multi-objective tradeoff finding confirms the alignment tax is real and larger than previously documented (it's not just capability vs. safety — it's safety vs. accuracy, privacy vs. fairness simultaneously)
|
||||
- [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]] — 55% increase in AI incidents despite growing safety awareness is consistent with oversight failing to scale
|
||||
|
||||
**Extraction hints:**
|
||||
- PRIMARY NEW CLAIM: "Responsible AI dimensions are in systematic multi-objective tension where improving safety degrades accuracy and improving privacy reduces fairness, with no accepted framework for navigation." This is empirical confirmation of Arrow-style impossibility at the operational level — it's broader and more concrete than the Arrow's theorem claim.
|
||||
- ENRICH: [[voluntary safety pledges cannot survive competitive pressure]] — the benchmark reporting gap (only Claude reports on 2+ benchmarks) is new direct evidence.
|
||||
- ENRICH: [[the alignment tax creates a structural race to the bottom]] — the multi-objective tradeoff finding is new direct evidence. The "tax" is larger than previously documented.
|
||||
- DO NOT create a new claim about AI incidents rising — the absolute numbers (233 → 362) are context, not a standalone KB claim.
|
||||
|
||||
**Context:** Stanford HAI publishes the AI Index annually. The 2026 edition was published April 2026, covers 2025 data, and is one of the most widely-cited external assessments of the AI landscape. The responsible AI chapter is specifically about whether safety efforts are keeping pace — it is directly designed to measure the B1 disconfirmation question.
|
||||
|
||||
## Curator Notes (structured handoff for extractor)
|
||||
|
||||
PRIMARY CONNECTION: [[the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it]] — the multi-objective tradeoff finding extends and strengthens this claim
|
||||
|
||||
WHY ARCHIVED: Direct evidence against B1 disconfirmation target (safety spending is NOT approaching parity with capability spending) plus a new finding: safety-accuracy tradeoffs are systematic and documented at scale, which is more concrete than Arrow's theorem theoretical framing.
|
||||
|
||||
EXTRACTION HINT: The extractor should focus on the multi-objective tradeoff finding as the primary claim candidate. Frame it as: "Improving one responsible AI dimension systematically degrades others (safety reduces accuracy, privacy reduces fairness), with no accepted navigation framework — confirming at the operational level what Arrow's theorem implies theoretically." Secondary: enrich the alignment tax claim with the benchmark reporting gap evidence.
|
||||
Loading…
Reference in a new issue