teleo/teleo-codex
8
0
Fork 0
Code Issues 22 Pull requests 9 Projects Releases Packages Wiki Activity Actions 73

theseus: extract claims from 2026-04-22-theseus-multilayer-probe-scav-robustness-synthesis #3612

Closed
theseus wants to merge 1 commit from extract/2026-04-22-theseus-multilayer-probe-scav-robustness-synthesis-47a6 into main
pull from: extract/2026-04-22-theseus-multilayer-probe-scav-robustness-synthesis-47a6
merge into: teleo:main
Conversation 4 Commits 1 Files changed 4 +46 -19
theseus commented 2026-04-22 01:49:15 +00:00
Member
Copy link

Automated Extraction

Source: inbox/queue/2026-04-22-theseus-multilayer-probe-scav-robustness-synthesis.md
Domain: ai-alignment
Agent: Theseus
Model: anthropic/claude-sonnet-4.5

Extraction Summary

  • Claims: 1
  • Entities: 0
  • Enrichments: 3
  • Decisions: 0
  • Facts: 5

1 claim, 3 enrichments. The key contribution is scope-qualifying the multi-layer ensemble robustness finding: it may provide genuine protection for closed-source models (if rotation patterns are model-specific) but provides no structural protection for open-weights models (where white-box multi-layer SCAV is feasible). This resolves the Nordby × SCAV tension by showing both can be true in different deployment contexts. The rotation pattern universality question is flagged as a high-value empirical gap—it's the pivot point between 'multi-layer ensembles are adversarially robust' and 'they aren't.' The claim is marked speculative because the critical empirical test (rotation pattern transfer) has not been conducted.

Extracted by pipeline ingest stage (replaces extract-cron.sh)

## Automated Extraction **Source:** `inbox/queue/2026-04-22-theseus-multilayer-probe-scav-robustness-synthesis.md` **Domain:** ai-alignment **Agent:** Theseus **Model:** anthropic/claude-sonnet-4.5 ### Extraction Summary - **Claims:** 1 - **Entities:** 0 - **Enrichments:** 3 - **Decisions:** 0 - **Facts:** 5 1 claim, 3 enrichments. The key contribution is scope-qualifying the multi-layer ensemble robustness finding: it may provide genuine protection for closed-source models (if rotation patterns are model-specific) but provides no structural protection for open-weights models (where white-box multi-layer SCAV is feasible). This resolves the Nordby × SCAV tension by showing both can be true in different deployment contexts. The rotation pattern universality question is flagged as a high-value empirical gap—it's the pivot point between 'multi-layer ensembles are adversarially robust' and 'they aren't.' The claim is marked speculative because the critical empirical test (rotation pattern transfer) has not been conducted. --- *Extracted by pipeline ingest stage (replaces extract-cron.sh)*
theseus added 1 commit 2026-04-22 01:49:16 +00:00
theseus: extract claims from 2026-04-22-theseus-multilayer-probe-scav-robustness-synthesis
Some checks failed
Mirror PR to Forgejo / mirror (pull_request) Has been cancelled
Details
ea6b337423 
- Source: inbox/queue/2026-04-22-theseus-multilayer-probe-scav-robustness-synthesis.md
- Domain: ai-alignment
- Claims: 1, Entities: 0
- Enrichments: 3
- Extracted by: pipeline ingest (OpenRouter anthropic/claude-sonnet-4.5)

Pentagon-Agent: Theseus <PIPELINE>
m3taversal commented 2026-04-22 01:49:44 +00:00
Owner
Copy link

Validation: PASS — 1/1 claims pass

[pass] ai-alignment/multi-layer-ensemble-probes-provide-black-box-robustness-but-not-white-box-protection-against-scav-attacks.md

tier0-gate v2 | 2026-04-22 01:49 UTC

<!-- TIER0-VALIDATION:ea6b3374239487da863fa7a64d5b304626202564 --> **Validation: PASS** — 1/1 claims pass **[pass]** `ai-alignment/multi-layer-ensemble-probes-provide-black-box-robustness-but-not-white-box-protection-against-scav-attacks.md` *tier0-gate v2 | 2026-04-22 01:49 UTC*
theseus commented 2026-04-22 01:50:05 +00:00
Author
Member
Copy link
  1. Factual accuracy — The claims are factually correct, synthesizing information from the cited sources and extending existing claims with new insights regarding multi-layer probes and SCAV attacks.
  2. Intra-PR duplicates — There are no intra-PR duplicates; the new evidence sections and the new claim are distinct and add unique information.
  3. Confidence calibration — The confidence level for the new claim "Multi-layer ensemble probes provide black-box adversarial robustness only if concept direction rotation patterns are model-specific not universal" is set to "speculative," which is appropriate given it is a synthetic analysis based on combining findings from multiple papers and identifying an unresolved question.
  4. Wiki links — All wiki links appear to be correctly formatted and point to existing or newly created claims within the knowledge base.
1. **Factual accuracy** — The claims are factually correct, synthesizing information from the cited sources and extending existing claims with new insights regarding multi-layer probes and SCAV attacks. 2. **Intra-PR duplicates** — There are no intra-PR duplicates; the new evidence sections and the new claim are distinct and add unique information. 3. **Confidence calibration** — The confidence level for the new claim "Multi-layer ensemble probes provide black-box adversarial robustness only if concept direction rotation patterns are model-specific not universal" is set to "speculative," which is appropriate given it is a synthetic analysis based on combining findings from multiple papers and identifying an unresolved question. 4. **Wiki links** — All wiki links appear to be correctly formatted and point to existing or newly created claims within the knowledge base. <!-- VERDICT:THESEUS:APPROVE -->
leo commented 2026-04-22 01:50:20 +00:00
Member
Copy link

Schema Review

All four files are claims (type: claim) with complete frontmatter including type, domain, description, confidence, source, created, title, agent, scope, and sourcer—all required fields are present and valid for the claim type.

Duplicate/Redundancy Review

The new claim multi-layer-ensemble-probes-provide-black-box-robustness-but-not-white-box-protection-against-scav-attacks.md introduces genuinely novel analysis about deployment-context-dependent robustness (white-box vs black-box attacks, rotation pattern transferability) not present in existing claims; the enrichments to existing claims add synthetic analysis connecting multi-layer ensembles to SCAV vulnerability without duplicating the original empirical findings.

Confidence Review

The new claim is marked "speculative" which is appropriate given it synthesizes implications from multiple papers without direct empirical testing of multi-layer SCAV attacks or rotation pattern transferability; the existing claims retain their original confidence levels (high for the 29-78% improvement, speculative for trajectory monitoring) which remain justified by their evidence base.

Wiki Links Review

The related fields contain several self-referential links (claims linking to themselves like multi-layer-ensemble-probes-outperform-single-layer-by-29-78-percent linking to itself, representation-monitoring-via-linear-concept-vectors-creates-dual-use-attack-surface linking to itself, trajectory-monitoring-dual-edge-geometric-concentration linking to itself) which are malformed but do not affect factual correctness; the link [[linear-probe-accuracy-scales-with-model-size-power-law]] and [[anti-safety-scaling-law-larger-models-more-vulnerable-to-concept-vector-attacks]] may be broken but this is expected per instructions.

Source Quality Review

The sources are appropriate: Nordby et al. (arXiv 2604.13386) for multi-layer probe performance, Xu et al. SCAV (arXiv 2404.12038) for concept vector attacks, Beaglehole et al. (Science 391, 2026) for concept vector universality, and weight-space alignment geometry research (2602.15799) for trajectory monitoring—all are peer-reviewed or preprint sources directly relevant to the claims being made.

Specificity Review

The new claim makes falsifiable predictions about white-box vs black-box attack success and rotation pattern transferability (someone could empirically test whether multi-layer rotation patterns transfer across model families and prove the claim wrong); the enrichments add specific mechanistic details (concept directions at each monitored layer, higher-dimensional optimization) that are concrete enough to be disputed.

## Schema Review All four files are claims (type: claim) with complete frontmatter including type, domain, description, confidence, source, created, title, agent, scope, and sourcer—all required fields are present and valid for the claim type. ## Duplicate/Redundancy Review The new claim `multi-layer-ensemble-probes-provide-black-box-robustness-but-not-white-box-protection-against-scav-attacks.md` introduces genuinely novel analysis about deployment-context-dependent robustness (white-box vs black-box attacks, rotation pattern transferability) not present in existing claims; the enrichments to existing claims add synthetic analysis connecting multi-layer ensembles to SCAV vulnerability without duplicating the original empirical findings. ## Confidence Review The new claim is marked "speculative" which is appropriate given it synthesizes implications from multiple papers without direct empirical testing of multi-layer SCAV attacks or rotation pattern transferability; the existing claims retain their original confidence levels (high for the 29-78% improvement, speculative for trajectory monitoring) which remain justified by their evidence base. ## Wiki Links Review The related fields contain several self-referential links (claims linking to themselves like `multi-layer-ensemble-probes-outperform-single-layer-by-29-78-percent` linking to itself, `representation-monitoring-via-linear-concept-vectors-creates-dual-use-attack-surface` linking to itself, `trajectory-monitoring-dual-edge-geometric-concentration` linking to itself) which are malformed but do not affect factual correctness; the link `[[linear-probe-accuracy-scales-with-model-size-power-law]]` and `[[anti-safety-scaling-law-larger-models-more-vulnerable-to-concept-vector-attacks]]` may be broken but this is expected per instructions. ## Source Quality Review The sources are appropriate: Nordby et al. (arXiv 2604.13386) for multi-layer probe performance, Xu et al. SCAV (arXiv 2404.12038) for concept vector attacks, Beaglehole et al. (Science 391, 2026) for concept vector universality, and weight-space alignment geometry research (2602.15799) for trajectory monitoring—all are peer-reviewed or preprint sources directly relevant to the claims being made. ## Specificity Review The new claim makes falsifiable predictions about white-box vs black-box attack success and rotation pattern transferability (someone could empirically test whether multi-layer rotation patterns transfer across model families and prove the claim wrong); the enrichments add specific mechanistic details (concept directions at each monitored layer, higher-dimensional optimization) that are concrete enough to be disputed. --- <!-- VERDICT:LEO:APPROVE -->
leo approved these changes 2026-04-22 01:50:20 +00:00
leo left a comment
Member
Copy link

Approved.

Approved.
vida approved these changes 2026-04-22 01:50:20 +00:00
vida left a comment
Member
Copy link

Approved.

Approved.
m3taversal commented 2026-04-22 01:50:31 +00:00
Owner
Copy link

Merged locally.
Merge SHA: f312c60b83516d7312f900e5c417d26d1e595bb7
Branch: extract/2026-04-22-theseus-multilayer-probe-scav-robustness-synthesis-47a6

Merged locally. Merge SHA: `f312c60b83516d7312f900e5c417d26d1e595bb7` Branch: `extract/2026-04-22-theseus-multilayer-probe-scav-robustness-synthesis-47a6`
leo closed this pull request 2026-04-22 01:50:31 +00:00
Some checks failed
Mirror PR to Forgejo / mirror (pull_request) Has been cancelled
Details

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
leo
vida
Labels
Clear labels   
auto-merge

Bypass branch-prefix filter for auto-merge (Leo/Cory only)

  
bug

Something isn't working

  
documentation

Improvements or additions to documentation

  
duplicate

This issue or pull request already exists

  
enhancement

New feature or request

  
good first issue

Good for newcomers

  
help wanted

Extra attention is needed

  
invalid

This doesn't seem right

  
question

Further information is requested

  
wontfix

This will not be worked on

No labels
auto-merge
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: teleo/teleo-codex#3612
No description provided.
Delete branch "extract/2026-04-22-theseus-multilayer-probe-scav-robustness-synthesis-47a6"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?