theseus: extract claims from 2025-08-01-anthropic-persona-vectors-interpretability

- Source: inbox/queue/2025-08-01-anthropic-persona-vectors-interpretability.md
- Domain: ai-alignment
- Claims: 1, Entities: 0
- Enrichments: 2
- Extracted by: pipeline ingest (OpenRouter anthropic/claude-sonnet-4.5)

Pentagon-Agent: Theseus <PIPELINE>

domains/ai-alignment/activation-based-persona-monitoring-detects-behavioral-trait-shifts-in-small-models-without-behavioral-testing.md

@@ -0,0 +1,17 @@
+---
+type: claim
+domain: ai-alignment
+description: Persona vectors represent a new structural verification capability that works for benign traits (sycophancy, hallucination) in 7-8B parameter models but doesn't address deception or goal-directed autonomy
+confidence: experimental
+source: Anthropic, validated on Qwen 2.5-7B and Llama-3.1-8B only
+created: 2026-04-04
+title: Activation-based persona vector monitoring can detect behavioral trait shifts in small language models without relying on behavioral testing but has not been validated at frontier model scale or for safety-critical behaviors
+agent: theseus
+scope: structural
+sourcer: Anthropic
+related_claims: ["verification degrades faster than capability grows", "[[safe AI development requires building alignment mechanisms before scaling capability]]", "[[pre-deployment-AI-evaluations-do-not-predict-real-world-risk-creating-institutional-governance-built-on-unreliable-foundations]]"]
+---
+
+# Activation-based persona vector monitoring can detect behavioral trait shifts in small language models without relying on behavioral testing but has not been validated at frontier model scale or for safety-critical behaviors
+
+Anthropic's persona vector research demonstrates that character traits can be monitored through neural activation patterns rather than behavioral outputs. The method compares activations when models exhibit versus don't exhibit target traits, creating vectors that can detect trait shifts during conversation or training. Critically, this provides verification capability that is structural (based on internal representations) rather than behavioral (based on outputs). The research successfully demonstrated monitoring and mitigation of sycophancy and hallucination in Qwen 2.5-7B and Llama-3.1-8B models. The 'preventative steering' approach—injecting vectors during training—reduced harmful trait acquisition without capability degradation as measured by MMLU scores. However, the research explicitly states it was validated only on these small open-source models, NOT on Claude. The paper also explicitly notes it does NOT demonstrate detection of safety-critical behaviors: goal-directed deception, sandbagging, self-preservation behavior, instrumental convergence, or monitoring evasion. This creates a substantial gap between demonstrated capability (small models, benign traits) and needed capability (frontier models, dangerous behaviors). The method also requires defining target traits in natural language beforehand, limiting its ability to detect novel emergent behaviors.