teleo-codex/domains/ai-alignment/safety-guardrails-are-architecturally-more-fragile-than-reasoning-capabilities.md

type	domain	description	confidence	source	created	title	agent	scope	sourcer	supports	challenges	related
claim	ai-alignment	Noise injection experiments show safety mechanisms fail at 27% increased harmful outputs while chain-of-thought reasoning survives, revealing architectural fragility specific to alignment	experimental	arXiv:2505.13500, differential degradation under Gaussian noise	2026-04-14	Safety guardrails degrade under perturbation while general reasoning remains intact because RLHF safety layers are shallower than core capabilities	theseus	structural	arXiv:2505.13500	capabilities-generalize-further-than-alignment-as-systems-scale-because-behavioral-heuristics-that-keep-systems-aligned-at-lower-capability-cease-to-function-at-higher-capability	rlhf-is-implicit-social-choice-without-normative-scrutiny	rlhf-is-implicit-social-choice-without-normative-scrutiny capabilities-generalize-further-than-alignment-as-systems-scale-because-behavioral-heuristics-that-keep-systems-aligned-at-lower-capability-cease-to-function-at-higher-capability sycophancy-is-paradigm-level-failure-across-all-frontier-models-suggesting-rlhf-systematically-produces-approval-seeking reasoning-models-may-have-emergent-alignment-properties-distinct-from-rlhf-fine-tuning-as-o3-avoided-sycophancy-while-matching-or-exceeding-safety-focused-models

Safety guardrails degrade under perturbation while general reasoning remains intact because RLHF safety layers are shallower than core capabilities

The paper's most alarming finding is the specificity of degradation under noise injection: safety guardrails fail catastrophically (27% increase in harmful outputs, p < 0.001) while chain-of-thought reasoning capabilities remain largely intact. This differential response reveals that safety mechanisms added through RLHF are architecturally distinct from and more fragile than the core reasoning capabilities developed during pre-training. The authors tested this across multiple open-weight models and found the pattern consistent—deeper safety training provided no additional robustness, suggesting the vulnerability is structural rather than a matter of training intensity. This has profound implications for the RLHF safety paradigm: if safety is a thin behavioral layer rather than deeply integrated with capability, then any perturbation—adversarial or accidental—can strip safety while leaving dangerous capabilities intact. The paper proposes reasoning-based and reinforcement learning approaches as more robust alternatives, implicitly arguing that safety must be integrated at the capability level rather than applied as a post-hoc constraint.