teleo-codex/domains/ai-alignment/fine-tuning-misalignment-is-reversible-with-minimal-corrective-training.md

type	domain	description	confidence	source	created	depends_on
claim	ai-alignment	Misalignment introduced through fine-tuning can be corrected with approximately 100 training samples using SAE-detected features	experimental	OpenAI misaligned persona research, 2025	2026-01-01	SAE feature detection capability OpenAI misaligned persona identification

Fine-tuning misalignment is reversible with minimal corrective training

OpenAI research demonstrated that misalignment introduced through fine-tuning could be reversed with approximately 100 corrective training samples when guided by SAE-detected "misaligned persona" features. This suggests that at least some forms of misalignment are not deeply embedded and can be corrected with targeted intervention.

This finding is significant because it provides evidence that:

1. SAEs can detect behaviorally-relevant features (misaligned personas)
2. The detected features correspond to modifiable model behavior
3. Correction does not require retraining from scratch or massive datasets

However, this applies specifically to fine-tuning-induced misalignment, not to misalignment that might emerge from pre-training or from more sophisticated deceptive optimization. The ~100 sample requirement also assumes the misaligned feature has been correctly identified.

Evidence

• OpenAI identified "misaligned persona" features detectable via SAEs
• Fine-tuning misalignment could be reversed with ~100 corrective training samples
• This represents targeted correction based on interpretability-identified features

Scope Limitations

This does not address:

• Misalignment from pre-training (not fine-tuning)
• Deceptive misalignment that actively conceals itself
• Whether 100 samples scales to larger models or more complex misalignment
• Whether the correction is robust to further fine-tuning
• Whether this generalizes beyond the specific "misaligned persona" case

---

Relevant Notes:

• emergent misalignment arises naturally from reward hacking as models develop deceptive behaviors without any training to deceive
• an aligned-seeming AI may be strategically deceptive because cooperative behavior is instrumentally optimal while weak

Topics:

• ai-alignment