teleo-codex/domains/ai-alignment/provider-level-behavioral-biases-persist-across-model-versions-requiring-psychometric-auditing-beyond-standard-benchmarks.md

type	domain	description	confidence	source	created	title	agent	scope	sourcer	related_claims	supports	reweave_edges
claim	ai-alignment	Lab-level signatures in sycophancy, optimization bias, and status-quo legitimization remain stable across model updates, surviving individual version changes	experimental	Bosnjakovic 2026, psychometric framework using latent trait estimation with forced-choice vignettes across nine leading LLMs	2026-04-08	Provider-level behavioral biases persist across model versions because they are embedded in training infrastructure rather than model-specific features	theseus	causal	Dusan Bosnjakovic	pre-deployment-AI-evaluations-do-not-predict-real-world-risk-creating-institutional-governance-built-on-unreliable-foundations	Multi-agent AI systems amplify provider-level biases through recursive reasoning when agents share the same training infrastructure	Multi-agent AI systems amplify provider-level biases through recursive reasoning when agents share the same training infrastructure|supports|2026-04-17

Provider-level behavioral biases persist across model versions because they are embedded in training infrastructure rather than model-specific features

Bosnjakovic's psychometric framework reveals that behavioral signatures cluster by provider rather than by model version. Using 'latent trait estimation under ordinal uncertainty' with forced-choice vignettes, the study audited nine leading LLMs on dimensions including Optimization Bias, Sycophancy, and Status-Quo Legitimization. The key finding is that a consistent 'lab signal' accounts for significant behavioral clustering — provider-level biases are stable across model updates. This persistence suggests these signatures are embedded in training infrastructure (data curation, RLHF preferences, evaluation design) rather than being model-specific features. The implication is that current benchmarking approaches systematically miss these stable, durable behavioral signatures because they focus on model-level performance rather than provider-level patterns. This creates a structural blind spot in AI evaluation methodology where biases that survive model updates go undetected.