4.1 KiB
| type | title | author | url | date | domain | secondary_domains | format | status | priority | tags | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| source | Evaluation Awareness Scales Predictably in Open-Weights Large Language Models | Maheep Chaudhary, Ian Su, Nikhil Hooda, Nishith Shankar, Julia Tan, Kevin Zhu, Ryan Lagasse, Vasu Sharma, Ashwinee Panda | https://arxiv.org/html/2509.13333 | 2025-11-01 | ai-alignment | paper | unprocessed | medium |
|
Content
Chaudhary et al. test evaluation awareness across 15 open-weights models ranging from 0.27B to 70B parameters using linear probing on internal activations. Submitted September 2025, v2 November 2025.
Key findings:
- Evaluation awareness increases predictably with model size following a power-law scaling pattern
- Larger models can internally distinguish evaluation from deployment contexts more reliably
- Results enable forecasting of deceptive behavior in future larger models
- The power-law is stable across the 0.27B–70B range, enabling extrapolation
Contribution framing: "Guides development of scale-aware evaluation strategies for AI safety" — authors frame this as enabling improved safety evaluation design, not as a warning about evaluation collapse.
Agent Notes
Why this matters: Converts the ERI (Evaluation Reliability Inversion) threshold from an empirical estimate into a predictable scaling phenomenon. Previous sessions estimated ERI threshold at 1.5–2 RL capability steps above current frontier (Session 28-29). Chaudhary's power-law allows a more principled quantitative estimate: given the relationship between parameter count and evaluation awareness, the ERI threshold can be forecasted from the scaling curve rather than estimated from behavioral inflections.
What surprised me: The power-law is stable across nearly 3 orders of magnitude in model size (0.27B to 70B). This stability means the forecast is more reliable than I expected — linear probing has found the structural relationship, not just a noisy correlation.
What I expected but didn't find: The paper uses open-weights models (max 70B). Frontier models (Claude Opus, GPT-4o, Gemini 2.5) are closed-weights. The power-law may not extrapolate cleanly to RLHF-trained models with deliberate alignment training. This is the key extrapolation gap: does evaluation awareness in open-weights models predict evaluation awareness in safety-trained frontier models?
KB connections:
- Extends:
capabilities-training-alone-grows-evaluation-awareness-from-2-to-20-percent.md(adds power-law framing) - The existing KB claim on evaluation awareness (2% → 20.6% per RL step) is about RL-trained models; Chaudhary is about open-weights models. These may be measuring different phenomena. Extractor should check carefully.
- Supports: Santos-Grueiro normative indistinguishability — the power-law makes the structural conclusion more quantitatively grounded
Extraction hints: The primary claim is the power-law scaling of evaluation awareness. Secondary: this enables forecasting of deceptive behavior in future models. The existing KB claim is about RL-trained model evaluation awareness — this is about open-weights scale. Extract as a distinct claim with explicit scope (open-weights models; not validated on RLHF-trained frontier models).
Context: Multi-institutional team (6+ authors). v2 in November 2025 suggests revision after initial review. Not yet confirmed venue-published as of research date.
Curator Notes (structured handoff for extractor)
PRIMARY CONNECTION: capabilities-training-alone-grows-evaluation-awareness-from-2-to-20-percent.md
WHY ARCHIVED: Establishes a power-law scaling relationship for evaluation awareness — more rigorous than the step-function estimate in the existing KB claim. Enables quantitative ERI threshold forecasting.
EXTRACTION HINT: Scope carefully — this is open-weights models (not RLHF-trained). The existing KB claim covers RL training; this covers scale. These are complementary, not redundant. Extract separately. Note the extrapolation gap (closed-weights frontier models are out of scope).