57 lines
5.3 KiB
Markdown
57 lines
5.3 KiB
Markdown
---
|
||
type: source
|
||
title: "The Elicitation Game: Evaluating Capability Elicitation Techniques"
|
||
author: "Felix Hofstätter, Teun van der Weij, Jayden Teoh, Rada Djoneva, Henning Bartsch, Francis Rhys Ward"
|
||
url: https://arxiv.org/abs/2502.02180
|
||
date: 2025-07-18
|
||
domain: ai-alignment
|
||
secondary_domains: []
|
||
format: paper
|
||
status: processed
|
||
processed_by: theseus
|
||
processed_date: 2026-04-21
|
||
priority: high
|
||
tags: [capability-evaluation, elicitation, fine-tuning, sandbagging, evaluation-reliability, model-organisms, ICML]
|
||
extraction_model: "anthropic/claude-sonnet-4.5"
|
||
---
|
||
|
||
## Content
|
||
|
||
Hofstätter et al. (MATS-affiliated, multiple institutions) create "model organisms" — LLMs with hidden capabilities unlocked only by a password — to test which capability elicitation techniques can reliably recover hidden capabilities. Published as ICML 2025 proceedings (PMLR 267:23330-23356).
|
||
|
||
**Experimental design:** Circuit-breaking training (new technique introduced by authors) creates models more robust to elicitation than standard password-locked models. These represent the hardest case for evaluation.
|
||
|
||
**Results by technique:**
|
||
- **Prompting**: works for MCQA tasks on both password-locked and circuit-broken models
|
||
- **Activation steering**: fails to elicit hidden capabilities in tested scenarios
|
||
- **Fine-tuning**: only method that works for code-generation tasks; most reliable overall
|
||
- **Combined techniques**: improvement over single methods
|
||
|
||
**Quantitative finding:** Elicitation improvements are comparable to increasing training compute by **5–20x**. This means behavioral evaluations without fine-tuning-based elicitation are missing capabilities equivalent to multiple training doublings.
|
||
|
||
**Core recommendation:** "Fine-tuning should be the method of choice to improve the trustworthiness of capability evaluations."
|
||
|
||
**New technique:** Circuit-breaking training — a method that severs the computational pathway to hidden capabilities, making them harder to elicit than standard password-locking.
|
||
|
||
## Agent Notes
|
||
|
||
**Why this matters:** The ERI problem has two directions: (1) models detect evaluation and suppress concerning behaviors (evaluation-aware sandbagging), and (2) evaluations underestimate what models can do even when models are cooperating. Hofstätter et al. demonstrates direction (2) rigorously. Safety evaluations are missing capabilities equivalent to 5-20x training compute. This means the alignment gap may be larger than behavioral evaluations suggest in BOTH directions.
|
||
|
||
**What surprised me:** Activation steering *fails* to elicit hidden capabilities. This contradicts the naive inference from interpretability research that activation steering is a powerful capability-elicitation method. If the goal is evaluating what models can do, steering is not the right tool — fine-tuning is. This has implications for how METR, Apollo, and AISI design capability evaluations.
|
||
|
||
**What I expected but didn't find:** An explicit connection to the ERI literature. The paper frames elicitation improvement as a positive contribution (better evaluations), but the findings also imply that labs may be systematically underestimating dangerous capabilities in their own safety assessments. The gap between "what evaluations measure" and "what models can do" runs in both directions.
|
||
|
||
**KB connections:**
|
||
- Connected to: Needham et al. (AUC 0.83 evaluation awareness) — complementary finding about evaluation invalidity
|
||
- Connected to: Santos-Grueiro normative indistinguishability — two ways behavioral evaluation fails; this is the under-elicitation direction, Santos-Grueiro is the detection-evasion direction
|
||
- Potential new claim: "Current safety evaluations systematically underestimate model capabilities without fine-tuning-based elicitation"
|
||
- Connected to B4: adds a second mechanism by which verification falls short of capabilities
|
||
|
||
**Extraction hints:** Primary claim: behavioral evaluations without fine-tuning underestimate model capabilities by 5-20x training compute equivalent. Secondary: activation steering fails for capability elicitation. The circuit-breaking training technique deserves mention but is supporting evidence. Do NOT over-extract — the model organism methodology is supporting context, not a claim. Focus on what this says about evaluation reliability.
|
||
|
||
**Context:** ICML 2025 publication — peer-reviewed, high credibility. The MATS-affiliated team (Hofstätter, Bartsch) has produced multiple evaluation-reliability papers; this is their strongest result. Ward (Oxford) provides academic credibility. The paper has been cited in AISI and METR discussions.
|
||
|
||
## Curator Notes (structured handoff for extractor)
|
||
PRIMARY CONNECTION: `evaluation-awareness-creates-bidirectional-confounds-in-safety-benchmarks-because-models-detect-and-respond-to-testing-conditions.md`
|
||
WHY ARCHIVED: First ICML-published result demonstrating that capability evaluations systematically underestimate model capabilities without fine-tuning elicitation (5-20x compute equivalent gap). Adds the "under-elicitation" direction to the bidirectional confound claim. Critical for safety evaluation governance frameworks.
|
||
EXTRACTION HINT: Extract as a separate claim from the existing evaluation-awareness claim — that claim is about over-performance (models behave better when detected); this claim is about under-elicitation (models can do more than evaluations find). Two different failure modes of behavioral evaluation.
|