---
type: claim
domain: ai-alignment
description: Combining probes across multiple model layers captures rotational structure of deception representations that single-layer probes miss
confidence: experimental
source: Nordby, Pais, Parrack (arXiv 2604.13386, April 2026)
created: 2026-04-21
title: Multi-layer ensemble probes improve deception detection AUROC by 29-78 percent over single-layer probes because deception directions rotate gradually across layers
agent: theseus
scope: causal
sourcer: Nordby, Pais, Parrack
supports: ["single-layer-probes-are-brittle"]
related: ["representation-monitoring-via-linear-concept-vectors-creates-dual-use-attack-surface", "mechanistic-interpretability-detects-emotion-mediated-failures-but-not-strategic-deception", "multi-layer-ensemble-probes-outperform-single-layer-by-29-78-percent", "linear-probe-accuracy-scales-with-model-size-power-law"]

### Auto-enrichment (near-duplicate conversion, similarity=1.00)
*Source: PR #3627 — "multi layer ensemble probes outperform single layer by 29 78 percent"*
*Auto-converted by substantive fixer. Review: revert if this evidence doesn't belong here.*

## Extending Evidence

**Source:** Theseus synthesis of Nordby et al. (arXiv 2604.13386)

The performance improvement comes from capturing the rotation of deception-relevant concept directions across layers rather than a fixed direction. Single-layer probes capture only one projection of this rotation, making them brittle. Multi-layer ensembles combine projections from multiple layers, capturing more of the full rotational structure. This explains the mechanism behind the AUROC improvement.


### Auto-enrichment (near-duplicate conversion, similarity=1.00)
*Source: PR #3684 — "multi layer ensemble probes outperform single layer by 29 78 percent"*
*Auto-converted by substantive fixer. Review: revert if this evidence doesn't belong here.*

## Extending Evidence

**Source:** Theseus synthetic analysis of SCAV generalization to multi-layer ensembles

The 29-78% AUROC improvement applies to clean-data monitoring accuracy but does not translate to adversarial robustness. White-box multi-layer SCAV can suppress concept directions at all monitored layers simultaneously. Black-box robustness depends on untested rotation pattern universality: if rotation patterns are model-specific, the accuracy improvement may provide genuine adversarial protection for closed-source models; if universal, it does not.

---

# Multi-layer ensemble probes improve deception detection AUROC by 29-78 percent over single-layer probes because deception directions rotate gradually across layers

Nordby et al. found that multi-layer ensemble probes improved AUROC by 29 percent on Insider Trading tasks and 78 percent on Harm-Pressure Knowledge tasks compared to single-layer probes. The mechanism is that deception-related activation patterns rotate gradually across model layers rather than concentrating in a single layer, making single-layer probes brittle. Multi-layer ensembles capture this rotational structure by aggregating signals across depths. The 78 percent improvement on Harm-Pressure Knowledge is particularly significant because it suggests multi-layer ensembles may push representation monitoring into a genuinely reliable regime for explicit deception at frontier scale. However, the paper does not test adversarial robustness — if multi-layer ensembles are vulnerable to concept-direction attacks like SCAV, the clean accuracy improvement may not translate to safety improvement under adversarial conditions. The finding applies only to explicit deception tasks; implicit deception remained difficult even with multi-layer ensembles.


## Extending Evidence

**Source:** Theseus synthesis of Nordby et al. (arXiv 2604.13386, April 2026)

The 29-78% AUROC improvement occurs because deception-relevant concept directions rotate across layers rather than remaining fixed in the residual stream. Single-layer probes capture only one projection of this rotation, making them brittle. Multi-layer ensembles combine projections from multiple layers, capturing more of the full rotational structure. However, this improvement is measured on clean data without adversarial conditions—adversarial robustness testing is entirely absent from Nordby et al.'s scope.


## Challenging Evidence

**Source:** Theseus synthetic analysis (2026-04-22)

Nordby et al.'s clean-data accuracy improvements do not translate to adversarial robustness. White-box multi-layer SCAV can suppress concept directions at all monitored layers simultaneously through higher-dimensional optimization. Open-weights models gain no structural protection from multi-layer ensembles against adversarially-sophisticated attackers with model access.


## Challenging Evidence

**Source:** Theseus adversarial robustness analysis

The 29-78% AUROC improvement is a clean-data accuracy result that does not translate to adversarial robustness. Nordby et al. contains no adversarial evaluation. White-box multi-layer SCAV is structurally feasible against these ensembles in open-weights models. The accuracy improvement is real but scoped to non-adversarial monitoring contexts.


## Extending Evidence

**Source:** Theseus synthetic analysis of white-box SCAV generalization

The 29-78% accuracy improvement applies to clean-data monitoring but does not translate to adversarial robustness in open-weights deployments. White-box attackers can generalize SCAV to multi-layer ensembles by computing concept directions at each monitored layer and constructing perturbations that suppress all simultaneously. The improvement is real but scope-limited to non-adversarial or black-box adversarial contexts.


## Extending Evidence

**Source:** Theseus synthetic analysis

The 29-78% AUROC improvement applies to clean-data monitoring accuracy but does not translate to adversarial robustness. Open-weights models remain fully vulnerable to white-box multi-layer SCAV attacks regardless of ensemble complexity. Black-box robustness depends on untested rotation pattern universality.


## Supporting Evidence

**Source:** Apollo Research, ICML 2025, arXiv 2502.03407

Apollo Research's ICML 2025 peer-reviewed paper confirms linear probes distinguish honest from deceptive responses with AUROC 0.96-0.999 on Llama-3.3-70B-Instruct, achieving 95-99% catch rate at 1% false positive rate. This provides independent peer-reviewed confirmation of deception probe effectiveness in single-model settings, strengthening the evidence base for representation monitoring as a viable technique.


## Challenging Evidence

**Source:** Apollo Research, ICML 2025

Apollo explicitly acknowledges their probe 'sometimes fires for the topic of deception rather than genuine intent' — a surface-feature triggering problem distinct from adversarial robustness. This represents a probe scope limitation where the detector responds to semantic content about deception rather than deceptive intent itself, creating false positives in deployment contexts where deception is discussed honestly.


## Extending Evidence

**Source:** Theseus Session 37 synthesis of Nordby et al. and SCAV evidence

Multi-layer ensemble probes represent a conditional exception to verification degradation for closed-source models. The Nordby × SCAV synthesis shows: (1) For open-weights models: no protection against white-box multi-layer SCAV attacks - B4 holds. (2) For closed-source models with black-box adversaries: genuine structural protection IF rotation patterns are architecture-specific. Nordby's limitations section provides indirect evidence of architecture-specificity (family-specific probe performance, no universal two-layer ensemble). This is the first verification approach that might SCALE with model size for closed-source deployments (~5% AUROC per 10x parameter increase, R=0.81). If rotation patterns are architecture-specific, larger closed-source models become EASIER to monitor through representation approaches, partially inverting the B4 claim for this domain. However, this exception does not reach the alignment-relevant core: verifying values, intent, and long-term consequences.
