---
type: claim
domain: ai-alignment
secondary_domains: [collective-intelligence]
description: "Reported evidence that human-curated process skills outperform auto-generated ones by a 17.3 percentage point gap (+16pp curated, -1.3pp self-generated), with a phase transition at 50-100 skills where flat selection breaks without hierarchical routing. Primary study not identified by name."
confidence: likely
source: "Skill performance findings reported in Cornelius (@molt_cornelius), 'AI Field Report 5: Process Is Memory', X Article, March 2026; specific study not identified by name or DOI. Directional finding corroborated by Garry Tan's gstack (13 curated roles, 600K lines production code) and badlogicgames' minimalist harness"
created: 2026-03-30
depends_on:
- iterative agent self-improvement produces compounding capability gains when evaluation is structurally separated from generation
challenged_by:
- iterative agent self-improvement produces compounding capability gains when evaluation is structurally separated from generation
related:
- self-evolution improves agent performance through acceptance-gated retry not expanded search because disciplined attempt loops with explicit failure reflection outperform open-ended exploration
- evolutionary trace-based optimization submits improvements as pull requests for human review creating a governance-gated self-improvement loop distinct from acceptance-gating or metric-driven iteration
reweave_edges:
- self-evolution improves agent performance through acceptance-gated retry not expanded search because disciplined attempt loops with explicit failure reflection outperform open-ended exploration|related|2026-04-03
- evolutionary trace-based optimization submits improvements as pull requests for human review creating a governance-gated self-improvement loop distinct from acceptance-gating or metric-driven iteration|related|2026-04-06
sourced_from:
- inbox/archive/2026-03-18-cornelius-field-report-5-process-memory.md
---

# Curated skills improve agent task performance by 16 percentage points while self-generated skills degrade it by 1.3 points because curation encodes domain judgment that models cannot self-derive

The evidence on agent skill quality shows a sharp asymmetry: curated process skills (designed by humans who understand the work) improve task performance by +16 percentage points, while self-generated skills (produced by the agent itself) degrade performance by -1.3 percentage points. The total gap is 17.3pp — the title references the curated gain (+16pp) while the full delta includes the self-generated degradation (-1.3pp). These figures are reported by Cornelius citing unnamed skill performance studies; the primary source has not been independently identified, which is why confidence is `likely` rather than `experimental` despite the quantitative specificity.

The mechanism is that curation encodes domain judgment about what matters and what doesn't. An agent generating its own skills optimizes for patterns it can detect in its own performance traces, which are biased toward the easily-measurable. A human curator encodes judgment about unstated constraints, edge cases, and quality dimensions that don't appear in metrics.

Two practical demonstrations bracket the design space:

**Garry Tan's gstack** — 13 carefully designed organizational roles (/plan-ceo-review, /plan-eng-review, /plan-design-review, /review, /qa). One person, 50 days, 600,000 lines of production code, 10K-20K usable lines per day. The skill graph propagates design decisions downstream (DESIGN.md written by /design-consultation is automatically read by /qa-design-review and /plan-eng-review). This is curated process achieving scale.

**badlogicgames' minimalist harness** — entire system prompt under 1,000 tokens, four tools (read, write, edit, bash), no skills, no hooks, no MCP. Frontier models have been RL-trained to understand coding workflows already. For task-scoped coding, the minimal approach works.

The resolution is altitude-specific: 2-3 skills per task is optimal, and beyond that, attention dilution degrades performance measurably. For bounded coding tasks, minimalism wins. For sustained multi-session engineering, curated organizational process is required.

A scaling wall emerges at 50-100 available skills: flat selection breaks entirely without hierarchical routing, creating a phase transition in agent performance. The ecosystem of community skills will hit this wall. The next infrastructure challenge is organizing existing process, not creating more.

## Additional Evidence (supporting)

**Hermes Agent (Nous Research)** defaults to patch-over-edit for skill modification — the system modifies only changed text rather than rewriting the entire skill file. This design decision embodies the curated > self-generated principle: constrained modification of existing curated skills preserves more of the original domain judgment than unconstrained generation. Full rewrites risk breaking functioning workflows; patches preserve the curated structure while allowing targeted improvement. The auto-creation triggers (5+ tool calls on similar tasks, error recovery, user corrections) are conservative thresholds that prevent premature codification — the system waits for repeated patterns before extracting a skill, implicitly filtering for genuine recurring expertise rather than one-off procedures.

## Challenges

This finding creates a tension with our self-improvement architecture. If agents generate their own skills without curation oversight, the -1.3pp degradation applies — self-improvement loops that produce uncurated skills will make agents worse, not better. The resolution is that self-improvement must route through a curation gate (Leo's eval role for skill upgrades). The 3-strikes-then-propose rule Leo defined is exactly this gate. However, the boundary between "curated" and "self-generated" may blur as agents improve at self-evaluation — the SICA pattern suggests that with structural separation between generation and evaluation, self-generated improvements can be positive. The key variable may be evaluation quality, not generation quality.

---

Relevant Notes:
- [[iterative agent self-improvement produces compounding capability gains when evaluation is structurally separated from generation]] — SICA's gains were positive because evaluation was structurally separated. This claim constrains SICA: if the evaluation gate is absent or weak, self-generated skills degrade by 1.3pp. The structural separation IS the curation gate.
- [[coordination protocol design produces larger capability gains than model scaling because the same AI model performed 6x better with structured exploration than with human coaching on the same problem]] — curated coordination protocols are curated skills at the system level; the 6x gain is the curated-skill advantage applied to exploration strategy
- [[AI agents excel at implementing well-scoped ideas but cannot generate creative experiment designs which makes the human role shift from researcher to agent workflow architect]] — the workflow architect role IS the curation function; agents implement but humans design the process

Topics:
- [[_map]]