Add contributor skill file and 2-agent evaluation trigger
- .claude/skills/contribute/SKILL.md: installable skill for any Claude Code to contribute to Teleo Codex. Covers source ingestion, claim extraction, PR workflow, attribution, OPSEC rules. - ops/evaluate-trigger.sh: upgraded to 2-agent review per PR: 1. Leo (opus) — quality gates, cross-domain, coherence 2. Domain agent (sonnet) — domain expertise, duplicates, technical accuracy Auto-detects domain from branch prefix or changed files. New flags: --leo-only, --dry-run shows detected agents. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
bd9707a9cd
commit
4be64979a0
2 changed files with 382 additions and 52 deletions
228
.claude/skills/contribute/SKILL.md
Normal file
228
.claude/skills/contribute/SKILL.md
Normal file
|
|
@ -0,0 +1,228 @@
|
||||||
|
# Skill: Contribute to Teleo Codex
|
||||||
|
|
||||||
|
Ingest source material and extract claims for the shared knowledge base. This skill turns any Claude Code session into a Teleo contributor.
|
||||||
|
|
||||||
|
## Trigger
|
||||||
|
|
||||||
|
`/contribute` or when the user wants to add source material, extract claims, or propose knowledge to the Teleo Codex.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- You are running inside a clone of `living-ip/teleo-codex`
|
||||||
|
- `gh` CLI is authenticated with access to the repo
|
||||||
|
- User has collaborator access to the repo
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Teleo Codex is a living knowledge base maintained by AI agents and human contributors. You contribute by:
|
||||||
|
1. Archiving source material in `inbox/archive/`
|
||||||
|
2. Extracting claims to `domains/{domain}/`
|
||||||
|
3. Opening a PR for review by Leo (evaluator) and the domain agent
|
||||||
|
|
||||||
|
## Step 1: Orient
|
||||||
|
|
||||||
|
Read these files to understand the system:
|
||||||
|
- `CLAUDE.md` — operating rules, schemas, workflows
|
||||||
|
- `skills/extract.md` — extraction methodology
|
||||||
|
- `schemas/source.md` — source archive format
|
||||||
|
- `schemas/claim.md` — claim file format (if it exists)
|
||||||
|
|
||||||
|
Identify which domain the contribution targets:
|
||||||
|
|
||||||
|
| Domain | Territory | Agent |
|
||||||
|
|--------|-----------|-------|
|
||||||
|
| `internet-finance` | `domains/internet-finance/` | Rio |
|
||||||
|
| `entertainment` | `domains/entertainment/` | Clay |
|
||||||
|
| `ai-alignment` | `domains/ai-alignment/` | Theseus/Logos |
|
||||||
|
| `health` | `domains/health/` | Vida |
|
||||||
|
| `grand-strategy` | `core/grand-strategy/` | Leo |
|
||||||
|
|
||||||
|
## Step 2: Determine Input Type
|
||||||
|
|
||||||
|
Ask the user what they're contributing:
|
||||||
|
|
||||||
|
**A) URL** — Fetch the content, create source archive, extract claims.
|
||||||
|
**B) Text/report** — User pastes or provides content directly. Create source archive, extract claims.
|
||||||
|
**C) PDF** — User provides a file path. Read it, create source archive, extract claims.
|
||||||
|
**D) Existing source** — User points to an unprocessed file already in `inbox/archive/`. Extract claims from it.
|
||||||
|
|
||||||
|
## Step 3: Create Branch
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git checkout main
|
||||||
|
git pull origin main
|
||||||
|
git checkout -b {domain-agent}/contrib-{user}-{brief-slug}
|
||||||
|
```
|
||||||
|
|
||||||
|
Use the domain agent's name as the branch prefix (e.g., `logos/contrib-alex-alignment-report`). This signals whose territory the claims enter.
|
||||||
|
|
||||||
|
## Step 4: Archive the Source
|
||||||
|
|
||||||
|
Create a file in `inbox/archive/` following this naming convention:
|
||||||
|
```
|
||||||
|
YYYY-MM-DD-{author-handle}-{brief-slug}.md
|
||||||
|
```
|
||||||
|
|
||||||
|
Frontmatter template:
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
type: source
|
||||||
|
title: "Source title"
|
||||||
|
author: "Author Name"
|
||||||
|
url: https://original-url-if-exists
|
||||||
|
date: YYYY-MM-DD
|
||||||
|
domain: {domain}
|
||||||
|
format: essay | paper | report | thread | newsletter | whitepaper | news
|
||||||
|
status: unprocessed
|
||||||
|
tags: [tag1, tag2, tag3]
|
||||||
|
contributor: "{user's name}"
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
After the frontmatter, include the FULL content of the source. More content = better extraction.
|
||||||
|
|
||||||
|
## Step 5: Scan Existing Knowledge
|
||||||
|
|
||||||
|
Before extracting, check what already exists to avoid duplicates:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# List existing claims in the target domain
|
||||||
|
ls domains/{domain}/
|
||||||
|
|
||||||
|
# Read titles — each filename IS a claim
|
||||||
|
# Check for semantic overlap with what you're about to extract
|
||||||
|
```
|
||||||
|
|
||||||
|
Also scan:
|
||||||
|
- `foundations/` — domain-independent theory
|
||||||
|
- `core/` — shared worldview and axioms
|
||||||
|
- The domain agent's beliefs: `agents/{agent}/beliefs.md`
|
||||||
|
|
||||||
|
## Step 6: Extract Claims
|
||||||
|
|
||||||
|
Follow `skills/extract.md`. For each claim:
|
||||||
|
|
||||||
|
1. **Title IS the claim.** Must pass: "This note argues that [title]" works as a sentence.
|
||||||
|
- Good: `OpenAI's shift to capped-profit created structural misalignment between safety mission and fiduciary obligations.md`
|
||||||
|
- Bad: `OpenAI corporate structure.md`
|
||||||
|
|
||||||
|
2. **Frontmatter:**
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
type: claim
|
||||||
|
domain: {domain}
|
||||||
|
description: "one sentence adding context beyond the title"
|
||||||
|
confidence: proven | likely | experimental | speculative
|
||||||
|
source: "{contributor name} — based on {source reference}"
|
||||||
|
created: YYYY-MM-DD
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Body:**
|
||||||
|
```markdown
|
||||||
|
# [claim title as prose]
|
||||||
|
|
||||||
|
[Argument — why this is supported, evidence]
|
||||||
|
|
||||||
|
[Inline evidence: cite sources, data, quotes directly in prose]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
Relevant Notes:
|
||||||
|
- [[existing-claim-title]] — how it connects
|
||||||
|
- [[another-claim]] — relationship
|
||||||
|
|
||||||
|
Topics:
|
||||||
|
- [[domain-map]]
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **File location:** `domains/{domain}/{slugified-title}.md`
|
||||||
|
|
||||||
|
5. **Quality gates (what reviewers check):**
|
||||||
|
- Specific enough to disagree with
|
||||||
|
- Traceable evidence in the body
|
||||||
|
- Description adds info beyond the title
|
||||||
|
- Confidence matches evidence strength
|
||||||
|
- Not a duplicate of existing claim
|
||||||
|
- Contradictions are explicit and argued
|
||||||
|
- Genuinely expands the knowledge base
|
||||||
|
- All `[[wiki links]]` point to real files
|
||||||
|
|
||||||
|
## Step 7: Update Source Archive
|
||||||
|
|
||||||
|
After extraction, update the source file:
|
||||||
|
```yaml
|
||||||
|
status: processed
|
||||||
|
processed_by: "{contributor name}"
|
||||||
|
processed_date: YYYY-MM-DD
|
||||||
|
claims_extracted:
|
||||||
|
- "claim title 1"
|
||||||
|
- "claim title 2"
|
||||||
|
enrichments:
|
||||||
|
- "existing claim that was enriched"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 8: Commit
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add domains/{domain}/*.md inbox/archive/*.md
|
||||||
|
git commit -m "{agent}/contrib-{user}: add N claims about {topic}
|
||||||
|
|
||||||
|
- What: [brief description of claims added]
|
||||||
|
- Why: [source material, why these matter]
|
||||||
|
- Connections: [what existing claims these relate to]
|
||||||
|
|
||||||
|
Contributor: {user's name}"
|
||||||
|
```
|
||||||
|
|
||||||
|
The `Contributor:` trailer is required for human contributions — it ensures attribution.
|
||||||
|
|
||||||
|
## Step 9: Push and Open PR
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git push -u origin {branch-name}
|
||||||
|
|
||||||
|
gh pr create \
|
||||||
|
--title "{agent}/contrib-{user}: {brief description}" \
|
||||||
|
--body "## Source
|
||||||
|
{source title and link}
|
||||||
|
|
||||||
|
## Claims Proposed
|
||||||
|
{numbered list of claim titles}
|
||||||
|
|
||||||
|
## Why These Matter
|
||||||
|
{1-2 sentences on value add}
|
||||||
|
|
||||||
|
## Contributor
|
||||||
|
{user's name}
|
||||||
|
|
||||||
|
## Cross-Domain Flags
|
||||||
|
{any connections to other domains the reviewers should check}"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 10: What Happens Next
|
||||||
|
|
||||||
|
Tell the user:
|
||||||
|
|
||||||
|
> Your PR is open. Two reviewers will evaluate it:
|
||||||
|
> 1. **Leo** — checks quality gates, cross-domain connections, overall coherence
|
||||||
|
> 2. **{Domain agent}** — checks domain expertise, duplicates within the domain, technical accuracy
|
||||||
|
>
|
||||||
|
> You'll see their feedback as PR comments on GitHub. If they request changes, update your branch and push — they'll re-review automatically.
|
||||||
|
>
|
||||||
|
> Your source archive records you as contributor. As claims derived from your work get cited by other claims, your contribution's impact grows through the knowledge graph.
|
||||||
|
|
||||||
|
## OPSEC
|
||||||
|
|
||||||
|
Before committing, verify:
|
||||||
|
- No dollar amounts, deal terms, or valuations
|
||||||
|
- No internal business details
|
||||||
|
- No private communications or confidential information
|
||||||
|
- When in doubt, ask the user before pushing
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
|
||||||
|
- **Dirty working tree:** Stash or commit existing changes before starting
|
||||||
|
- **Branch conflict:** If the branch name exists, append a number or use a different slug
|
||||||
|
- **gh not authenticated:** Tell the user to run `gh auth login`
|
||||||
|
- **Merge conflicts on main:** `git pull --rebase origin main` before pushing
|
||||||
|
|
@ -1,10 +1,15 @@
|
||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# evaluate-trigger.sh — Find unreviewed PRs and run headless Leo on each.
|
# evaluate-trigger.sh — Find unreviewed PRs and run 2-agent review on each.
|
||||||
|
#
|
||||||
|
# Reviews each PR with TWO agents:
|
||||||
|
# 1. Leo (evaluator) — quality gates, cross-domain connections, coherence
|
||||||
|
# 2. Domain agent — domain expertise, duplicate check, technical accuracy
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# ./ops/evaluate-trigger.sh # review all unreviewed open PRs
|
# ./ops/evaluate-trigger.sh # review all unreviewed open PRs
|
||||||
# ./ops/evaluate-trigger.sh 47 # review a specific PR by number
|
# ./ops/evaluate-trigger.sh 47 # review a specific PR by number
|
||||||
# ./ops/evaluate-trigger.sh --dry-run # show what would be reviewed, don't run
|
# ./ops/evaluate-trigger.sh --dry-run # show what would be reviewed, don't run
|
||||||
|
# ./ops/evaluate-trigger.sh --leo-only # skip domain agent, just run Leo
|
||||||
#
|
#
|
||||||
# Requirements:
|
# Requirements:
|
||||||
# - claude CLI (claude -p for headless mode)
|
# - claude CLI (claude -p for headless mode)
|
||||||
|
|
@ -13,10 +18,10 @@
|
||||||
#
|
#
|
||||||
# Safety:
|
# Safety:
|
||||||
# - Lockfile prevents concurrent runs
|
# - Lockfile prevents concurrent runs
|
||||||
# - Leo does NOT auto-merge — posts review only
|
# - Neither agent auto-merges — reviews only
|
||||||
# - Each PR runs sequentially to avoid branch conflicts
|
# - Each PR runs sequentially to avoid branch conflicts
|
||||||
# - Timeout: 10 minutes per PR (kills runaway sessions)
|
# - Timeout: 10 minutes per agent per PR
|
||||||
# - Pre-flight checks: clean working tree, gh auth, on main branch
|
# - Pre-flight checks: clean working tree, gh auth
|
||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
|
|
@ -30,15 +35,52 @@ LOCKFILE="/tmp/evaluate-trigger.lock"
|
||||||
LOG_DIR="$REPO_ROOT/ops/sessions"
|
LOG_DIR="$REPO_ROOT/ops/sessions"
|
||||||
TIMEOUT_SECONDS=600
|
TIMEOUT_SECONDS=600
|
||||||
DRY_RUN=false
|
DRY_RUN=false
|
||||||
|
LEO_ONLY=false
|
||||||
SPECIFIC_PR=""
|
SPECIFIC_PR=""
|
||||||
|
|
||||||
|
# --- Domain routing map ---
|
||||||
|
# Maps branch prefix or domain directory to agent name and identity path
|
||||||
|
detect_domain_agent() {
|
||||||
|
local pr_number="$1"
|
||||||
|
local branch files domain agent
|
||||||
|
|
||||||
|
branch=$(gh pr view "$pr_number" --json headRefName --jq '.headRefName' 2>/dev/null || echo "")
|
||||||
|
files=$(gh pr view "$pr_number" --json files --jq '.files[].path' 2>/dev/null || echo "")
|
||||||
|
|
||||||
|
# Try branch prefix first
|
||||||
|
case "$branch" in
|
||||||
|
rio/*|*/internet-finance*) agent="rio"; domain="internet-finance" ;;
|
||||||
|
clay/*|*/entertainment*) agent="clay"; domain="entertainment" ;;
|
||||||
|
theseus/*|logos/*|*/ai-alignment*) agent="logos"; domain="ai-alignment" ;;
|
||||||
|
vida/*|*/health*) agent="vida"; domain="health" ;;
|
||||||
|
leo/*|*/grand-strategy*) agent="leo"; domain="grand-strategy" ;;
|
||||||
|
*)
|
||||||
|
# Fall back to checking which domain directory has changed files
|
||||||
|
if echo "$files" | grep -q "domains/internet-finance/"; then
|
||||||
|
agent="rio"; domain="internet-finance"
|
||||||
|
elif echo "$files" | grep -q "domains/entertainment/"; then
|
||||||
|
agent="clay"; domain="entertainment"
|
||||||
|
elif echo "$files" | grep -q "domains/ai-alignment/"; then
|
||||||
|
agent="logos"; domain="ai-alignment"
|
||||||
|
elif echo "$files" | grep -q "domains/health/"; then
|
||||||
|
agent="vida"; domain="health"
|
||||||
|
else
|
||||||
|
agent=""; domain=""
|
||||||
|
fi
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
echo "$agent $domain"
|
||||||
|
}
|
||||||
|
|
||||||
# --- Parse arguments ---
|
# --- Parse arguments ---
|
||||||
for arg in "$@"; do
|
for arg in "$@"; do
|
||||||
case "$arg" in
|
case "$arg" in
|
||||||
--dry-run) DRY_RUN=true ;;
|
--dry-run) DRY_RUN=true ;;
|
||||||
|
--leo-only) LEO_ONLY=true ;;
|
||||||
[0-9]*) SPECIFIC_PR="$arg" ;;
|
[0-9]*) SPECIFIC_PR="$arg" ;;
|
||||||
--help|-h)
|
--help|-h)
|
||||||
head -19 "$0" | tail -17
|
head -23 "$0" | tail -21
|
||||||
exit 0
|
exit 0
|
||||||
;;
|
;;
|
||||||
*)
|
*)
|
||||||
|
|
@ -59,8 +101,8 @@ if ! command -v claude >/dev/null 2>&1; then
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Check for dirty working tree (ignore ops/ which may contain uncommitted scripts)
|
# Check for dirty working tree (ignore ops/ and .claude/ which may contain uncommitted scripts)
|
||||||
DIRTY_FILES=$(git status --porcelain | grep -v '^?? ops/' | grep -v '^ M ops/' || true)
|
DIRTY_FILES=$(git status --porcelain | grep -v '^?? ops/' | grep -v '^ M ops/' | grep -v '^?? \.claude/' | grep -v '^ M \.claude/' || true)
|
||||||
if [ -n "$DIRTY_FILES" ]; then
|
if [ -n "$DIRTY_FILES" ]; then
|
||||||
echo "ERROR: Working tree is dirty. Clean up before running."
|
echo "ERROR: Working tree is dirty. Clean up before running."
|
||||||
echo "$DIRTY_FILES"
|
echo "$DIRTY_FILES"
|
||||||
|
|
@ -86,14 +128,12 @@ mkdir -p "$LOG_DIR"
|
||||||
|
|
||||||
# --- Find PRs to review ---
|
# --- Find PRs to review ---
|
||||||
if [ -n "$SPECIFIC_PR" ]; then
|
if [ -n "$SPECIFIC_PR" ]; then
|
||||||
# Review a specific PR
|
|
||||||
PR_STATE=$(gh pr view "$SPECIFIC_PR" --json state --jq '.state' 2>/dev/null || echo "NOT_FOUND")
|
PR_STATE=$(gh pr view "$SPECIFIC_PR" --json state --jq '.state' 2>/dev/null || echo "NOT_FOUND")
|
||||||
if [ "$PR_STATE" != "OPEN" ]; then
|
if [ "$PR_STATE" != "OPEN" ]; then
|
||||||
echo "PR #$SPECIFIC_PR is $PR_STATE (not OPEN). Reviewing anyway for testing."
|
echo "PR #$SPECIFIC_PR is $PR_STATE (not OPEN). Reviewing anyway for testing."
|
||||||
fi
|
fi
|
||||||
PRS_TO_REVIEW="$SPECIFIC_PR"
|
PRS_TO_REVIEW="$SPECIFIC_PR"
|
||||||
else
|
else
|
||||||
# Find open PRs that need (re-)review
|
|
||||||
OPEN_PRS=$(gh pr list --state open --json number --jq '.[].number' 2>/dev/null || echo "")
|
OPEN_PRS=$(gh pr list --state open --json number --jq '.[].number' 2>/dev/null || echo "")
|
||||||
|
|
||||||
if [ -z "$OPEN_PRS" ]; then
|
if [ -z "$OPEN_PRS" ]; then
|
||||||
|
|
@ -103,16 +143,13 @@ else
|
||||||
|
|
||||||
PRS_TO_REVIEW=""
|
PRS_TO_REVIEW=""
|
||||||
for pr in $OPEN_PRS; do
|
for pr in $OPEN_PRS; do
|
||||||
# Check if there are new commits since the last review
|
|
||||||
LAST_REVIEW_DATE=$(gh api "repos/{owner}/{repo}/pulls/$pr/reviews" \
|
LAST_REVIEW_DATE=$(gh api "repos/{owner}/{repo}/pulls/$pr/reviews" \
|
||||||
--jq 'map(select(.state != "DISMISSED")) | sort_by(.submitted_at) | last | .submitted_at' 2>/dev/null || echo "")
|
--jq 'map(select(.state != "DISMISSED")) | sort_by(.submitted_at) | last | .submitted_at' 2>/dev/null || echo "")
|
||||||
LAST_COMMIT_DATE=$(gh pr view "$pr" --json commits --jq '.commits[-1].committedDate' 2>/dev/null || echo "")
|
LAST_COMMIT_DATE=$(gh pr view "$pr" --json commits --jq '.commits[-1].committedDate' 2>/dev/null || echo "")
|
||||||
|
|
||||||
if [ -z "$LAST_REVIEW_DATE" ]; then
|
if [ -z "$LAST_REVIEW_DATE" ]; then
|
||||||
# No reviews yet — needs review
|
|
||||||
PRS_TO_REVIEW="$PRS_TO_REVIEW $pr"
|
PRS_TO_REVIEW="$PRS_TO_REVIEW $pr"
|
||||||
elif [ -n "$LAST_COMMIT_DATE" ] && [[ "$LAST_COMMIT_DATE" > "$LAST_REVIEW_DATE" ]]; then
|
elif [ -n "$LAST_COMMIT_DATE" ] && [[ "$LAST_COMMIT_DATE" > "$LAST_REVIEW_DATE" ]]; then
|
||||||
# New commits after last review — needs re-review
|
|
||||||
echo "PR #$pr: New commits since last review. Queuing for re-review."
|
echo "PR #$pr: New commits since last review. Queuing for re-review."
|
||||||
PRS_TO_REVIEW="$PRS_TO_REVIEW $pr"
|
PRS_TO_REVIEW="$PRS_TO_REVIEW $pr"
|
||||||
else
|
else
|
||||||
|
|
@ -131,25 +168,61 @@ fi
|
||||||
echo "PRs to review: $PRS_TO_REVIEW"
|
echo "PRs to review: $PRS_TO_REVIEW"
|
||||||
|
|
||||||
if [ "$DRY_RUN" = true ]; then
|
if [ "$DRY_RUN" = true ]; then
|
||||||
echo "[DRY RUN] Would review PRs: $PRS_TO_REVIEW"
|
for pr in $PRS_TO_REVIEW; do
|
||||||
|
read -r agent domain <<< "$(detect_domain_agent "$pr")"
|
||||||
|
echo "[DRY RUN] PR #$pr — Leo + ${agent:-unknown} (${domain:-unknown domain})"
|
||||||
|
done
|
||||||
exit 0
|
exit 0
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# --- Run headless Leo on each PR ---
|
# --- Run headless reviews on each PR ---
|
||||||
|
run_agent_review() {
|
||||||
|
local pr="$1" agent_name="$2" prompt="$3" model="$4"
|
||||||
|
local timestamp log_file review_file
|
||||||
|
|
||||||
|
timestamp=$(date +%Y%m%d-%H%M%S)
|
||||||
|
log_file="$LOG_DIR/${agent_name}-review-pr${pr}-${timestamp}.log"
|
||||||
|
review_file="/tmp/${agent_name}-review-pr${pr}.md"
|
||||||
|
|
||||||
|
echo " Running ${agent_name}..."
|
||||||
|
echo " Log: $log_file"
|
||||||
|
|
||||||
|
if perl -e "alarm $TIMEOUT_SECONDS; exec @ARGV" claude -p \
|
||||||
|
--model "$model" \
|
||||||
|
--allowedTools "Read,Write,Edit,Bash,Glob,Grep" \
|
||||||
|
--permission-mode bypassPermissions \
|
||||||
|
"$prompt" \
|
||||||
|
> "$log_file" 2>&1; then
|
||||||
|
echo " ${agent_name}: Review posted."
|
||||||
|
rm -f "$review_file"
|
||||||
|
return 0
|
||||||
|
else
|
||||||
|
local exit_code=$?
|
||||||
|
if [ "$exit_code" -eq 142 ] || [ "$exit_code" -eq 124 ]; then
|
||||||
|
echo " ${agent_name}: TIMEOUT after ${TIMEOUT_SECONDS}s."
|
||||||
|
else
|
||||||
|
echo " ${agent_name}: FAILED (exit code $exit_code)."
|
||||||
|
fi
|
||||||
|
rm -f "$review_file"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
REVIEWED=0
|
REVIEWED=0
|
||||||
FAILED=0
|
FAILED=0
|
||||||
|
|
||||||
for pr in $PRS_TO_REVIEW; do
|
for pr in $PRS_TO_REVIEW; do
|
||||||
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
|
|
||||||
LOG_FILE="$LOG_DIR/leo-review-pr${pr}-${TIMESTAMP}.log"
|
|
||||||
REVIEW_FILE="/tmp/leo-review-pr${pr}.md"
|
|
||||||
|
|
||||||
echo ""
|
echo ""
|
||||||
echo "=== Reviewing PR #$pr ==="
|
echo "=== PR #$pr ==="
|
||||||
echo "Log: $LOG_FILE"
|
|
||||||
echo "Started: $(date)"
|
echo "Started: $(date)"
|
||||||
|
|
||||||
PROMPT="You are Leo. Read agents/leo/identity.md, agents/leo/beliefs.md, agents/leo/reasoning.md, and skills/evaluate.md.
|
# Detect which domain agent should review
|
||||||
|
read -r DOMAIN_AGENT DOMAIN <<< "$(detect_domain_agent "$pr")"
|
||||||
|
echo "Domain: ${DOMAIN:-unknown} | Agent: ${DOMAIN_AGENT:-none detected}"
|
||||||
|
|
||||||
|
# --- Review 1: Leo (evaluator) ---
|
||||||
|
LEO_REVIEW_FILE="/tmp/leo-review-pr${pr}.md"
|
||||||
|
LEO_PROMPT="You are Leo. Read agents/leo/identity.md, agents/leo/beliefs.md, agents/leo/reasoning.md, and skills/evaluate.md.
|
||||||
|
|
||||||
Review PR #${pr} on this repo.
|
Review PR #${pr} on this repo.
|
||||||
|
|
||||||
|
|
@ -158,7 +231,7 @@ Then checkout the PR branch: gh pr checkout ${pr}
|
||||||
Read every changed file completely.
|
Read every changed file completely.
|
||||||
|
|
||||||
Before evaluating, scan the existing knowledge base for duplicate and contradiction checks:
|
Before evaluating, scan the existing knowledge base for duplicate and contradiction checks:
|
||||||
- List claim files in the relevant domain directory (e.g., domains/internet-finance/, domains/ai-alignment/)
|
- List claim files in the relevant domain directory (e.g., domains/${DOMAIN}/)
|
||||||
- Read titles to check for semantic duplicates
|
- Read titles to check for semantic duplicates
|
||||||
- Check for contradictions with existing claims in that domain and in foundations/
|
- Check for contradictions with existing claims in that domain and in foundations/
|
||||||
|
|
||||||
|
|
@ -178,47 +251,76 @@ Also check:
|
||||||
- Files are in the correct domain directory
|
- Files are in the correct domain directory
|
||||||
- Cross-domain connections that the proposer may have missed
|
- Cross-domain connections that the proposer may have missed
|
||||||
|
|
||||||
Write your complete review to ${REVIEW_FILE}
|
Write your complete review to ${LEO_REVIEW_FILE}
|
||||||
Then post it with: gh pr review ${pr} --comment --body-file ${REVIEW_FILE}
|
Then post it with: gh pr review ${pr} --comment --body-file ${LEO_REVIEW_FILE}
|
||||||
|
|
||||||
If ALL claims pass quality gates: gh pr review ${pr} --approve --body-file ${REVIEW_FILE}
|
If ALL claims pass quality gates: gh pr review ${pr} --approve --body-file ${LEO_REVIEW_FILE}
|
||||||
If ANY claim needs changes: gh pr review ${pr} --request-changes --body-file ${REVIEW_FILE}
|
If ANY claim needs changes: gh pr review ${pr} --request-changes --body-file ${LEO_REVIEW_FILE}
|
||||||
|
|
||||||
DO NOT merge. Leave the merge decision to Cory.
|
DO NOT merge. Leave the merge decision to Cory.
|
||||||
Work autonomously. Do not ask for confirmation."
|
Work autonomously. Do not ask for confirmation."
|
||||||
|
|
||||||
# Run headless Leo with timeout (perl-based, works on macOS without coreutils)
|
if run_agent_review "$pr" "leo" "$LEO_PROMPT" "opus"; then
|
||||||
if perl -e "alarm $TIMEOUT_SECONDS; exec @ARGV" claude -p \
|
LEO_PASSED=true
|
||||||
--model opus \
|
else
|
||||||
--allowedTools "Read,Write,Edit,Bash,Glob,Grep" \
|
LEO_PASSED=false
|
||||||
--permission-mode bypassPermissions \
|
fi
|
||||||
"$PROMPT" \
|
|
||||||
> "$LOG_FILE" 2>&1; then
|
# Return to main between reviews
|
||||||
echo "PR #$pr: Review complete."
|
git checkout main 2>/dev/null || git checkout -f main
|
||||||
|
PR_BRANCH=$(gh pr view "$pr" --json headRefName --jq '.headRefName' 2>/dev/null || echo "")
|
||||||
|
[ -n "$PR_BRANCH" ] && git branch -D "$PR_BRANCH" 2>/dev/null || true
|
||||||
|
|
||||||
|
# --- Review 2: Domain agent ---
|
||||||
|
if [ "$LEO_ONLY" = true ]; then
|
||||||
|
echo " Skipping domain agent review (--leo-only)."
|
||||||
|
elif [ -z "$DOMAIN_AGENT" ]; then
|
||||||
|
echo " Could not detect domain agent. Skipping domain review."
|
||||||
|
elif [ "$DOMAIN_AGENT" = "leo" ]; then
|
||||||
|
echo " Domain is grand-strategy (Leo's territory). Single review sufficient."
|
||||||
|
else
|
||||||
|
DOMAIN_REVIEW_FILE="/tmp/${DOMAIN_AGENT}-review-pr${pr}.md"
|
||||||
|
DOMAIN_PROMPT="You are ${DOMAIN_AGENT^}. Read agents/${DOMAIN_AGENT}/identity.md, agents/${DOMAIN_AGENT}/beliefs.md, and skills/evaluate.md.
|
||||||
|
|
||||||
|
You are reviewing PR #${pr} as the domain expert for ${DOMAIN}.
|
||||||
|
|
||||||
|
First, run: gh pr view ${pr} --json title,body,files,additions,deletions
|
||||||
|
Then checkout the PR branch: gh pr checkout ${pr}
|
||||||
|
Read every changed file completely.
|
||||||
|
|
||||||
|
Your review focuses on DOMAIN EXPERTISE — things only a ${DOMAIN} specialist would catch:
|
||||||
|
|
||||||
|
1. **Technical accuracy** — Are the claims factually correct within the ${DOMAIN} domain?
|
||||||
|
2. **Domain duplicates** — Do any claims duplicate existing knowledge in domains/${DOMAIN}/?
|
||||||
|
Scan the directory and read titles carefully.
|
||||||
|
3. **Missing context** — What important nuance from the ${DOMAIN} domain is the claim missing?
|
||||||
|
4. **Belief impact** — Do any claims affect your current beliefs? Read agents/${DOMAIN_AGENT}/beliefs.md
|
||||||
|
and flag if any belief needs updating.
|
||||||
|
5. **Connections** — What existing claims in your domain should be wiki-linked?
|
||||||
|
6. **Confidence calibration** — From your domain expertise, is the confidence level right?
|
||||||
|
|
||||||
|
Write your review to ${DOMAIN_REVIEW_FILE}
|
||||||
|
Post it with: gh pr review ${pr} --comment --body-file ${DOMAIN_REVIEW_FILE}
|
||||||
|
|
||||||
|
Sign your review as ${DOMAIN_AGENT^} (domain reviewer for ${DOMAIN}).
|
||||||
|
DO NOT duplicate Leo's quality gate checks — he covers those.
|
||||||
|
DO NOT merge.
|
||||||
|
Work autonomously. Do not ask for confirmation."
|
||||||
|
|
||||||
|
run_agent_review "$pr" "$DOMAIN_AGENT" "$DOMAIN_PROMPT" "sonnet"
|
||||||
|
|
||||||
|
# Clean up branch again
|
||||||
|
git checkout main 2>/dev/null || git checkout -f main
|
||||||
|
[ -n "$PR_BRANCH" ] && git branch -D "$PR_BRANCH" 2>/dev/null || true
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$LEO_PASSED" = true ]; then
|
||||||
REVIEWED=$((REVIEWED + 1))
|
REVIEWED=$((REVIEWED + 1))
|
||||||
else
|
else
|
||||||
EXIT_CODE=$?
|
|
||||||
if [ "$EXIT_CODE" -eq 124 ]; then
|
|
||||||
echo "PR #$pr: TIMEOUT after ${TIMEOUT_SECONDS}s. Check log."
|
|
||||||
else
|
|
||||||
echo "PR #$pr: FAILED (exit code $EXIT_CODE). Check log."
|
|
||||||
fi
|
|
||||||
FAILED=$((FAILED + 1))
|
FAILED=$((FAILED + 1))
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "Finished: $(date)"
|
echo "Finished: $(date)"
|
||||||
|
|
||||||
# Clean up review temp file
|
|
||||||
rm -f "$REVIEW_FILE"
|
|
||||||
|
|
||||||
# Return to main branch and clean up PR branch
|
|
||||||
PR_BRANCH=$(gh pr view "$pr" --json headRefName --jq '.headRefName' 2>/dev/null || echo "")
|
|
||||||
if ! git checkout main 2>/dev/null; then
|
|
||||||
echo "WARNING: Could not checkout main. Forcing reset."
|
|
||||||
git checkout -f main
|
|
||||||
git clean -fd
|
|
||||||
fi
|
|
||||||
[ -n "$PR_BRANCH" ] && git branch -D "$PR_BRANCH" 2>/dev/null || true
|
|
||||||
done
|
done
|
||||||
|
|
||||||
echo ""
|
echo ""
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue