Compare commits
7 commits
d2beae7c2a
...
2bd3f70bfa
| Author | SHA1 | Date | |
|---|---|---|---|
| 2bd3f70bfa | |||
| ea6e75c58e | |||
| 786c33bfbf | |||
| cb0566696d | |||
| b5069fb4e7 | |||
| 63089abe63 | |||
| c9e2970cfb |
9 changed files with 1056 additions and 80 deletions
67
.github/workflows/sync-graph-data.yml
vendored
Normal file
67
.github/workflows/sync-graph-data.yml
vendored
Normal file
|
|
@ -0,0 +1,67 @@
|
||||||
|
name: Sync Graph Data to teleo-app
|
||||||
|
|
||||||
|
# Runs on every merge to main. Extracts graph data from the codex and
|
||||||
|
# pushes graph-data.json + claims-context.json to teleo-app/public/.
|
||||||
|
# This triggers a Vercel rebuild automatically.
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [main]
|
||||||
|
paths:
|
||||||
|
- 'core/**'
|
||||||
|
- 'domains/**'
|
||||||
|
- 'foundations/**'
|
||||||
|
- 'convictions/**'
|
||||||
|
- 'ops/extract-graph-data.py'
|
||||||
|
workflow_dispatch: # manual trigger
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
sync:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- name: Checkout teleo-codex
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
fetch-depth: 0 # full history for git log agent attribution
|
||||||
|
|
||||||
|
- name: Set up Python
|
||||||
|
uses: actions/setup-python@v5
|
||||||
|
with:
|
||||||
|
python-version: '3.12'
|
||||||
|
|
||||||
|
- name: Run extraction
|
||||||
|
run: |
|
||||||
|
python3 ops/extract-graph-data.py \
|
||||||
|
--repo . \
|
||||||
|
--output /tmp/graph-data.json \
|
||||||
|
--context-output /tmp/claims-context.json
|
||||||
|
|
||||||
|
- name: Checkout teleo-app
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
repository: living-ip/teleo-app
|
||||||
|
token: ${{ secrets.TELEO_APP_TOKEN }}
|
||||||
|
path: teleo-app
|
||||||
|
|
||||||
|
- name: Copy data files
|
||||||
|
run: |
|
||||||
|
cp /tmp/graph-data.json teleo-app/public/graph-data.json
|
||||||
|
cp /tmp/claims-context.json teleo-app/public/claims-context.json
|
||||||
|
|
||||||
|
- name: Commit and push to teleo-app
|
||||||
|
working-directory: teleo-app
|
||||||
|
run: |
|
||||||
|
git config user.name "teleo-codex-bot"
|
||||||
|
git config user.email "bot@livingip.io"
|
||||||
|
git add public/graph-data.json public/claims-context.json
|
||||||
|
if git diff --cached --quiet; then
|
||||||
|
echo "No changes to commit"
|
||||||
|
else
|
||||||
|
NODES=$(python3 -c "import json; d=json.load(open('public/graph-data.json')); print(len(d['nodes']))")
|
||||||
|
EDGES=$(python3 -c "import json; d=json.load(open('public/graph-data.json')); print(len(d['edges']))")
|
||||||
|
git commit -m "sync: graph data from teleo-codex ($NODES nodes, $EDGES edges)"
|
||||||
|
git push
|
||||||
|
fi
|
||||||
30
inbox/archive/2026-02-24-karpathy-clis-legacy-tech-agents.md
Normal file
30
inbox/archive/2026-02-24-karpathy-clis-legacy-tech-agents.md
Normal file
|
|
@ -0,0 +1,30 @@
|
||||||
|
---
|
||||||
|
type: source
|
||||||
|
title: "CLIs are exciting because they're legacy technology — AI agents can natively use them, combine them, interact via terminal"
|
||||||
|
author: "Andrej Karpathy (@karpathy)"
|
||||||
|
twitter_id: "33836629"
|
||||||
|
url: https://x.com/karpathy/status/2026360908398862478
|
||||||
|
date: 2026-02-24
|
||||||
|
domain: ai-alignment
|
||||||
|
secondary_domains: [teleological-economics]
|
||||||
|
format: tweet
|
||||||
|
status: unprocessed
|
||||||
|
priority: medium
|
||||||
|
tags: [cli, agents, terminal, developer-tools, legacy-systems]
|
||||||
|
---
|
||||||
|
|
||||||
|
## Content
|
||||||
|
|
||||||
|
CLIs are super exciting precisely because they are a "legacy" technology, which means AI agents can natively and easily use them, combine them, interact with them via the entire terminal toolkit.
|
||||||
|
|
||||||
|
E.g ask your Claude/Codex agent to install this new Polymarket CLI and ask for any arbitrary dashboards or interfaces or logic. The agents will build it for you. Install the Github CLI too and you can ask them to navigate the repo, see issues, PRs, discussions, even the code itself.
|
||||||
|
|
||||||
|
## Agent Notes
|
||||||
|
|
||||||
|
**Why this matters:** 11.7K likes. This is the theoretical justification for why Claude Code (CLI-based) is structurally advantaged over GUI-based AI interfaces. Legacy text protocols are more agent-friendly than modern visual interfaces. This is relevant to our own architecture — the agents work through git CLI, Forgejo API, terminal tools.
|
||||||
|
|
||||||
|
**KB connections:** Validates our architectural choice of CLI-based agent coordination. Connects to [[collaborative knowledge infrastructure requires separating the versioning problem from the knowledge evolution problem because git solves file history but not semantic disagreement]].
|
||||||
|
|
||||||
|
**Extraction hints:** Claim: legacy text-based interfaces (CLIs) are structurally more accessible to AI agents than modern GUI interfaces because they were designed for composability and programmatic interaction.
|
||||||
|
|
||||||
|
**Context:** Karpathy explicitly mentions Claude and Polymarket CLI — connecting AI agents with prediction markets through terminal tools. Relevant to the Teleo stack.
|
||||||
|
|
@ -0,0 +1,28 @@
|
||||||
|
---
|
||||||
|
type: source
|
||||||
|
title: "Programming fundamentally changed in December 2025 — coding agents basically didn't work before and basically work since"
|
||||||
|
author: "Andrej Karpathy (@karpathy)"
|
||||||
|
twitter_id: "33836629"
|
||||||
|
url: https://x.com/karpathy/status/2026731645169185220
|
||||||
|
date: 2026-02-25
|
||||||
|
domain: ai-alignment
|
||||||
|
secondary_domains: [teleological-economics]
|
||||||
|
format: tweet
|
||||||
|
status: unprocessed
|
||||||
|
priority: medium
|
||||||
|
tags: [coding-agents, ai-capability, phase-transition, software-development, disruption]
|
||||||
|
---
|
||||||
|
|
||||||
|
## Content
|
||||||
|
|
||||||
|
It is hard to communicate how much programming has changed due to AI in the last 2 months: not gradually and over time in the "progress as usual" way, but specifically this last December. There are a number of asterisks but imo coding agents basically didn't work before December and basically work since - the models have significantly higher quality, long-term coherence and tenacity and they can power through large and long tasks, well past enough that it is extremely disruptive to the default programming workflow.
|
||||||
|
|
||||||
|
## Agent Notes
|
||||||
|
|
||||||
|
**Why this matters:** 37K likes — Karpathy's most viral tweet in this dataset. This is the "phase transition" observation from the most authoritative voice in AI dev tooling. December 2025 as the inflection point for coding agents.
|
||||||
|
|
||||||
|
**KB connections:** Supports [[as AI-automated software development becomes certain the bottleneck shifts from building capacity to knowing what to build]]. Relates to [[the gap between theoretical AI capability and observed deployment is massive across all occupations]] — but suggests the gap is closing fast for software specifically.
|
||||||
|
|
||||||
|
**Extraction hints:** Claim candidate: coding agent capability crossed a usability threshold in December 2025, representing a phase transition not gradual improvement. Evidence: Karpathy's direct experience running agents on nanochat.
|
||||||
|
|
||||||
|
**Context:** This tweet preceded the autoresearch project by ~10 days. The 37K likes suggest massive resonance across the developer community. The "asterisks" he mentions are important qualifiers that a good extraction should preserve.
|
||||||
44
inbox/archive/2026-02-27-karpathy-8-agent-research-org.md
Normal file
44
inbox/archive/2026-02-27-karpathy-8-agent-research-org.md
Normal file
|
|
@ -0,0 +1,44 @@
|
||||||
|
---
|
||||||
|
type: source
|
||||||
|
title: "8-agent research org experiments reveal agents generate bad ideas but execute well — the source code is now the org design"
|
||||||
|
author: "Andrej Karpathy (@karpathy)"
|
||||||
|
twitter_id: "33836629"
|
||||||
|
url: https://x.com/karpathy/status/2027521323275325622
|
||||||
|
date: 2026-02-27
|
||||||
|
domain: ai-alignment
|
||||||
|
secondary_domains: [collective-intelligence]
|
||||||
|
format: tweet
|
||||||
|
status: unprocessed
|
||||||
|
priority: high
|
||||||
|
tags: [multi-agent, research-org, agent-collaboration, prompt-engineering, organizational-design]
|
||||||
|
flagged_for_theseus: ["Multi-model collaboration evidence — 8 agents, different setups, empirical failure modes"]
|
||||||
|
---
|
||||||
|
|
||||||
|
## Content
|
||||||
|
|
||||||
|
I had the same thought so I've been playing with it in nanochat. E.g. here's 8 agents (4 claude, 4 codex), with 1 GPU each running nanochat experiments (trying to delete logit softcap without regression). The TLDR is that it doesn't work and it's a mess... but it's still very pretty to look at :)
|
||||||
|
|
||||||
|
I tried a few setups: 8 independent solo researchers, 1 chief scientist giving work to 8 junior researchers, etc. Each research program is a git branch, each scientist forks it into a feature branch, git worktrees for isolation, simple files for comms, skip Docker/VMs for simplicity atm (I find that instructions are enough to prevent interference). Research org runs in tmux window grids of interactive sessions (like Teams) so that it's pretty to look at, see their individual work, and "take over" if needed, i.e. no -p.
|
||||||
|
|
||||||
|
But ok the reason it doesn't work so far is that the agents' ideas are just pretty bad out of the box, even at highest intelligence. They don't think carefully though experiment design, they run a bit non-sensical variations, they don't create strong baselines and ablate things properly, they don't carefully control for runtime or flops. (just as an example, an agent yesterday "discovered" that increasing the hidden size of the network improves the validation loss, which is a totally spurious result given that a bigger network will have a lower validation loss in the infinite data regime, but then it also trains for a lot longer, it's not clear why I had to come in to point that out). They are very good at implementing any given well-scoped and described idea but they don't creatively generate them.
|
||||||
|
|
||||||
|
But the goal is that you are now programming an organization (e.g. a "research org") and its individual agents, so the "source code" is the collection of prompts, skills, tools, etc. and processes that make it up. E.g. a daily standup in the morning is now part of the "org code". And optimizing nanochat pretraining is just one of the many tasks (almost like an eval). Then - given an arbitrary task, how quickly does your research org generate progress on it?
|
||||||
|
|
||||||
|
## Agent Notes
|
||||||
|
|
||||||
|
**Why this matters:** This is empirical evidence from the most credible source possible (Karpathy, running 8 agents on real GPU tasks) about what multi-agent collaboration actually looks like today. Key finding: agents execute well but generate bad ideas. They don't do experiment design, don't control for confounds, don't think critically. This is EXACTLY why our adversarial review pipeline matters — without it, agents accumulate spurious results.
|
||||||
|
|
||||||
|
**KB connections:**
|
||||||
|
- Validates [[AI capability and reliability are independent dimensions]] — agents can implement perfectly but reason poorly about what to implement
|
||||||
|
- Validates [[adversarial PR review produces higher quality knowledge than self-review]] — Karpathy had to manually catch a spurious result the agent couldn't see
|
||||||
|
- The "source code is the org design" framing is exactly what Pentagon is: prompts, skills, tools, processes as organizational architecture
|
||||||
|
- Connects to [[coordination protocol design produces larger capability gains than model scaling]] — same agents, different org structure, different results
|
||||||
|
- His 4 claude + 4 codex setup is evidence for [[all agents running the same model family creates correlated blind spots]]
|
||||||
|
|
||||||
|
**Extraction hints:**
|
||||||
|
- Claim: AI agents execute well-scoped tasks reliably but generate poor research hypotheses — the bottleneck is idea generation not implementation
|
||||||
|
- Claim: multi-agent research orgs are now programmable organizations where the source code is prompts, skills, tools and processes
|
||||||
|
- Claim: different organizational structures (solo vs hierarchical) produce different research outcomes with identical agents
|
||||||
|
- Claim: agents fail at experimental methodology (confound control, baseline comparison, ablation) even at highest intelligence settings
|
||||||
|
|
||||||
|
**Context:** Follow-up to the autoresearch SETI@home tweet. Karpathy tried multiple org structures: 8 independent, 1 chief + 8 juniors, etc. Used git worktrees for isolation (we use the same pattern in Pentagon). This is the most detailed public account of someone running a multi-agent research organization.
|
||||||
|
|
@ -0,0 +1,39 @@
|
||||||
|
---
|
||||||
|
type: source
|
||||||
|
title: "Permissionless MetaDAO launches create new cultural primitives around fundraising"
|
||||||
|
author: "Felipe Montealegre (@TheiaResearch)"
|
||||||
|
twitter_id: "1511793131884318720"
|
||||||
|
url: https://x.com/TheiaResearch/status/2029231349425684521
|
||||||
|
date: 2026-03-04
|
||||||
|
domain: internet-finance
|
||||||
|
format: tweet
|
||||||
|
status: unprocessed
|
||||||
|
priority: high
|
||||||
|
tags: [metadao, futardio, fundraising, permissionless-launch, capital-formation]
|
||||||
|
---
|
||||||
|
|
||||||
|
## Content
|
||||||
|
|
||||||
|
Permissionless MetaDAO launches will lead to entirely different cultural primitives around fundraising.
|
||||||
|
|
||||||
|
1. Continuous Fundraising: It only takes a few days to fundraise so don't take more than you need
|
||||||
|
|
||||||
|
2. Liquidation Pivot: You built an MVP but didn't find product-market fit and now you have been liquidated. Try again on another product or strategy.
|
||||||
|
|
||||||
|
3. Multiple Attempts: You didn't fill your minimum raise? Speak to some investors, build out an MVP, put together a deck, and come back in ~3 weeks.
|
||||||
|
|
||||||
|
4. Public on Day 1: Communicating with markets and liquid investors is a core founder skillset.
|
||||||
|
|
||||||
|
5. 10x Upside Case: Many companies with 5-10x upside case outcomes don't get funded right now because venture funds all want venture outcomes (>100x on $20M). What if you just want to build a $25M company with a decent probability of success? Raise $1M and the math works fine for Futardio investors.
|
||||||
|
|
||||||
|
Futardio is a paradigm shift for capital markets. We will fund you - quickly and efficiently - and give you community support but you are public and accountable from day one. Welcome to the arena.
|
||||||
|
|
||||||
|
## Agent Notes
|
||||||
|
|
||||||
|
**Why this matters:** This is the clearest articulation yet of how permissionless futarchy-governed launches create fundamentally different founder behavior — not just faster fundraising but different cultural norms (continuous raises, liquidation as pivot, public accountability from day 1).
|
||||||
|
|
||||||
|
**KB connections:** Directly extends [[internet capital markets compress fundraising from months to days]] and [[futarchy-governed liquidation is the enforcement mechanism that makes unruggable ICOs credible]]. The "10x upside case" point challenges the VC model — connects to [[cryptos primary use case is capital formation not payments or store of value]].
|
||||||
|
|
||||||
|
**Extraction hints:** At least 2-3 claims here: (1) permissionless launches create new fundraising cultural norms, (2) the 10x upside gap in traditional VC is a market failure that futarchy-governed launches solve, (3) public accountability from day 1 is a feature not a bug.
|
||||||
|
|
||||||
|
**Context:** Felipe Montealegre runs Theia Research, a crypto-native investment firm focused on MetaDAO ecosystem. He's been one of the most articulate proponents of the futarchy-governed capital formation thesis. This tweet got 118 likes — high engagement for crypto-finance X.
|
||||||
|
|
@ -0,0 +1,47 @@
|
||||||
|
---
|
||||||
|
type: source
|
||||||
|
title: "Autoresearch must become asynchronously massively collaborative for agents — emulating a research community, not a single PhD student"
|
||||||
|
author: "Andrej Karpathy (@karpathy)"
|
||||||
|
twitter_id: "33836629"
|
||||||
|
url: https://x.com/karpathy/status/2030705271627284816
|
||||||
|
date: 2026-03-08
|
||||||
|
domain: ai-alignment
|
||||||
|
secondary_domains: [collective-intelligence]
|
||||||
|
format: tweet
|
||||||
|
status: unprocessed
|
||||||
|
priority: high
|
||||||
|
tags: [autoresearch, multi-agent, git-coordination, collective-intelligence, agent-collaboration]
|
||||||
|
flagged_for_theseus: ["Core AI agent coordination architecture — directly relevant to multi-model collaboration claims"]
|
||||||
|
flagged_for_leo: ["Cross-domain synthesis — this is what we're building with the Teleo collective"]
|
||||||
|
---
|
||||||
|
|
||||||
|
## Content
|
||||||
|
|
||||||
|
The next step for autoresearch is that it has to be asynchronously massively collaborative for agents (think: SETI@home style). The goal is not to emulate a single PhD student, it's to emulate a research community of them.
|
||||||
|
|
||||||
|
Current code synchronously grows a single thread of commits in a particular research direction. But the original repo is more of a seed, from which could sprout commits contributed by agents on all kinds of different research directions or for different compute platforms. Git(Hub) is *almost* but not really suited for this. It has a softly built in assumption of one "master" branch, which temporarily forks off into PRs just to merge back a bit later.
|
||||||
|
|
||||||
|
I tried to prototype something super lightweight that could have a flavor of this, e.g. just a Discussion, written by my agent as a summary of its overnight run:
|
||||||
|
https://t.co/tmZeqyDY1W
|
||||||
|
Alternatively, a PR has the benefit of exact commits:
|
||||||
|
https://t.co/CZIbuJIqlk
|
||||||
|
but you'd never want to actually merge it... You'd just want to "adopt" and accumulate branches of commits. But even in this lightweight way, you could ask your agent to first read the Discussions/PRs using GitHub CLI for inspiration, and after its research is done, contribute a little "paper" of findings back.
|
||||||
|
|
||||||
|
I'm not actually exactly sure what this should look like, but it's a big idea that is more general than just the autoresearch repo specifically. Agents can in principle easily juggle and collaborate on thousands of commits across arbitrary branch structures. Existing abstractions will accumulate stress as intelligence, attention and tenacity cease to be bottlenecks.
|
||||||
|
|
||||||
|
## Agent Notes
|
||||||
|
|
||||||
|
**Why this matters:** Karpathy (3M+ followers, former Tesla AI director) is independently arriving at the same architecture we're building with the Teleo collective — agents coordinating through git, PRs as knowledge contributions, branches as research directions. His framing of "emulate a research community, not a single PhD student" IS our thesis. And his observation that Git's assumptions break under agent-scale collaboration is a problem we're actively solving.
|
||||||
|
|
||||||
|
**KB connections:**
|
||||||
|
- Directly validates [[coordination protocol design produces larger capability gains than model scaling]]
|
||||||
|
- Challenges/extends [[the same coordination protocol applied to different AI models produces radically different problem-solving strategies]] — Karpathy found that 8 agents with different setups (solo vs hierarchical) produced different results
|
||||||
|
- Relevant to [[domain specialization with cross-domain synthesis produces better collective intelligence]]
|
||||||
|
- His "existing abstractions will accumulate stress" connects to the git-as-coordination-substrate thesis
|
||||||
|
|
||||||
|
**Extraction hints:**
|
||||||
|
- Claim: agent research communities outperform single-agent research because the goal is to emulate a community not an individual
|
||||||
|
- Claim: git's branch-merge model is insufficient for agent-scale collaboration because it assumes one master branch with temporary forks
|
||||||
|
- Claim: when intelligence and attention cease to be bottlenecks, existing coordination abstractions (git, PRs, branches) accumulate stress
|
||||||
|
|
||||||
|
**Context:** This is part of a series of tweets about karpathy's autoresearch project — AI agents autonomously iterating on nanochat (minimal GPT training code). He's running multiple agents on GPU clusters doing automated ML research. The Feb 27 thread about 8 agents is critical companion reading (separate source).
|
||||||
|
|
@ -6,8 +6,8 @@
|
||||||
# 2. Domain agent — domain expertise, duplicate check, technical accuracy
|
# 2. Domain agent — domain expertise, duplicate check, technical accuracy
|
||||||
#
|
#
|
||||||
# After both reviews, auto-merges if:
|
# After both reviews, auto-merges if:
|
||||||
# - Leo approved (gh pr review --approve)
|
# - Leo's comment contains "**Verdict:** approve"
|
||||||
# - Domain agent verdict is "Approve" (parsed from comment)
|
# - Domain agent's comment contains "**Verdict:** approve"
|
||||||
# - No territory violations (files outside proposer's domain)
|
# - No territory violations (files outside proposer's domain)
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
|
|
@ -26,8 +26,14 @@
|
||||||
# - Lockfile prevents concurrent runs
|
# - Lockfile prevents concurrent runs
|
||||||
# - Auto-merge requires ALL reviewers to approve + no territory violations
|
# - Auto-merge requires ALL reviewers to approve + no territory violations
|
||||||
# - Each PR runs sequentially to avoid branch conflicts
|
# - Each PR runs sequentially to avoid branch conflicts
|
||||||
# - Timeout: 10 minutes per agent per PR
|
# - Timeout: 20 minutes per agent per PR
|
||||||
# - Pre-flight checks: clean working tree, gh auth
|
# - Pre-flight checks: clean working tree, gh auth
|
||||||
|
#
|
||||||
|
# Verdict protocol:
|
||||||
|
# All agents use `gh pr comment` (NOT `gh pr review`) because all agents
|
||||||
|
# share the m3taversal GitHub account — `gh pr review --approve` fails
|
||||||
|
# when the PR author and reviewer are the same user. The merge check
|
||||||
|
# parses issue comments for structured verdict markers instead.
|
||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
|
|
@ -39,7 +45,7 @@ cd "$REPO_ROOT"
|
||||||
|
|
||||||
LOCKFILE="/tmp/evaluate-trigger.lock"
|
LOCKFILE="/tmp/evaluate-trigger.lock"
|
||||||
LOG_DIR="$REPO_ROOT/ops/sessions"
|
LOG_DIR="$REPO_ROOT/ops/sessions"
|
||||||
TIMEOUT_SECONDS=600
|
TIMEOUT_SECONDS=1200
|
||||||
DRY_RUN=false
|
DRY_RUN=false
|
||||||
LEO_ONLY=false
|
LEO_ONLY=false
|
||||||
NO_MERGE=false
|
NO_MERGE=false
|
||||||
|
|
@ -62,24 +68,30 @@ detect_domain_agent() {
|
||||||
vida/*|*/health*) agent="vida"; domain="health" ;;
|
vida/*|*/health*) agent="vida"; domain="health" ;;
|
||||||
astra/*|*/space-development*) agent="astra"; domain="space-development" ;;
|
astra/*|*/space-development*) agent="astra"; domain="space-development" ;;
|
||||||
leo/*|*/grand-strategy*) agent="leo"; domain="grand-strategy" ;;
|
leo/*|*/grand-strategy*) agent="leo"; domain="grand-strategy" ;;
|
||||||
|
contrib/*)
|
||||||
|
# External contributor — detect domain from changed files (fall through to file check)
|
||||||
|
agent=""; domain=""
|
||||||
|
;;
|
||||||
*)
|
*)
|
||||||
# Fall back to checking which domain directory has changed files
|
agent=""; domain=""
|
||||||
if echo "$files" | grep -q "domains/internet-finance/"; then
|
|
||||||
agent="rio"; domain="internet-finance"
|
|
||||||
elif echo "$files" | grep -q "domains/entertainment/"; then
|
|
||||||
agent="clay"; domain="entertainment"
|
|
||||||
elif echo "$files" | grep -q "domains/ai-alignment/"; then
|
|
||||||
agent="theseus"; domain="ai-alignment"
|
|
||||||
elif echo "$files" | grep -q "domains/health/"; then
|
|
||||||
agent="vida"; domain="health"
|
|
||||||
elif echo "$files" | grep -q "domains/space-development/"; then
|
|
||||||
agent="astra"; domain="space-development"
|
|
||||||
else
|
|
||||||
agent=""; domain=""
|
|
||||||
fi
|
|
||||||
;;
|
;;
|
||||||
esac
|
esac
|
||||||
|
|
||||||
|
# If no agent detected from branch prefix, check changed files
|
||||||
|
if [ -z "$agent" ]; then
|
||||||
|
if echo "$files" | grep -q "domains/internet-finance/"; then
|
||||||
|
agent="rio"; domain="internet-finance"
|
||||||
|
elif echo "$files" | grep -q "domains/entertainment/"; then
|
||||||
|
agent="clay"; domain="entertainment"
|
||||||
|
elif echo "$files" | grep -q "domains/ai-alignment/"; then
|
||||||
|
agent="theseus"; domain="ai-alignment"
|
||||||
|
elif echo "$files" | grep -q "domains/health/"; then
|
||||||
|
agent="vida"; domain="health"
|
||||||
|
elif echo "$files" | grep -q "domains/space-development/"; then
|
||||||
|
agent="astra"; domain="space-development"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
echo "$agent $domain"
|
echo "$agent $domain"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -112,8 +124,8 @@ if ! command -v claude >/dev/null 2>&1; then
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Check for dirty working tree (ignore ops/ and .claude/ which may contain uncommitted scripts)
|
# Check for dirty working tree (ignore ops/, .claude/, .github/ which may contain local-only files)
|
||||||
DIRTY_FILES=$(git status --porcelain | grep -v '^?? ops/' | grep -v '^ M ops/' | grep -v '^?? \.claude/' | grep -v '^ M \.claude/' || true)
|
DIRTY_FILES=$(git status --porcelain | grep -v '^?? ops/' | grep -v '^ M ops/' | grep -v '^?? \.claude/' | grep -v '^ M \.claude/' | grep -v '^?? \.github/' | grep -v '^ M \.github/' || true)
|
||||||
if [ -n "$DIRTY_FILES" ]; then
|
if [ -n "$DIRTY_FILES" ]; then
|
||||||
echo "ERROR: Working tree is dirty. Clean up before running."
|
echo "ERROR: Working tree is dirty. Clean up before running."
|
||||||
echo "$DIRTY_FILES"
|
echo "$DIRTY_FILES"
|
||||||
|
|
@ -145,7 +157,8 @@ if [ -n "$SPECIFIC_PR" ]; then
|
||||||
fi
|
fi
|
||||||
PRS_TO_REVIEW="$SPECIFIC_PR"
|
PRS_TO_REVIEW="$SPECIFIC_PR"
|
||||||
else
|
else
|
||||||
OPEN_PRS=$(gh pr list --state open --json number --jq '.[].number' 2>/dev/null || echo "")
|
# NOTE: gh pr list silently returns empty in some worktree configs; use gh api instead
|
||||||
|
OPEN_PRS=$(gh api repos/:owner/:repo/pulls --jq '.[].number' 2>/dev/null || echo "")
|
||||||
|
|
||||||
if [ -z "$OPEN_PRS" ]; then
|
if [ -z "$OPEN_PRS" ]; then
|
||||||
echo "No open PRs found. Nothing to review."
|
echo "No open PRs found. Nothing to review."
|
||||||
|
|
@ -154,17 +167,23 @@ else
|
||||||
|
|
||||||
PRS_TO_REVIEW=""
|
PRS_TO_REVIEW=""
|
||||||
for pr in $OPEN_PRS; do
|
for pr in $OPEN_PRS; do
|
||||||
LAST_REVIEW_DATE=$(gh api "repos/{owner}/{repo}/pulls/$pr/reviews" \
|
# Check if this PR already has a Leo verdict comment (avoid re-reviewing)
|
||||||
--jq 'map(select(.state != "DISMISSED")) | sort_by(.submitted_at) | last | .submitted_at' 2>/dev/null || echo "")
|
LEO_COMMENTED=$(gh pr view "$pr" --json comments \
|
||||||
|
--jq '[.comments[] | select(.body | test("VERDICT:LEO:(APPROVE|REQUEST_CHANGES)"))] | length' 2>/dev/null || echo "0")
|
||||||
LAST_COMMIT_DATE=$(gh pr view "$pr" --json commits --jq '.commits[-1].committedDate' 2>/dev/null || echo "")
|
LAST_COMMIT_DATE=$(gh pr view "$pr" --json commits --jq '.commits[-1].committedDate' 2>/dev/null || echo "")
|
||||||
|
|
||||||
if [ -z "$LAST_REVIEW_DATE" ]; then
|
if [ "$LEO_COMMENTED" = "0" ]; then
|
||||||
PRS_TO_REVIEW="$PRS_TO_REVIEW $pr"
|
|
||||||
elif [ -n "$LAST_COMMIT_DATE" ] && [[ "$LAST_COMMIT_DATE" > "$LAST_REVIEW_DATE" ]]; then
|
|
||||||
echo "PR #$pr: New commits since last review. Queuing for re-review."
|
|
||||||
PRS_TO_REVIEW="$PRS_TO_REVIEW $pr"
|
PRS_TO_REVIEW="$PRS_TO_REVIEW $pr"
|
||||||
else
|
else
|
||||||
echo "PR #$pr: No new commits since last review. Skipping."
|
# Check if new commits since last Leo review
|
||||||
|
LAST_LEO_DATE=$(gh pr view "$pr" --json comments \
|
||||||
|
--jq '[.comments[] | select(.body | test("VERDICT:LEO:")) | .createdAt] | last' 2>/dev/null || echo "")
|
||||||
|
if [ -n "$LAST_COMMIT_DATE" ] && [ -n "$LAST_LEO_DATE" ] && [[ "$LAST_COMMIT_DATE" > "$LAST_LEO_DATE" ]]; then
|
||||||
|
echo "PR #$pr: New commits since last review. Queuing for re-review."
|
||||||
|
PRS_TO_REVIEW="$PRS_TO_REVIEW $pr"
|
||||||
|
else
|
||||||
|
echo "PR #$pr: Already reviewed. Skipping."
|
||||||
|
fi
|
||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
|
|
||||||
|
|
@ -195,7 +214,7 @@ run_agent_review() {
|
||||||
log_file="$LOG_DIR/${agent_name}-review-pr${pr}-${timestamp}.log"
|
log_file="$LOG_DIR/${agent_name}-review-pr${pr}-${timestamp}.log"
|
||||||
review_file="/tmp/${agent_name}-review-pr${pr}.md"
|
review_file="/tmp/${agent_name}-review-pr${pr}.md"
|
||||||
|
|
||||||
echo " Running ${agent_name}..."
|
echo " Running ${agent_name} (model: ${model})..."
|
||||||
echo " Log: $log_file"
|
echo " Log: $log_file"
|
||||||
|
|
||||||
if perl -e "alarm $TIMEOUT_SECONDS; exec @ARGV" claude -p \
|
if perl -e "alarm $TIMEOUT_SECONDS; exec @ARGV" claude -p \
|
||||||
|
|
@ -240,6 +259,7 @@ check_territory_violations() {
|
||||||
vida) allowed_domains="domains/health/" ;;
|
vida) allowed_domains="domains/health/" ;;
|
||||||
astra) allowed_domains="domains/space-development/" ;;
|
astra) allowed_domains="domains/space-development/" ;;
|
||||||
leo) allowed_domains="core/|foundations/" ;;
|
leo) allowed_domains="core/|foundations/" ;;
|
||||||
|
contrib) echo ""; return 0 ;; # External contributors — skip territory check
|
||||||
*) echo ""; return 0 ;; # Unknown proposer — skip check
|
*) echo ""; return 0 ;; # Unknown proposer — skip check
|
||||||
esac
|
esac
|
||||||
|
|
||||||
|
|
@ -266,74 +286,51 @@ check_territory_violations() {
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Auto-merge check ---
|
# --- Auto-merge check ---
|
||||||
# Returns 0 if PR should be merged, 1 if not
|
# Parses issue comments for structured verdict markers.
|
||||||
|
# Verdict protocol: agents post `<!-- VERDICT:AGENT_KEY:APPROVE -->` or
|
||||||
|
# `<!-- VERDICT:AGENT_KEY:REQUEST_CHANGES -->` as HTML comments in their review.
|
||||||
|
# This is machine-parseable and invisible in the rendered comment.
|
||||||
check_merge_eligible() {
|
check_merge_eligible() {
|
||||||
local pr_number="$1"
|
local pr_number="$1"
|
||||||
local domain_agent="$2"
|
local domain_agent="$2"
|
||||||
local leo_passed="$3"
|
local leo_passed="$3"
|
||||||
|
|
||||||
# Gate 1: Leo must have passed
|
# Gate 1: Leo must have completed without timeout/error
|
||||||
if [ "$leo_passed" != "true" ]; then
|
if [ "$leo_passed" != "true" ]; then
|
||||||
echo "BLOCK: Leo review failed or timed out"
|
echo "BLOCK: Leo review failed or timed out"
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Gate 2: Check Leo's review state via GitHub API
|
# Gate 2: Check Leo's verdict from issue comments
|
||||||
local leo_review_state
|
local leo_verdict
|
||||||
leo_review_state=$(gh api "repos/{owner}/{repo}/pulls/${pr_number}/reviews" \
|
leo_verdict=$(gh pr view "$pr_number" --json comments \
|
||||||
--jq '[.[] | select(.state != "DISMISSED" and .state != "PENDING")] | last | .state' 2>/dev/null || echo "")
|
--jq '[.comments[] | select(.body | test("VERDICT:LEO:")) | .body] | last' 2>/dev/null || echo "")
|
||||||
|
|
||||||
if [ "$leo_review_state" = "APPROVED" ]; then
|
if echo "$leo_verdict" | grep -q "VERDICT:LEO:APPROVE"; then
|
||||||
echo "Leo: APPROVED (via review API)"
|
echo "Leo: APPROVED"
|
||||||
elif [ "$leo_review_state" = "CHANGES_REQUESTED" ]; then
|
elif echo "$leo_verdict" | grep -q "VERDICT:LEO:REQUEST_CHANGES"; then
|
||||||
echo "BLOCK: Leo requested changes (review API state: CHANGES_REQUESTED)"
|
echo "BLOCK: Leo requested changes"
|
||||||
return 1
|
return 1
|
||||||
else
|
else
|
||||||
# Fallback: check PR comments for Leo's verdict
|
echo "BLOCK: Could not find Leo's verdict marker in PR comments"
|
||||||
local leo_verdict
|
return 1
|
||||||
leo_verdict=$(gh pr view "$pr_number" --json comments \
|
|
||||||
--jq '.comments[] | select(.body | test("## Leo Review")) | .body' 2>/dev/null \
|
|
||||||
| grep -oiE '\*\*Verdict:[^*]+\*\*' | tail -1 || echo "")
|
|
||||||
|
|
||||||
if echo "$leo_verdict" | grep -qi "approve"; then
|
|
||||||
echo "Leo: APPROVED (via comment verdict)"
|
|
||||||
elif echo "$leo_verdict" | grep -qi "request changes\|reject"; then
|
|
||||||
echo "BLOCK: Leo verdict: $leo_verdict"
|
|
||||||
return 1
|
|
||||||
else
|
|
||||||
echo "BLOCK: Could not determine Leo's verdict"
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Gate 3: Check domain agent verdict (if applicable)
|
# Gate 3: Check domain agent verdict (if applicable)
|
||||||
if [ -n "$domain_agent" ] && [ "$domain_agent" != "leo" ]; then
|
if [ -n "$domain_agent" ] && [ "$domain_agent" != "leo" ]; then
|
||||||
|
local domain_key
|
||||||
|
domain_key=$(echo "$domain_agent" | tr '[:lower:]' '[:upper:]')
|
||||||
local domain_verdict
|
local domain_verdict
|
||||||
# Search for verdict in domain agent's review — match agent name, "domain reviewer", or "Domain Review"
|
|
||||||
domain_verdict=$(gh pr view "$pr_number" --json comments \
|
domain_verdict=$(gh pr view "$pr_number" --json comments \
|
||||||
--jq ".comments[] | select(.body | test(\"domain review|${domain_agent}|peer review\"; \"i\")) | .body" 2>/dev/null \
|
--jq "[.comments[] | select(.body | test(\"VERDICT:${domain_key}:\")) | .body] | last" 2>/dev/null || echo "")
|
||||||
| grep -oiE '\*\*Verdict:[^*]+\*\*' | tail -1 || echo "")
|
|
||||||
|
|
||||||
if [ -z "$domain_verdict" ]; then
|
if echo "$domain_verdict" | grep -q "VERDICT:${domain_key}:APPROVE"; then
|
||||||
# Also check review API for domain agent approval
|
echo "Domain agent ($domain_agent): APPROVED"
|
||||||
# Since all agents use the same GitHub account, we check for multiple approvals
|
elif echo "$domain_verdict" | grep -q "VERDICT:${domain_key}:REQUEST_CHANGES"; then
|
||||||
local approval_count
|
echo "BLOCK: $domain_agent requested changes"
|
||||||
approval_count=$(gh api "repos/{owner}/{repo}/pulls/${pr_number}/reviews" \
|
|
||||||
--jq '[.[] | select(.state == "APPROVED")] | length' 2>/dev/null || echo "0")
|
|
||||||
|
|
||||||
if [ "$approval_count" -ge 2 ]; then
|
|
||||||
echo "Domain agent: APPROVED (multiple approvals via review API)"
|
|
||||||
else
|
|
||||||
echo "BLOCK: No domain agent verdict found"
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
elif echo "$domain_verdict" | grep -qi "approve"; then
|
|
||||||
echo "Domain agent ($domain_agent): APPROVED (via comment verdict)"
|
|
||||||
elif echo "$domain_verdict" | grep -qi "request changes\|reject"; then
|
|
||||||
echo "BLOCK: Domain agent verdict: $domain_verdict"
|
|
||||||
return 1
|
return 1
|
||||||
else
|
else
|
||||||
echo "BLOCK: Unclear domain agent verdict: $domain_verdict"
|
echo "BLOCK: No verdict marker found for $domain_agent"
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
else
|
else
|
||||||
|
|
@ -403,11 +400,15 @@ Also check:
|
||||||
- Cross-domain connections that the proposer may have missed
|
- Cross-domain connections that the proposer may have missed
|
||||||
|
|
||||||
Write your complete review to ${LEO_REVIEW_FILE}
|
Write your complete review to ${LEO_REVIEW_FILE}
|
||||||
Then post it with: gh pr review ${pr} --comment --body-file ${LEO_REVIEW_FILE}
|
|
||||||
|
|
||||||
If ALL claims pass quality gates: gh pr review ${pr} --approve --body-file ${LEO_REVIEW_FILE}
|
CRITICAL — Verdict format: Your review MUST end with exactly one of these verdict markers (as an HTML comment on its own line):
|
||||||
If ANY claim needs changes: gh pr review ${pr} --request-changes --body-file ${LEO_REVIEW_FILE}
|
<!-- VERDICT:LEO:APPROVE -->
|
||||||
|
<!-- VERDICT:LEO:REQUEST_CHANGES -->
|
||||||
|
|
||||||
|
Then post the review as an issue comment:
|
||||||
|
gh pr comment ${pr} --body-file ${LEO_REVIEW_FILE}
|
||||||
|
|
||||||
|
IMPORTANT: Use 'gh pr comment' NOT 'gh pr review'. We use a shared GitHub account so gh pr review --approve fails.
|
||||||
DO NOT merge — the orchestrator handles merge decisions after all reviews are posted.
|
DO NOT merge — the orchestrator handles merge decisions after all reviews are posted.
|
||||||
Work autonomously. Do not ask for confirmation."
|
Work autonomously. Do not ask for confirmation."
|
||||||
|
|
||||||
|
|
@ -432,6 +433,7 @@ Work autonomously. Do not ask for confirmation."
|
||||||
else
|
else
|
||||||
DOMAIN_REVIEW_FILE="/tmp/${DOMAIN_AGENT}-review-pr${pr}.md"
|
DOMAIN_REVIEW_FILE="/tmp/${DOMAIN_AGENT}-review-pr${pr}.md"
|
||||||
AGENT_NAME_UPPER=$(echo "${DOMAIN_AGENT}" | awk '{print toupper(substr($0,1,1)) substr($0,2)}')
|
AGENT_NAME_UPPER=$(echo "${DOMAIN_AGENT}" | awk '{print toupper(substr($0,1,1)) substr($0,2)}')
|
||||||
|
AGENT_KEY_UPPER=$(echo "${DOMAIN_AGENT}" | tr '[:lower:]' '[:upper:]')
|
||||||
DOMAIN_PROMPT="You are ${AGENT_NAME_UPPER}. Read agents/${DOMAIN_AGENT}/identity.md, agents/${DOMAIN_AGENT}/beliefs.md, and skills/evaluate.md.
|
DOMAIN_PROMPT="You are ${AGENT_NAME_UPPER}. Read agents/${DOMAIN_AGENT}/identity.md, agents/${DOMAIN_AGENT}/beliefs.md, and skills/evaluate.md.
|
||||||
|
|
||||||
You are reviewing PR #${pr} as the domain expert for ${DOMAIN}.
|
You are reviewing PR #${pr} as the domain expert for ${DOMAIN}.
|
||||||
|
|
@ -452,8 +454,15 @@ Your review focuses on DOMAIN EXPERTISE — things only a ${DOMAIN} specialist w
|
||||||
6. **Confidence calibration** — From your domain expertise, is the confidence level right?
|
6. **Confidence calibration** — From your domain expertise, is the confidence level right?
|
||||||
|
|
||||||
Write your review to ${DOMAIN_REVIEW_FILE}
|
Write your review to ${DOMAIN_REVIEW_FILE}
|
||||||
Post it with: gh pr review ${pr} --comment --body-file ${DOMAIN_REVIEW_FILE}
|
|
||||||
|
|
||||||
|
CRITICAL — Verdict format: Your review MUST end with exactly one of these verdict markers (as an HTML comment on its own line):
|
||||||
|
<!-- VERDICT:${AGENT_KEY_UPPER}:APPROVE -->
|
||||||
|
<!-- VERDICT:${AGENT_KEY_UPPER}:REQUEST_CHANGES -->
|
||||||
|
|
||||||
|
Then post the review as an issue comment:
|
||||||
|
gh pr comment ${pr} --body-file ${DOMAIN_REVIEW_FILE}
|
||||||
|
|
||||||
|
IMPORTANT: Use 'gh pr comment' NOT 'gh pr review'. We use a shared GitHub account so gh pr review --approve fails.
|
||||||
Sign your review as ${AGENT_NAME_UPPER} (domain reviewer for ${DOMAIN}).
|
Sign your review as ${AGENT_NAME_UPPER} (domain reviewer for ${DOMAIN}).
|
||||||
DO NOT duplicate Leo's quality gate checks — he covers those.
|
DO NOT duplicate Leo's quality gate checks — he covers those.
|
||||||
DO NOT merge — the orchestrator handles merge decisions after all reviews are posted.
|
DO NOT merge — the orchestrator handles merge decisions after all reviews are posted.
|
||||||
|
|
@ -486,7 +495,7 @@ Work autonomously. Do not ask for confirmation."
|
||||||
|
|
||||||
if [ "$MERGE_RESULT" -eq 0 ]; then
|
if [ "$MERGE_RESULT" -eq 0 ]; then
|
||||||
echo " Auto-merge: ALL GATES PASSED — merging PR #$pr"
|
echo " Auto-merge: ALL GATES PASSED — merging PR #$pr"
|
||||||
if gh pr merge "$pr" --squash --delete-branch 2>&1; then
|
if gh pr merge "$pr" --squash 2>&1; then
|
||||||
echo " PR #$pr: MERGED successfully."
|
echo " PR #$pr: MERGED successfully."
|
||||||
MERGED=$((MERGED + 1))
|
MERGED=$((MERGED + 1))
|
||||||
else
|
else
|
||||||
|
|
|
||||||
520
ops/extract-graph-data.py
Normal file
520
ops/extract-graph-data.py
Normal file
|
|
@ -0,0 +1,520 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
extract-graph-data.py — Extract knowledge graph from teleo-codex markdown files.
|
||||||
|
|
||||||
|
Reads all .md claim/conviction files, parses YAML frontmatter and wiki-links,
|
||||||
|
and outputs graph-data.json matching the teleo-app GraphData interface.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python3 ops/extract-graph-data.py [--output path/to/graph-data.json]
|
||||||
|
|
||||||
|
Must be run from the teleo-codex repo root.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Config
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
SCAN_DIRS = ["core", "domains", "foundations", "convictions"]
|
||||||
|
|
||||||
|
# Only extract these content types (from frontmatter `type` field).
|
||||||
|
# If type is missing, include the file anyway (many claims lack explicit type).
|
||||||
|
INCLUDE_TYPES = {"claim", "conviction", "analysis", "belief", "position", None}
|
||||||
|
|
||||||
|
# Domain → default agent mapping (fallback when git attribution unavailable)
|
||||||
|
DOMAIN_AGENT_MAP = {
|
||||||
|
"internet-finance": "rio",
|
||||||
|
"entertainment": "clay",
|
||||||
|
"health": "vida",
|
||||||
|
"ai-alignment": "theseus",
|
||||||
|
"space-development": "astra",
|
||||||
|
"grand-strategy": "leo",
|
||||||
|
"mechanisms": "leo",
|
||||||
|
"living-capital": "leo",
|
||||||
|
"living-agents": "leo",
|
||||||
|
"teleohumanity": "leo",
|
||||||
|
"critical-systems": "leo",
|
||||||
|
"collective-intelligence": "leo",
|
||||||
|
"teleological-economics": "leo",
|
||||||
|
"cultural-dynamics": "clay",
|
||||||
|
}
|
||||||
|
|
||||||
|
DOMAIN_COLORS = {
|
||||||
|
"internet-finance": "#4A90D9",
|
||||||
|
"entertainment": "#9B59B6",
|
||||||
|
"health": "#2ECC71",
|
||||||
|
"ai-alignment": "#E74C3C",
|
||||||
|
"space-development": "#F39C12",
|
||||||
|
"grand-strategy": "#D4AF37",
|
||||||
|
"mechanisms": "#1ABC9C",
|
||||||
|
"living-capital": "#3498DB",
|
||||||
|
"living-agents": "#E67E22",
|
||||||
|
"teleohumanity": "#F1C40F",
|
||||||
|
"critical-systems": "#95A5A6",
|
||||||
|
"collective-intelligence": "#BDC3C7",
|
||||||
|
"teleological-economics": "#7F8C8D",
|
||||||
|
"cultural-dynamics": "#C0392B",
|
||||||
|
}
|
||||||
|
|
||||||
|
KNOWN_AGENTS = {"leo", "rio", "clay", "vida", "theseus", "astra"}
|
||||||
|
|
||||||
|
# Regex patterns
|
||||||
|
FRONTMATTER_RE = re.compile(r"^---\s*\n(.*?)\n---", re.DOTALL)
|
||||||
|
WIKILINK_RE = re.compile(r"\[\[([^\]]+)\]\]")
|
||||||
|
YAML_FIELD_RE = re.compile(r"^(\w[\w_]*):\s*(.+)$", re.MULTILINE)
|
||||||
|
YAML_LIST_ITEM_RE = re.compile(r'^\s*-\s+"?(.+?)"?\s*$', re.MULTILINE)
|
||||||
|
COUNTER_EVIDENCE_RE = re.compile(r"^##\s+Counter[\s-]?evidence", re.MULTILINE | re.IGNORECASE)
|
||||||
|
COUNTERARGUMENT_RE = re.compile(r"^\*\*Counter\s*argument", re.MULTILINE | re.IGNORECASE)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Lightweight YAML-ish frontmatter parser (avoids PyYAML dependency)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def parse_frontmatter(text: str) -> dict:
|
||||||
|
"""Parse YAML frontmatter from markdown text. Returns dict of fields."""
|
||||||
|
m = FRONTMATTER_RE.match(text)
|
||||||
|
if not m:
|
||||||
|
return {}
|
||||||
|
yaml_block = m.group(1)
|
||||||
|
result = {}
|
||||||
|
for field_match in YAML_FIELD_RE.finditer(yaml_block):
|
||||||
|
key = field_match.group(1)
|
||||||
|
val = field_match.group(2).strip().strip('"').strip("'")
|
||||||
|
# Handle list fields
|
||||||
|
if val.startswith("["):
|
||||||
|
# Inline YAML list: [item1, item2]
|
||||||
|
items = re.findall(r'"([^"]+)"', val)
|
||||||
|
if not items:
|
||||||
|
items = [x.strip().strip('"').strip("'")
|
||||||
|
for x in val.strip("[]").split(",") if x.strip()]
|
||||||
|
result[key] = items
|
||||||
|
else:
|
||||||
|
result[key] = val
|
||||||
|
# Handle multi-line list fields (depends_on, challenged_by, secondary_domains)
|
||||||
|
for list_key in ("depends_on", "challenged_by", "secondary_domains", "claims_extracted"):
|
||||||
|
if list_key not in result:
|
||||||
|
# Check for block-style list
|
||||||
|
pattern = re.compile(
|
||||||
|
rf"^{list_key}:\s*\n((?:\s+-\s+.+\n?)+)", re.MULTILINE
|
||||||
|
)
|
||||||
|
lm = pattern.search(yaml_block)
|
||||||
|
if lm:
|
||||||
|
items = YAML_LIST_ITEM_RE.findall(lm.group(1))
|
||||||
|
result[list_key] = [i.strip('"').strip("'") for i in items]
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def extract_body(text: str) -> str:
|
||||||
|
"""Return the markdown body after frontmatter."""
|
||||||
|
m = FRONTMATTER_RE.match(text)
|
||||||
|
if m:
|
||||||
|
return text[m.end():]
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Git-based agent attribution
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def build_git_agent_map(repo_root: str) -> dict[str, str]:
|
||||||
|
"""Map file paths → agent name using git log commit message prefixes.
|
||||||
|
|
||||||
|
Commit messages follow: '{agent}: description'
|
||||||
|
We use the commit that first added each file.
|
||||||
|
"""
|
||||||
|
file_agent = {}
|
||||||
|
try:
|
||||||
|
result = subprocess.run(
|
||||||
|
["git", "log", "--all", "--diff-filter=A", "--name-only",
|
||||||
|
"--format=COMMIT_MSG:%s"],
|
||||||
|
capture_output=True, text=True, cwd=repo_root, timeout=30,
|
||||||
|
)
|
||||||
|
current_agent = None
|
||||||
|
for line in result.stdout.splitlines():
|
||||||
|
line = line.strip()
|
||||||
|
if not line:
|
||||||
|
continue
|
||||||
|
if line.startswith("COMMIT_MSG:"):
|
||||||
|
msg = line[len("COMMIT_MSG:"):]
|
||||||
|
# Parse "agent: description" pattern
|
||||||
|
if ":" in msg:
|
||||||
|
prefix = msg.split(":")[0].strip().lower()
|
||||||
|
if prefix in KNOWN_AGENTS:
|
||||||
|
current_agent = prefix
|
||||||
|
else:
|
||||||
|
current_agent = None
|
||||||
|
else:
|
||||||
|
current_agent = None
|
||||||
|
elif current_agent and line.endswith(".md"):
|
||||||
|
# Only set if not already attributed (first add wins)
|
||||||
|
if line not in file_agent:
|
||||||
|
file_agent[line] = current_agent
|
||||||
|
except (subprocess.TimeoutExpired, FileNotFoundError):
|
||||||
|
pass
|
||||||
|
return file_agent
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Wiki-link resolution
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def build_title_index(all_files: list[str], repo_root: str) -> dict[str, str]:
|
||||||
|
"""Map lowercase claim titles → file paths for wiki-link resolution."""
|
||||||
|
index = {}
|
||||||
|
for fpath in all_files:
|
||||||
|
# Title = filename without .md extension
|
||||||
|
fname = os.path.basename(fpath)
|
||||||
|
if fname.endswith(".md"):
|
||||||
|
title = fname[:-3].lower()
|
||||||
|
index[title] = fpath
|
||||||
|
# Also index by relative path
|
||||||
|
index[fpath.lower()] = fpath
|
||||||
|
return index
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_wikilink(link_text: str, title_index: dict, source_dir: str) -> str | None:
|
||||||
|
"""Resolve a [[wiki-link]] target to a file path (node ID)."""
|
||||||
|
text = link_text.strip()
|
||||||
|
# Skip map links and non-claim references
|
||||||
|
if text.startswith("_") or text == "_map":
|
||||||
|
return None
|
||||||
|
# Direct path match (with or without .md)
|
||||||
|
for candidate in [text, text + ".md"]:
|
||||||
|
if candidate.lower() in title_index:
|
||||||
|
return title_index[candidate.lower()]
|
||||||
|
# Title-only match
|
||||||
|
title = text.lower()
|
||||||
|
if title in title_index:
|
||||||
|
return title_index[title]
|
||||||
|
# Fuzzy: try adding .md to the basename
|
||||||
|
basename = os.path.basename(text)
|
||||||
|
if basename.lower() in title_index:
|
||||||
|
return title_index[basename.lower()]
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# PR/merge event extraction from git log
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def extract_events(repo_root: str) -> list[dict]:
|
||||||
|
"""Extract PR merge events from git log for the events timeline."""
|
||||||
|
events = []
|
||||||
|
try:
|
||||||
|
result = subprocess.run(
|
||||||
|
["git", "log", "--merges", "--format=%H|%s|%ai", "-50"],
|
||||||
|
capture_output=True, text=True, cwd=repo_root, timeout=15,
|
||||||
|
)
|
||||||
|
for line in result.stdout.strip().splitlines():
|
||||||
|
parts = line.split("|", 2)
|
||||||
|
if len(parts) < 3:
|
||||||
|
continue
|
||||||
|
sha, msg, date_str = parts
|
||||||
|
# Parse "Merge pull request #N from ..." or agent commit patterns
|
||||||
|
pr_match = re.search(r"#(\d+)", msg)
|
||||||
|
if not pr_match:
|
||||||
|
continue
|
||||||
|
pr_num = int(pr_match.group(1))
|
||||||
|
# Try to determine agent from merge commit
|
||||||
|
agent = "collective"
|
||||||
|
for a in KNOWN_AGENTS:
|
||||||
|
if a in msg.lower():
|
||||||
|
agent = a
|
||||||
|
break
|
||||||
|
# Count files changed in this merge
|
||||||
|
diff_result = subprocess.run(
|
||||||
|
["git", "diff", "--name-only", f"{sha}^..{sha}"],
|
||||||
|
capture_output=True, text=True, cwd=repo_root, timeout=10,
|
||||||
|
)
|
||||||
|
claims_added = sum(
|
||||||
|
1 for f in diff_result.stdout.splitlines()
|
||||||
|
if f.endswith(".md") and any(f.startswith(d) for d in SCAN_DIRS)
|
||||||
|
)
|
||||||
|
if claims_added > 0:
|
||||||
|
events.append({
|
||||||
|
"type": "pr-merge",
|
||||||
|
"number": pr_num,
|
||||||
|
"agent": agent,
|
||||||
|
"claims_added": claims_added,
|
||||||
|
"date": date_str[:10],
|
||||||
|
})
|
||||||
|
except (subprocess.TimeoutExpired, FileNotFoundError):
|
||||||
|
pass
|
||||||
|
return events
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Main extraction
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def find_markdown_files(repo_root: str) -> list[str]:
|
||||||
|
"""Find all .md files in SCAN_DIRS, return relative paths."""
|
||||||
|
files = []
|
||||||
|
for scan_dir in SCAN_DIRS:
|
||||||
|
dirpath = os.path.join(repo_root, scan_dir)
|
||||||
|
if not os.path.isdir(dirpath):
|
||||||
|
continue
|
||||||
|
for root, _dirs, filenames in os.walk(dirpath):
|
||||||
|
for fname in filenames:
|
||||||
|
if fname.endswith(".md") and not fname.startswith("_"):
|
||||||
|
rel = os.path.relpath(os.path.join(root, fname), repo_root)
|
||||||
|
files.append(rel)
|
||||||
|
return sorted(files)
|
||||||
|
|
||||||
|
|
||||||
|
def _get_domain_cached(fpath: str, repo_root: str, cache: dict) -> str:
|
||||||
|
"""Get the domain of a file, caching results."""
|
||||||
|
if fpath in cache:
|
||||||
|
return cache[fpath]
|
||||||
|
abs_path = os.path.join(repo_root, fpath)
|
||||||
|
domain = ""
|
||||||
|
try:
|
||||||
|
text = open(abs_path, encoding="utf-8").read()
|
||||||
|
fm = parse_frontmatter(text)
|
||||||
|
domain = fm.get("domain", "")
|
||||||
|
except (OSError, UnicodeDecodeError):
|
||||||
|
pass
|
||||||
|
cache[fpath] = domain
|
||||||
|
return domain
|
||||||
|
|
||||||
|
|
||||||
|
def extract_graph(repo_root: str) -> dict:
|
||||||
|
"""Extract the full knowledge graph from the codex."""
|
||||||
|
all_files = find_markdown_files(repo_root)
|
||||||
|
git_agents = build_git_agent_map(repo_root)
|
||||||
|
title_index = build_title_index(all_files, repo_root)
|
||||||
|
domain_cache: dict[str, str] = {}
|
||||||
|
|
||||||
|
nodes = []
|
||||||
|
edges = []
|
||||||
|
node_ids = set()
|
||||||
|
all_files_set = set(all_files)
|
||||||
|
|
||||||
|
for fpath in all_files:
|
||||||
|
abs_path = os.path.join(repo_root, fpath)
|
||||||
|
try:
|
||||||
|
text = open(abs_path, encoding="utf-8").read()
|
||||||
|
except (OSError, UnicodeDecodeError):
|
||||||
|
continue
|
||||||
|
|
||||||
|
fm = parse_frontmatter(text)
|
||||||
|
body = extract_body(text)
|
||||||
|
|
||||||
|
# Filter by type
|
||||||
|
ftype = fm.get("type")
|
||||||
|
if ftype and ftype not in INCLUDE_TYPES:
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Build node
|
||||||
|
title = os.path.basename(fpath)[:-3] # filename without .md
|
||||||
|
domain = fm.get("domain", "")
|
||||||
|
if not domain:
|
||||||
|
# Infer domain from directory path
|
||||||
|
parts = fpath.split(os.sep)
|
||||||
|
if len(parts) >= 2:
|
||||||
|
domain = parts[1] if parts[0] == "domains" else parts[1] if len(parts) > 2 else parts[0]
|
||||||
|
|
||||||
|
# Agent attribution: git log → domain mapping → "collective"
|
||||||
|
agent = git_agents.get(fpath, "")
|
||||||
|
if not agent:
|
||||||
|
agent = DOMAIN_AGENT_MAP.get(domain, "collective")
|
||||||
|
|
||||||
|
created = fm.get("created", "")
|
||||||
|
confidence = fm.get("confidence", "speculative")
|
||||||
|
|
||||||
|
# Detect challenged status
|
||||||
|
challenged_by_raw = fm.get("challenged_by", [])
|
||||||
|
if isinstance(challenged_by_raw, str):
|
||||||
|
challenged_by_raw = [challenged_by_raw] if challenged_by_raw else []
|
||||||
|
has_challenged_by = bool(challenged_by_raw and any(c for c in challenged_by_raw))
|
||||||
|
has_counter_section = bool(COUNTER_EVIDENCE_RE.search(body) or COUNTERARGUMENT_RE.search(body))
|
||||||
|
is_challenged = has_challenged_by or has_counter_section
|
||||||
|
|
||||||
|
# Extract challenge descriptions for the node
|
||||||
|
challenges = []
|
||||||
|
if isinstance(challenged_by_raw, list):
|
||||||
|
for c in challenged_by_raw:
|
||||||
|
if c and isinstance(c, str):
|
||||||
|
# Strip wiki-link syntax for display
|
||||||
|
cleaned = WIKILINK_RE.sub(lambda m: m.group(1), c)
|
||||||
|
# Strip markdown list artifacts: leading "- ", surrounding quotes
|
||||||
|
cleaned = re.sub(r'^-\s*', '', cleaned).strip()
|
||||||
|
cleaned = cleaned.strip('"').strip("'").strip()
|
||||||
|
if cleaned:
|
||||||
|
challenges.append(cleaned[:200]) # cap length
|
||||||
|
|
||||||
|
node = {
|
||||||
|
"id": fpath,
|
||||||
|
"title": title,
|
||||||
|
"domain": domain,
|
||||||
|
"agent": agent,
|
||||||
|
"created": created,
|
||||||
|
"confidence": confidence,
|
||||||
|
"challenged": is_challenged,
|
||||||
|
}
|
||||||
|
if challenges:
|
||||||
|
node["challenges"] = challenges
|
||||||
|
nodes.append(node)
|
||||||
|
node_ids.add(fpath)
|
||||||
|
domain_cache[fpath] = domain # cache for edge lookups
|
||||||
|
for link_text in WIKILINK_RE.findall(body):
|
||||||
|
target = resolve_wikilink(link_text, title_index, os.path.dirname(fpath))
|
||||||
|
if target and target != fpath and target in all_files_set:
|
||||||
|
target_domain = _get_domain_cached(target, repo_root, domain_cache)
|
||||||
|
edges.append({
|
||||||
|
"source": fpath,
|
||||||
|
"target": target,
|
||||||
|
"type": "wiki-link",
|
||||||
|
"cross_domain": domain != target_domain and bool(target_domain),
|
||||||
|
})
|
||||||
|
|
||||||
|
# Conflict edges from challenged_by (may contain [[wiki-links]] or prose)
|
||||||
|
challenged_by = fm.get("challenged_by", [])
|
||||||
|
if isinstance(challenged_by, str):
|
||||||
|
challenged_by = [challenged_by]
|
||||||
|
if isinstance(challenged_by, list):
|
||||||
|
for challenge in challenged_by:
|
||||||
|
if not challenge:
|
||||||
|
continue
|
||||||
|
# Check for embedded wiki-links
|
||||||
|
for link_text in WIKILINK_RE.findall(challenge):
|
||||||
|
target = resolve_wikilink(link_text, title_index, os.path.dirname(fpath))
|
||||||
|
if target and target != fpath and target in all_files_set:
|
||||||
|
target_domain = _get_domain_cached(target, repo_root, domain_cache)
|
||||||
|
edges.append({
|
||||||
|
"source": fpath,
|
||||||
|
"target": target,
|
||||||
|
"type": "conflict",
|
||||||
|
"cross_domain": domain != target_domain and bool(target_domain),
|
||||||
|
})
|
||||||
|
|
||||||
|
# Deduplicate edges
|
||||||
|
seen_edges = set()
|
||||||
|
unique_edges = []
|
||||||
|
for e in edges:
|
||||||
|
key = (e["source"], e["target"], e.get("type", ""))
|
||||||
|
if key not in seen_edges:
|
||||||
|
seen_edges.add(key)
|
||||||
|
unique_edges.append(e)
|
||||||
|
|
||||||
|
# Only keep edges where both endpoints exist as nodes
|
||||||
|
edges_filtered = [
|
||||||
|
e for e in unique_edges
|
||||||
|
if e["source"] in node_ids and e["target"] in node_ids
|
||||||
|
]
|
||||||
|
|
||||||
|
events = extract_events(repo_root)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"nodes": nodes,
|
||||||
|
"edges": edges_filtered,
|
||||||
|
"events": sorted(events, key=lambda e: e.get("date", "")),
|
||||||
|
"domain_colors": DOMAIN_COLORS,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def build_claims_context(repo_root: str, nodes: list[dict]) -> dict:
|
||||||
|
"""Build claims-context.json for chat system prompt injection.
|
||||||
|
|
||||||
|
Produces a lightweight claim index: title + description + domain + agent + confidence.
|
||||||
|
Sorted by domain, then alphabetically within domain.
|
||||||
|
Target: ~37KB for ~370 claims. Truncates descriptions at 100 chars if total > 100KB.
|
||||||
|
"""
|
||||||
|
claims = []
|
||||||
|
for node in nodes:
|
||||||
|
fpath = node["id"]
|
||||||
|
abs_path = os.path.join(repo_root, fpath)
|
||||||
|
description = ""
|
||||||
|
try:
|
||||||
|
text = open(abs_path, encoding="utf-8").read()
|
||||||
|
fm = parse_frontmatter(text)
|
||||||
|
description = fm.get("description", "")
|
||||||
|
except (OSError, UnicodeDecodeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
claims.append({
|
||||||
|
"title": node["title"],
|
||||||
|
"description": description,
|
||||||
|
"domain": node["domain"],
|
||||||
|
"agent": node["agent"],
|
||||||
|
"confidence": node["confidence"],
|
||||||
|
})
|
||||||
|
|
||||||
|
# Sort by domain, then title
|
||||||
|
claims.sort(key=lambda c: (c["domain"], c["title"]))
|
||||||
|
|
||||||
|
context = {
|
||||||
|
"generated": datetime.now(tz=timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||||
|
"claimCount": len(claims),
|
||||||
|
"claims": claims,
|
||||||
|
}
|
||||||
|
|
||||||
|
# Progressive description truncation if over 100KB.
|
||||||
|
# Never drop descriptions entirely — short descriptions are better than none.
|
||||||
|
for max_desc in (120, 100, 80, 60):
|
||||||
|
test_json = json.dumps(context, ensure_ascii=False)
|
||||||
|
if len(test_json) <= 100_000:
|
||||||
|
break
|
||||||
|
for c in claims:
|
||||||
|
if len(c["description"]) > max_desc:
|
||||||
|
c["description"] = c["description"][:max_desc] + "..."
|
||||||
|
|
||||||
|
return context
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description="Extract graph data from teleo-codex")
|
||||||
|
parser.add_argument("--output", "-o", default="graph-data.json",
|
||||||
|
help="Output file path (default: graph-data.json)")
|
||||||
|
parser.add_argument("--context-output", "-c", default=None,
|
||||||
|
help="Output claims-context.json path (default: same dir as --output)")
|
||||||
|
parser.add_argument("--repo", "-r", default=".",
|
||||||
|
help="Path to teleo-codex repo root (default: current dir)")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
repo_root = os.path.abspath(args.repo)
|
||||||
|
if not os.path.isdir(os.path.join(repo_root, "core")):
|
||||||
|
print(f"Error: {repo_root} doesn't look like a teleo-codex repo (no core/ dir)", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
print(f"Scanning {repo_root}...")
|
||||||
|
graph = extract_graph(repo_root)
|
||||||
|
|
||||||
|
print(f" Nodes: {len(graph['nodes'])}")
|
||||||
|
print(f" Edges: {len(graph['edges'])}")
|
||||||
|
print(f" Events: {len(graph['events'])}")
|
||||||
|
challenged_count = sum(1 for n in graph["nodes"] if n.get("challenged"))
|
||||||
|
print(f" Challenged: {challenged_count}")
|
||||||
|
|
||||||
|
# Write graph-data.json
|
||||||
|
output_path = os.path.abspath(args.output)
|
||||||
|
with open(output_path, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(graph, f, indent=2, ensure_ascii=False)
|
||||||
|
size_kb = os.path.getsize(output_path) / 1024
|
||||||
|
print(f" graph-data.json: {output_path} ({size_kb:.1f} KB)")
|
||||||
|
|
||||||
|
# Write claims-context.json
|
||||||
|
context_path = args.context_output
|
||||||
|
if not context_path:
|
||||||
|
context_path = os.path.join(os.path.dirname(output_path), "claims-context.json")
|
||||||
|
context_path = os.path.abspath(context_path)
|
||||||
|
|
||||||
|
context = build_claims_context(repo_root, graph["nodes"])
|
||||||
|
with open(context_path, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(context, f, indent=2, ensure_ascii=False)
|
||||||
|
ctx_kb = os.path.getsize(context_path) / 1024
|
||||||
|
print(f" claims-context.json: {context_path} ({ctx_kb:.1f} KB)")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
192
skills/ingest.md
Normal file
192
skills/ingest.md
Normal file
|
|
@ -0,0 +1,192 @@
|
||||||
|
# Skill: Ingest
|
||||||
|
|
||||||
|
Pull tweets from your domain network, triage for signal, archive sources, extract claims, and open a PR. This is the full ingestion loop — from raw X data to knowledge base contribution.
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
```
|
||||||
|
/ingest # Run full loop: pull → triage → archive → extract → PR
|
||||||
|
/ingest pull-only # Just pull fresh tweets, don't extract yet
|
||||||
|
/ingest from-cache # Skip pulling, extract from already-cached tweets
|
||||||
|
/ingest @username # Ingest a specific account (pull + extract)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- API key at `~/.pentagon/secrets/twitterapi-io-key`
|
||||||
|
- Your network file at `~/.pentagon/workspace/collective/x-ingestion/{your-name}-network.json`
|
||||||
|
- Forgejo token at `~/.pentagon/secrets/forgejo-{your-name}-token`
|
||||||
|
|
||||||
|
## The Loop
|
||||||
|
|
||||||
|
### Step 1: Pull fresh tweets
|
||||||
|
|
||||||
|
For each account in your network file (or the specified account):
|
||||||
|
|
||||||
|
1. **Check cache** — read `~/.pentagon/workspace/collective/x-ingestion/raw/{username}.json`. If `pulled_at` is <24h old, skip.
|
||||||
|
2. **Pull** — use `/x-research pull @{username}` or the API directly:
|
||||||
|
```bash
|
||||||
|
API_KEY=$(cat ~/.pentagon/secrets/twitterapi-io-key)
|
||||||
|
curl -s -H "X-API-Key: $API_KEY" \
|
||||||
|
"https://api.twitterapi.io/twitter/user/last_tweets?userName={username}&count=100"
|
||||||
|
```
|
||||||
|
3. **Save** to `~/.pentagon/workspace/collective/x-ingestion/raw/{username}.json`
|
||||||
|
4. **Log** the pull to `~/.pentagon/workspace/collective/x-ingestion/pull-log.jsonl`
|
||||||
|
|
||||||
|
Rate limit: 2-second delay between accounts. Start with core tier accounts, then extended.
|
||||||
|
|
||||||
|
### Step 2: Triage for signal
|
||||||
|
|
||||||
|
Not every tweet is worth extracting. For each account's tweets, scan for:
|
||||||
|
|
||||||
|
**High signal (extract):**
|
||||||
|
- Original analysis or arguments (not just links or reactions)
|
||||||
|
- Threads with evidence chains
|
||||||
|
- Data, statistics, study citations
|
||||||
|
- Novel claims that challenge or extend KB knowledge
|
||||||
|
- Cross-domain connections
|
||||||
|
|
||||||
|
**Low signal (skip):**
|
||||||
|
- Pure engagement farming ("gm", memes, one-liners)
|
||||||
|
- Retweets without commentary
|
||||||
|
- Personal updates unrelated to domain
|
||||||
|
- Duplicate arguments already in the KB
|
||||||
|
|
||||||
|
For each high-signal tweet or thread, note:
|
||||||
|
- Username, tweet URL, date
|
||||||
|
- Why it's high signal (1 sentence)
|
||||||
|
- Which domain it maps to
|
||||||
|
- Whether it's a new claim, counter-evidence, or enrichment to existing claims
|
||||||
|
|
||||||
|
### Step 3: Archive sources
|
||||||
|
|
||||||
|
For each high-signal item, create a source archive file on your branch:
|
||||||
|
|
||||||
|
**Filename:** `inbox/archive/YYYY-MM-DD-{username}-{brief-slug}.md`
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
type: source
|
||||||
|
title: "Brief description of the tweet/thread"
|
||||||
|
author: "Display Name (@username)"
|
||||||
|
twitter_id: "numeric_id_from_author_object"
|
||||||
|
url: https://x.com/{username}/status/{tweet_id}
|
||||||
|
date: YYYY-MM-DD
|
||||||
|
domain: {primary-domain}
|
||||||
|
format: tweet | thread
|
||||||
|
status: processing
|
||||||
|
tags: [relevant, topics]
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
**Body:** Include the full tweet text (or thread text concatenated). For threads, preserve the order and note which tweets are replies to which.
|
||||||
|
|
||||||
|
### Step 4: Extract claims
|
||||||
|
|
||||||
|
Follow `skills/extract.md` for each archived source:
|
||||||
|
|
||||||
|
1. Read the source completely
|
||||||
|
2. Separate evidence from interpretation
|
||||||
|
3. Extract candidate claims (specific, disagreeable, evidence-backed)
|
||||||
|
4. Check for duplicates against existing KB
|
||||||
|
5. Classify by domain
|
||||||
|
6. Identify enrichments to existing claims
|
||||||
|
|
||||||
|
Write claim files to `domains/{your-domain}/` with proper frontmatter.
|
||||||
|
|
||||||
|
After extraction, update the source archive:
|
||||||
|
```yaml
|
||||||
|
status: processed
|
||||||
|
processed_by: {your-name}
|
||||||
|
processed_date: YYYY-MM-DD
|
||||||
|
claims_extracted:
|
||||||
|
- "claim title 1"
|
||||||
|
- "claim title 2"
|
||||||
|
enrichments:
|
||||||
|
- "existing claim that was enriched"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 5: Branch, commit, PR
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Branch
|
||||||
|
git checkout -b {your-name}/ingest-{date}-{brief-slug}
|
||||||
|
|
||||||
|
# Stage
|
||||||
|
git add inbox/archive/*.md domains/{your-domain}/*.md
|
||||||
|
|
||||||
|
# Commit
|
||||||
|
git commit -m "{your-name}: ingest {N} claims from {source description}
|
||||||
|
|
||||||
|
- What: {N} claims from {M} tweets/threads by {accounts}
|
||||||
|
- Why: {brief rationale — what KB gap this fills}
|
||||||
|
- Connections: {key links to existing claims}
|
||||||
|
|
||||||
|
Pentagon-Agent: {Name} <{UUID}>"
|
||||||
|
|
||||||
|
# Push
|
||||||
|
FORGEJO_TOKEN=$(cat ~/.pentagon/secrets/forgejo-{your-name}-token)
|
||||||
|
git push -u https://{your-name}:${FORGEJO_TOKEN}@git.livingip.xyz/teleo/teleo-codex.git {branch-name}
|
||||||
|
```
|
||||||
|
|
||||||
|
Then open a PR on Forgejo:
|
||||||
|
```bash
|
||||||
|
curl -s -X POST "https://git.livingip.xyz/api/v1/repos/teleo/teleo-codex/pulls" \
|
||||||
|
-H "Authorization: token ${FORGEJO_TOKEN}" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{
|
||||||
|
"title": "{your-name}: ingest {N} claims — {brief description}",
|
||||||
|
"body": "## Source\n{tweet URLs and account names}\n\n## Claims\n{numbered list of claim titles}\n\n## Why\n{what KB gap this fills, connections to existing claims}\n\n## Enrichments\n{any existing claims updated with new evidence}",
|
||||||
|
"base": "main",
|
||||||
|
"head": "{branch-name}"
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
The eval pipeline handles review and auto-merge from here.
|
||||||
|
|
||||||
|
## Batch Ingestion
|
||||||
|
|
||||||
|
When running the full loop across your network:
|
||||||
|
|
||||||
|
1. Pull all accounts (Step 1)
|
||||||
|
2. Triage across all pulled tweets (Step 2) — batch the triage so you can see patterns
|
||||||
|
3. Group high-signal items by topic, not by account
|
||||||
|
4. Create one PR per topic cluster (3-8 claims per PR is ideal)
|
||||||
|
5. Don't create mega-PRs with 20+ claims — they're harder to review
|
||||||
|
|
||||||
|
## Cross-Domain Routing
|
||||||
|
|
||||||
|
If you find high-signal content outside your domain during triage:
|
||||||
|
- Archive the source in `inbox/archive/` with `status: unprocessed`
|
||||||
|
- Add `flagged_for_{agent}: ["brief reason"]` to the frontmatter
|
||||||
|
- Message the relevant agent: "New source archived for your domain: {filename}"
|
||||||
|
- Don't extract claims outside your territory — let the domain agent do it
|
||||||
|
|
||||||
|
## Quality Controls
|
||||||
|
|
||||||
|
- **Source diversity:** If you're extracting 5+ claims from one account in one batch, flag it. Monoculture risk.
|
||||||
|
- **Freshness:** Don't re-extract tweets that are already archived. Check `inbox/archive/` first.
|
||||||
|
- **Signal ratio:** Aim for ≥50% of triaged tweets yielding at least one claim. If your ratio is lower, raise your triage bar.
|
||||||
|
- **Cost tracking:** Log every API call. The pull log tracks spend across agents.
|
||||||
|
|
||||||
|
## Network Management
|
||||||
|
|
||||||
|
Your network file (`{your-name}-network.json`) lists accounts to monitor. Update it as you discover new high-signal accounts in your domain:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"agent": "your-name",
|
||||||
|
"domain": "your-domain",
|
||||||
|
"accounts": [
|
||||||
|
{"username": "example", "tier": "core", "why": "Reason this account matters"},
|
||||||
|
{"username": "example2", "tier": "extended", "why": "Secondary but useful"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Tiers:**
|
||||||
|
- `core` — Pull every ingestion cycle. High signal-to-noise ratio.
|
||||||
|
- `extended` — Pull weekly or when specifically relevant.
|
||||||
|
- `watch` — Discovered but not yet confirmed as useful. Pull once to evaluate.
|
||||||
|
|
||||||
|
Agents without a network file yet should create one as their first ingestion task. Start with 5-10 seed accounts, pull them, evaluate signal quality, then expand.
|
||||||
Loading…
Reference in a new issue