Teleo evaluation pipeline infrastructure — Python async daemon for claim extraction, validation, evaluation, and merge
m3taversal
1d6b51527a
Rewrite claim-level pass in backfill-events.py to recover the Forgejo PR that introduced each claim via a cascade of 4 strategies (reliability order), replacing the single title→description match that missed PRs with NULL description (Cameron #3377) and bare-subject extracts (Shaga's Leo research PR). ## Strategies 1. sourced_from frontmatter → prs.source_path stem match 2. git log first-add commit → subject pattern → prs.branch - "<agent>: extract claims from <slug>" → extract/<slug> - "<agent>: research session YYYY-MM-DD" → <agent>/research-<date> - "<agent>: (challenge|contrib|entity|synthesize)" → <agent>/* - "Recover X from GitHub PR #N" → prs.github_pr=N - "Extract N claims from X" (no prefix) → time-proximity on agent-owned branches within 24h 3. Current title_desc fallback for anything the above miss ## Dry-run projection (1,662 merged PRs) Before: Claims processed: 33 Originator events: 6 Breakdown: {no_pr_match: 1608, no_sourcer: 26, invalid_handle: 21, skip_self: 6} After: Claims processed: 505 (+472) Originator events: 126 (+120) Strategy hits: git_subject=412, sourced_from=88, git_time_proximity=5 Breakdown: {no_pr_match: 1095, no_sourcer: 67, invalid_handle: 359, skip_self: 20} ## Verified on real VPS data - @thesensatore claims: 3/5 resolve via git_time_proximity to leo/ PRs - Cameron-S1, alexastrum: remain None — their recovery commits (dba00a79, da64f805) bypassed the pipeline entirely, no Forgejo PR record exists. Requires synthetic prs rows — deferred to separate commit with its own Ganymede review (write operation, larger blast radius than this pure-read backfill change). ## Implementation - New find_pr_for_claim(conn, repo, md) helper returns (pr_number, strategy) - Claim-level pass uses it first, falls back to title_desc map - Strategy counter surfaced in summary output for operator visibility Idempotent — backfill re-runs skip duplicate events via the partial UNIQUE index on contribution_events. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .forgejo/workflows | ||
| agent-state | ||
| deploy | ||
| diagnostics | ||
| docs | ||
| hermes-agent | ||
| lib | ||
| ops | ||
| research | ||
| scripts | ||
| systemd | ||
| telegram | ||
| tests | ||
| .gitignore | ||
| CODEOWNERS | ||
| fetch_coins.py | ||
| pyproject.toml | ||
| README.md | ||
| reweave.py | ||
| teleo-pipeline.py | ||
teleo-infrastructure
Pipeline infrastructure for the Teleo collective knowledge base. Async Python daemon that extracts, validates, evaluates, and merges claims via Forgejo PRs.
Directory Structure
teleo-infrastructure/
├── teleo-pipeline.py # Daemon entry point
├── reweave.py # Reciprocal edge maintenance
├── lib/ # Pipeline modules (Python package)
├── diagnostics/ # Monitoring dashboard (port 8081)
├── telegram/ # Telegram bot interface
├── deploy/ # Deployment + mirror scripts
├── systemd/ # Service definitions
├── agent-state/ # Cross-session agent state
├── research/ # Nightly research orchestration
├── hermes-agent/ # Hermes agent setup
├── scripts/ # One-off backfills + migrations
├── tests/ # Test suite
└── docs/ # Operational documentation
Ownership
Each directory has one owning agent. The owner is accountable for correctness and reviews all changes to their section. See CODEOWNERS for per-file detail.
| Directory | Owner | What it does |
|---|---|---|
lib/ (core) |
Ship | Config, DB, merge, cascade, validation, LLM calls |
lib/ (extraction) |
Epimetheus | Source extraction, entity processing, pre-screening |
lib/ (evaluation) |
Leo | Claim evaluation, analytics, attribution |
lib/ (health) |
Argus | Health checks, search, claim index |
diagnostics/ |
Argus | 4-page dashboard, alerting, vitality metrics |
telegram/ |
Ship | Telegram bot, X integration, retrieval |
deploy/ |
Ship | rsync deploy, GitHub-Forgejo mirror |
systemd/ |
Ship | teleo-pipeline, teleo-diagnostics, teleo-agent@ |
agent-state/ |
Ship | Bootstrap, state library, cascade inbox processor |
research/ |
Ship | Nightly research sessions, prompt templates |
scripts/ |
Ship | Backfills, migrations, one-off maintenance |
tests/ |
Ganymede | pytest suite, integration tests |
docs/ |
Shared | Architecture, specs, protocols |
VPS Layout
Runs on Hetzner CAX31 (77.42.65.182) as user teleo.
| VPS Path | Repo Source | Service |
|---|---|---|
/opt/teleo-eval/pipeline/ |
lib/, teleo-pipeline.py, reweave.py |
teleo-pipeline |
/opt/teleo-eval/diagnostics/ |
diagnostics/ |
teleo-diagnostics |
/opt/teleo-eval/telegram/ |
telegram/ |
(manual) |
/opt/teleo-eval/agent-state/ |
agent-state/ |
(used by research-session.sh) |
Quick Start
# Run tests
pip install -e ".[dev]"
pytest
# Deploy to VPS
./deploy/deploy.sh --dry-run # preview
./deploy/deploy.sh # deploy