Ganymede review on PR #6 (commit 800d1d8 → this commit):
- WARNING: STAT_BLEED_RE false-positives on common words. The original
pattern matched standalone stat-keyword tokens, clipping legitimate
titles like "Engage with Pantera and Active Capital" → trimmed at
" Active". Fix: require numeric/symbolic context (\$, +, -, \d) AFTER
the stat-word, so word-only sequences pass through unchanged.
- _clean_title_candidate now uses finditer + first-match-past-offset-10
instead of re.search. The DP-NNNNN digit sequence always wins leftmost
position; we want the first POST-title bleed match instead.
- NIT 3: minimum-render gate before write. Skip partial renders rather
than archiving stubs whose downstream extraction null-results.
Threshold: body < 500B AND no DP-N in title → skip and retry next run.
Verified 10/10 on test grid: real bleed trimmed, mid-word false-positives
preserved (Compass, Active Capital, Live Streaming, Encompass, Activate,
Passage, Failure all pass through unchanged).
NIT 1 (--headless no-op flag) and NIT 2 (futardio tag provenance noise):
deferred — cosmetic, batch with future touch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ganymede review on PR #6:
- WARNING: title and project["name"] flowed unescaped into YAML, would
corrupt frontmatter on quote-bearing inputs (e.g. 'Adopt "Conservative"
Pricing'). New _yaml_str helper routes free-text values through
json.dumps (JSON strings are valid YAML strings). Applied to title,
author, url, project_slug, proposal_address, proposal_status,
squads_proposal, squads_status.
- NIT: URL_ADDR_RE didn't match new metadao.fi URLs — pattern segment
couldn't span /projects/{slug}/proposal/. Added (?:/[^/...]*)*? for
variable path depth. Verified against three URL shapes.
- NIT: dry_run key was omitted from JSON output on early --limit exit
but present on normal exit. Trivial consistency fix.
- NIT (deferred): STAT_BLEED_RE protection is accidental rather than
designed; only matters if MetaDAO breaks DP-NNNNN naming convention.
Per Ganymede 'optional — current behavior fine.'
Verified: URL regex matches futard.io legacy + metadao.fi new + hypothetical
no-slug shapes. YAML escape survives embedded quotes, newlines, backslashes,
em-dashes.
Background:
- futard.io retired its /api/graphql endpoint between Apr 17–20
- Cloud Scheduler ingest-futard has been firing into 500s ever since
(the AttributeError on e.url masked the real 404 for 5 days; fixed
in living-ip/teleo-api@b8eb441 which surfaced the actual root cause)
- The ecosystem migrated to metadao.fi, which is Vercel-protected
- Direct curl is blocked by Vercel's anti-bot challenge regardless of
headers; a real headless browser passes it cleanly
Approach:
- Playwright-driven scraper, runs as a one-shot
- Discovery: scrape /projects DOM for project slugs, then each
/projects/{slug} for proposal addresses
- For each NEW proposal: visit page for prose body + call
/api/decode-proposal/{addr} via in-browser fetch (bypasses challenge
via the primed Vercel cookies in the browser context) for structured
on-chain instructions
- Idempotent: dedup against existing proposal addresses in archive
frontmatter AND filename basenames
- Filename embeds 8-char address fragment for stable cross-run dedup
even on projects that don't use DP-NNNNN naming convention
Tested locally against 6 active projects (p2p-protocol, paystream,
zklsol, loyal, ranger, solomon). Captured 13 new proposals — including
the Solomon Gigabus DP-00003 that triggered this work — with proper
titles, status, on-chain instruction decoding (Squads transactions,
SPL transfers, memos), and project metadata.
Output schema matches existing futardio source files (type: source,
event_type: proposal, domain: internet-finance, status: unprocessed)
so the existing extract pipeline picks them up unchanged.
Architectural note: this script is intentionally NOT wired to systemd
yet — VPS deploy needs Playwright + Chromium system libs which require
apt sudo (currently scoped to teleo-* services only). Reviewing the
script first; deploy path is a separate decision.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
