_fetch_url_content was doing raw HTTP GET on X URLs which returns
JavaScript, not article content. Now routes X/Twitter URLs through
Ben's API via x_client.fetch_from_url which returns structured
article content (contents[] array with typed blocks).
Article content gets included in the archived source file so the
extraction pipeline has the actual content, not just Rio's response.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
Stale learning ("I don't have Robin Hanson data") overrode real KB data.
Ganymede review: dated entries expire after 7 days. Permanent entries
(communication style, identity) are undated and always included.
Prompt guard: "NEVER save a learning about what data you do or don't have"
prevents the bot from writing availability claims that go stale.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
When two related entities match (advisor hire + research grant), both need
full content so Opus can distinguish them and serve the right one.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
'the', 'full', 'text', 'proposal' etc. were matching irrelevant entities.
Robin Hanson record ranked #2 behind Drift because Drift matched 'the' and
'proposal' in its name. Now only meaningful tokens (>=3 chars, not stop
words) contribute to entity scoring.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
Before: "Robin Hanson MetaDAO proposal" returned 34 entities (39K chars)
with the target record buried at position 13. No relevance scoring.
After: entities scored by query token overlap (name 3x, alias 1x,
bigram 5x), limited to top 5 results. Decision records get full body
(2K chars) instead of 500-char truncation. Top result gets 2K in prompt,
rest get 500.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
MetaDAO queries now surface MetaDAO's decision records because
parent_entity: "[[metadao]]" is stripped and added to the alias set.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
Root cause: decision records have type: decision, but the entity indexer
only accepted type: entity and only scanned entities/. The claim indexer
scanned decisions/ but filtered out non-claim types. Result: decision
records fell through both indexes entirely — invisible to the bot.
Fix: add decisions/ to entity indexer scan paths, accept type: decision
alongside type: entity, include summary/proposer in search aliases.
Remove decisions/ from claim indexer (was silently dropping them anyway).
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
1. handle_research gets silent=True param. RESEARCH: tag triggers use
silent mode — archives tweets but posts no follow-up message.
Prevents "Queued N tweets" after Opus already responded.
2. KB retrieval now searches decisions/ directory alongside domains/,
core/, foundations/. Decision records (Robin Hanson proposal, etc.)
are now findable by the bot.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
Article endpoint returns body in "contents" array of typed blocks
(unstyled, header-two, markdown, list-item, blockquote, etc).
Was looking for article.text which is empty. Now parses all block types
into readable text. Also extracts engagement stats (likes, views).
Fixes: "Claude + Obsidian" article returned title but empty text.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
Primary path: GET /twitter/tweets?tweet_ids={id} — works for any tweet,
any age, returns full content. Replaces the fragile from:username search
pagination fallback.
Fallback: article endpoint for X long-form articles.
Last resort: placeholder with [Could not fetch] message.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
Major changes this session:
- fetch_tweet_by_url: extracts username+ID from X URLs, tries article endpoint,
falls back to from:username search. Tweets injected into Opus prompt.
- Haiku pre-pass: decides if X search needed before Opus responds. 2-3 word queries.
- systemd ProtectSystem paths fixed (ROOT CAUSE of all write failures since day 1)
- Research regex handles Telegram @botname suffix in groups
- Double research message prevented (skip RESEARCH: tag when Haiku already ran)
- Engagement filter dropped to 0 for niche crypto tweets
- Heuristic brevity in prompt (not hard cap)
- DM auto-respond gating (groups: reply-to only, DMs: auto-respond)
- All code now edited in pipeline-v2 repo, not /tmp
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
- Skip RESEARCH: tag when Haiku pre-pass already searched (no double-fire)
- Haiku told to use 2-3 word queries (was generating 6+ word queries that returned 0)
- Engagement filter dropped to 0 (niche crypto tweets have low engagement)
- systemd ProtectSystem paths fixed (root cause of ALL write failures)
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
Before Opus responds, Haiku evaluates: "Does this message need an X search?"
If YES, searches X, injects results into Opus prompt, archives as source.
Opus responds with KB knowledge + fresh tweet data combined.
Flow: user asks naturally ("what are people saying about P2P?") → Haiku
decides search needed → X search → results in Opus context → unified response.
~1s latency, ~$0.001 cost per message. Only fires when Haiku says YES.
Explicit /research command still works as direct path.
Also: fixed systemd ProtectSystem paths (Ganymede: root cause of all
write failures). Fixed research regex for Telegram group commands.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
Replaced hard rules with judgment heuristics:
- "Does every sentence add something the user doesn't already know?"
- "Earn every paragraph — each needs a distinct insight"
- "Short questions deserve short answers"
Restored max_tokens to 1024. Agent decides length, not a token cap.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
1. Response length: "BREVITY IS YOUR DEFAULT. Most responses 1-3 sentences.
A 4-paragraph response to a simple question is a failure."
max_tokens cut from 1024 to 512.
2. Research trigger: removed natural language regex (caused false positive
on "has accumulated" matching "search"). Only explicit /research command.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
Option A: history only contains actual bot-user exchanges, not unaddressed
group messages. Empty bot responses in history confused the model.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
Opus decides what to learn. Prompt instructs: append LEARNING: [category] [description]
at end of response when genuinely learning something new. Bot parses the line,
strips it from displayed response, calls _save_learning() to persist.
Zero additional API calls (Rhea's design). The model already has full context.
Categories: factual, communication, structured_data.
Most responses have no LEARNING line — only fires on genuine corrections.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
1. Conversation history now shows compressed context summary first
(tickers, key figures, exchange count) before full log.
"Discussing: $FUTARDIO | Key figures: $0.004, $39.5K | Exchanges: 3"
20 tokens, unmissable. Plus prompt instruction: "NEVER ask a question
your history already answers." (Ganymede: Option C+A)
2. Archive file writes decoupled from worktree lock. File written
unlocked (additive, no coordination needed). Git commit attempted
with lock — deferred on timeout, file persists on disk for next cycle.
Fixes "Read-only file system" archive failures. (Ganymede review)
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
Auto-respond stripped from conversation window. Bot only responds to @tag
and reply-to-bot. Window now silently tracks messages for context — when
the user does reply, the bot has full conversation history.
Also: prompt shortened to "1-2 sentences" default. "Do NOT respond to
messages that aren't directed at you."
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
User says "@FutAIrdBot /research P2P.me launch" → bot searches X via twitterapi.io
→ archives all tweets as ONE consolidated source file in inbox/queue/ → batch extract
picks up → claims land in KB.
Features (Ganymede+Rhea+Leo+Rio consensus):
- Regex + natural language intent detection (not CommandHandler)
- One source file per research query (not per-tweet)
- Full tweet metadata: author, followers, engagement, date
- Contributor attribution: proposed_by + contribution_type: research-direction
- Rate limit: 3 searches per user per day
- Min engagement filter (3 interactions)
- Worktree lock on source file write
Phase 2 (not built): domain alignment check before searching.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
Learnings file content now passes through sanitize_message() before injection
into the Opus prompt. Prevents prompt injection via crafted "corrections."
Rio UUID 5551F5AF confirmed as current Teleo v4 Rio.
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
Option D (Rhea+Rio+Leo consensus):
- _load_learnings(): reads agents/rio/learnings.md, injects into prompt before KB context
- _save_learning(): appends correction to learnings.md via worktree lock + direct commit
- Learnings prioritized over KB data when they conflict
- Three categories: communication, factual, structured_data
- Prompt updated: tells agent it can save corrections for future conversations
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
