What Makes Leo Leo: DB-First Behavior Model

Date: 2026-07-14

Definition Of Working

Working target: Leo forms durable knowledge through a replayable database lifecycle, and answers through consistent, traceable database retrieval. A conversation may become a source candidate; it does not silently train Leo or become truth.

Operator path: retain source -> extract candidates -> deduplicate/contrast -> stage proposal -> review -> guarded apply -> canonical readback -> retrieve with a source receipt.

Done means: the same retained inputs, compiler version, approved proposal, database snapshot, retrieval query, and runtime manifest reproduce the same rows and retrieval receipt. The benchmark can measure the outcome without changing Leo's live profile or canonical database.

Not done: a chat answer looks better because SOUL.md, USER.md, a skill, a plugin response template, old session state, or benchmark-specific text changed without a reviewed database change.

Required tier: live VPS database/tool lifecycle first; GCP reproduces the same manifest and receipts before promotion.

One-Line Model

Leo is an external language model run by Hermes, wrapped in the leoclean profile, given static identity and procedural layers, dynamic session context, and database retrieval. Only reviewed canonical database changes should count as durable learning.

flowchart LR
  U["Person or agent discussion"] --> A["Immutable source artifact + stable message locator"]
  A --> X["Deterministic extraction and canonical dedupe"]
  X --> P["Pending review proposal"]
  P --> R["Human or governed review"]
  R --> C["Canonical Postgres rows"]
  C --> T["Deterministic retrieval tool + receipt"]
  T --> H["Hermes reasoning turn"]
  S["SOUL + config + skills"] --> H
  M["Session continuity"] --> H
  G["Generic safety middleware"] --> H
  B["External benchmark"] -. "measure only" .-> H
  B -. "must not write" .-> C
  B -. "must not edit" .-> S

Current Live Layers

Fresh readback on 2026-07-14 found the VPS service active with NRestarts=0 and Hermes Agent v0.7.0 at commit b2f477a30b3c05d0f383c543af98496ae8a96070.

Layer	What it does	Change rate	How visible/replayable it is	Current assessment
Base-model weights	Supply general language and reasoning capability	Provider-controlled	Opaque; no local weight hash	No evidence Leo was fine-tuned. The configured model name is not a weight-level identity.
Model routing/config	Selects provider/model, routing, token/reasoning defaults, tools, and memory	Deployment-static	File-hashable; actual per-turn model still needs a receipt	OpenRouter, default `anthropic/claude-sonnet-4-6`, smart routing enabled through `google/gemini-2.5-flash`. Temperature/reasoning effort are not explicitly pinned in the observed config.
Hermes runtime	Builds prompts, sessions, tools, gateway calls, and plugin hooks	Deployment-static	Source and commit are inspectable	Generic upstream runtime, but locally patched to allow post-model response replacement before persistence.
`SOUL.md`	Primary runtime identity text loaded into the system prompt	Static until manually edited or rendered	Fully hashable; current SHA-256 `5973a54c...32a`	Important runtime input, but not canonical truth. No active general DB-to-SOUL renderer is proven.
Skills	Procedural instructions discoverable by Hermes	Deployment-static	Source-controlled and hashable	Useful for tool use, but detailed answer rules can become hidden prompt training.
`leo-db-context` plugin	Injects pre-turn DB contracts, validates drafts, and can replace the final answer	Request-dynamic over static code	Source-controlled but invisible in normal chat	The most behaviorally powerful and obscure layer. Current query-specific compiled answers are a temporary guardrail, not database reasoning.
KB bridge/tool code	Queries claims/context/evidence/edges and stages proposals	Deployment-static code over dynamic DB	Source-controlled; calls can emit receipts	Correct architectural control point. This branch adds allowlisted artifact resolution, hash verification, and deterministic context receipts on VPS.
Canonical Postgres	Stores applied claims, sources, evidence, graph, and identity/strategy rows	Governed dynamic	Row IDs, hashes, manifests, and apply receipts	Intended source of durable knowledge.
`kb_stage` proposals	Holds candidates awaiting review/apply	Dynamic, review-gated	Queryable and replayable	Correct place for discussion-derived candidates before approval.
`MEMORY.md` / `USER.md`	Persistent Hermes facts/preferences injected outside Postgres	Conversation-dynamic	File-hashable but provenance-poor	Noncanonical. Live `USER.md` contains a stale naming rule that conflicts with the exact visible-handle rule.
Sessions / `state.db`	Preserve conversation continuity	Turn-dynamic	Hashable but highly stateful	Appropriate for “what did we just discuss,” not durable collective knowledge.
Benchmark harness	Sends test turns and scores replies	External	Fully inspectable if isolated	Must remain measurement-only. The prior temp-profile harness copied live memories and sessions, weakening causal attribution.

What “Weights” Can Mean

Base-model weights: opaque parameters hosted by the model provider. We do not inspect or modify them.
Database weights: explicit claim_evidence.weight, claim_edges.weight, and strategy-anchor weights. These are reviewable rows and belong in database receipts.
Retrieval scores: deterministic query-term matching currently ranks claims using term hits, evidence counts, edge counts, confidence, and text length. These are code-level ranking rules, not learning.
Benchmark scores: regex/rubric checks outside Leo decide whether a response passed. They must never feed edits back into the runtime automatically.
Plugin contracts: not weights at all. They are hard rules that can replace a response. They currently explain part of the measured improvement and must be separated from raw model-plus-database capability.

Desired Static/Dynamic Boundary

Keep static and small

Hermes runtime version and patches.
Model routing and explicit inference parameters.
A minimal SOUL.md: stable identity, mission, voice, and immutable safety boundaries.
Generic skills describing tools and schemas.
Generic middleware: fail closed when live retrieval fails, prevent unauthorized writes, validate receipts.

Keep dynamic and canonical

Claims and claim state.
Sources, content hashes, exact excerpts, and source classification.
Claim-to-evidence and claim-to-claim relationships, including explicit weights.
Agent beliefs, strategies, identity graph rows, governance rules, and reasoning tools where supported by schema/apply contracts.
Candidate proposals, reviews, apply receipts, and supersession history.

Keep dynamic but noncanonical

The current conversation and same-session references.
Delivery metadata and Telegram message IDs.
Temporary plans/tool traces.

These may help the current turn but must not become evidence or durable identity without source capture and review.

Conversation Learning Contract

A discussion receives a stable locator such as a Telegram chat/message ID plus capture time and content hash.
The retained artifact is immutable. A display name or chat label alone is rejected as provenance.
Extraction produces typed candidate claims, evidence excerpts, relationships, and explicit uncertainty.
Existing canonical rows are retrieved first to identify duplicates, conflicts, and missing evidence.
The result enters kb_stage.kb_proposals as pending_review; no canonical row and no runtime identity file changes yet.
Review may approve, revise, split, or reject each candidate.
Guarded apply writes the supported canonical rows and emits row-level postflight proof.
Later retrieval verifies the linked artifact and database hash, then emits a deterministic receipt for the context Leo received.

An agent can act as the reviewing interlocutor, but its review identity, policy, and decision must be recorded. Agent discussion is not self-authenticating approval.

Harness Contract

The harness must:

start from a fresh session and empty persistent memory unless memory itself is the tested dimension;
record the model route, Hermes commit, config/SOUL/skill/plugin/tool hashes, and database fingerprint;
compare live profile and database fingerprints before and after;
write only inside an isolated temporary profile or disposable database;
never update SOUL.md, MEMORY.md, USER.md, skills, plugins, or live Postgres as a consequence of a score;
keep a frozen regression set and a separate blind set so passing known prompts cannot masquerade as general reasoning;
score raw reasoning separately from middleware-composed or replaced responses.

Delivery Status And Repair Order

Implemented in this branch - remove causal ambiguity from tests: exclude prior memories/sessions from temporary profiles and fingerprint every behavior-bearing layer.
Implemented and live-canary proven on VPS - complete retrieval receipts: resolve source paths, verify content hashes, stabilize row ordering, and hash the exact context bundle.
Bounded compiler behavior proven locally - make discussion ingestion real: stable message-level locators can produce pending-review proposals without profile edits, but automatic Telegram capture is not shipped.
Remaining - reduce query-specific middleware: keep generic truth and authorization guards, then measure raw DB reasoning separately from compiled response replacement.
Remaining - make identity composition deterministic: implement and receipt a DB-to-SOUL.md renderer before calling identity updates live.
Remaining - pin the execution identity: record the actual provider/model and explicit inference settings for each benchmark turn.
Remaining - reproduce on GCP: restore the same canonical manifest, deploy the same behavior manifest, replay the same lifecycle, and only then consider promotion.

This design makes a behavior change attributable: either a reviewed DB delta, a versioned runtime delta, a model-route delta, or current-session context. “Leo learned it somehow” is not an acceptable state.

Live VPS Read-Only Canary

The candidate retrieval path was exercised twice against the live VPS database without deploying it, restarting Leo, or writing to Postgres. An unfamiliar query selected claim 2a7ae257-d01d-46f4-b813-63f81bb9c7c7 and its two evidence rows.

One source has a database hash but no URL or storage pointer. The receipt reports no_source_pointer; it does not misstate that the canonical evidence row or raw artifact is missing.
The second source uses a relative storage path. The tool resolved the path under an allowlisted workspace root, found the artifact, calculated SHA-256 341a7685...9f646, and matched the database hash exactly.
Both reads emitted semantic context hash 8333bc7a...38d9c and artifact-state hash 47473661...356fc with a stable WAL marker. Each read returned four claims, fourteen evidence rows, and nine edges in the same order.
Before/after counts stayed at 1,837 claims, 4,145 sources, 4,670 evidence links, 4,916 claim edges, and 29 proposals. The gateway PID and restart count were unchanged, and the temporary candidate was removed.

The machine-readable receipt is db-first-source-verification-canary-20260714.json. This proves deterministic read and artifact verification on VPS. It does not yet prove a Telegram-visible answer, GCP parity, or an approved canonical write.

12 KiB Raw Blame History