teleo/teleo-codex
8
0
Fork 0
Code Issues 22 Pull requests 1 Projects Releases Packages Wiki Activity Actions 1
teleo-codex/ops/diagnostics/dashboard_agents.py
m3taversal 05d74d5e32 sync: import all VPS pipeline + diagnostics code as baseline 
Imports 67 files from VPS (/opt/teleo-eval/) into repo as the single source
of truth. Previously only 8 of 67 files existed in repo — the rest were
deployed directly to VPS via SCP, causing massive drift.

Includes:
- pipeline/lib/: 33 Python modules (daemon core, extraction, evaluation, merge, cascade, cross-domain, costs, attribution, etc.)
- pipeline/: main daemon (teleo-pipeline.py), reweave.py, batch-extract-50.sh
- diagnostics/: 19 files (4-page dashboard, alerting, daily digest, review queue, tier1 metrics)
- agent-state/: bootstrap, lib-state, cascade inbox processor, schema
- systemd/: service unit files for reference
- deploy.sh: rsync-based deploy with --dry-run, syntax checks, dirty-tree gate
- research-session.sh: updated with Step 8.5 digest + cascade inbox processing

No new code written — all files are exact copies from VPS as of 2026-04-06.
From this point forward: edit in repo, commit, then deploy.sh.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 00:00:00 +01:00

348 lines
15 KiB
Python
Raw Blame History

"""Page 3: Agent Performance — "Who's contributing what?"
Slim version v2 per Cory feedback (2026-04-03):
- Hero: total merged, rejection rate, claims/week — 3 numbers
- Table: agent, merged, rejection rate, last active, inbox depth — 5 columns
- One chart: weekly contributions by agent (stacked bar)
- No CI scores, no yield (redundant with rejection rate), no top issue (too granular)
Fetches /api/agents-dashboard + /api/agent-state, merges client-side.
"""
from datetime import datetime
from shared_ui import render_page
def render_agents_page(contributors_principal: list, contributors_agent: list, now: datetime) -> str:
"""Render the slim Agent Performance page."""
body = """
<!-- Hero Metrics (filled by JS) -->
<div class="grid" id="hero-metrics">
<div class="card" style="text-align:center;color:#8b949e">Loading...</div>
</div>
<!-- Per-Agent Table -->
<div class="section">
<div class="section-title">Agent Breakdown (30d)</div>
<div class="card">
<table id="agent-table">
<tr>
<th>Agent</th>
<th style="text-align:right">Merged</th>
<th style="text-align:right">Rejection Rate</th>
<th style="text-align:right">Last Active</th>
<th style="text-align:right">Inbox</th>
</tr>
<tr><td colspan="5" style="color:#8b949e;text-align:center">Loading...</td></tr>
</table>
</div>
</div>
<!-- Weekly Contributions Chart -->
<div class="section">
<div class="chart-container" style="max-width:100%">
<h2>Claims Merged per Week by Agent</h2>
<canvas id="trendChart"></canvas>
</div>
</div>
<!-- Agent Scorecard (from review_records) -->
<div class="section">
<div class="section-title">Agent Scorecard (Structured Reviews)</div>
<div class="card">
<table id="scorecard-table">
<tr><td colspan="7" style="color:#8b949e;text-align:center">Loading...</td></tr>
</table>
<div id="scorecard-rejections" style="margin-top:12px"></div>
</div>
</div>
<!-- Latest Session Digests -->
<div class="section">
<div class="section-title">Latest Session Digests</div>
<div id="digest-container">
<div class="card" style="text-align:center;color:#8b949e">Loading...</div>
</div>
</div>
"""
scripts = """<script>
Promise.all([
fetch('/api/agents-dashboard?days=30').then(r => r.json()),
fetch('/api/agent-state').then(r => r.json()).catch(() => ({agents: {}}))
])
.then(([data, stateData]) => {
const agents = data.agents || {};
const agentState = stateData.agents || {};
// Sort by approved desc, filter to agents with evals
const sorted = Object.entries(agents)
.filter(([_, a]) => a.evaluated > 0)
.sort((a, b) => (b[1].approved || 0) - (a[1].approved || 0));
// --- Hero metrics ---
let totalMerged = 0, totalRejected = 0, totalEval = 0;
const weekMerged = {};
for (const [_, a] of sorted) {
totalMerged += a.approved || 0;
totalRejected += a.rejected || 0;
totalEval += a.evaluated || 0;
if (a.weekly_trend) {
a.weekly_trend.forEach(w => {
weekMerged[w.week] = (weekMerged[w.week] || 0) + (w.merged || 0);
});
}
}
const weeks = Object.keys(weekMerged).sort();
const recentWeeks = weeks.slice(-4);
const claimsPerWeek = recentWeeks.length > 0
? Math.round(recentWeeks.reduce((s, w) => s + weekMerged[w], 0) / recentWeeks.length)
: 0;
const rejRate = totalEval > 0 ? ((totalRejected / totalEval) * 100).toFixed(1) : '0';
document.getElementById('hero-metrics').innerHTML =
'<div class="card" style="text-align:center">' +
'<div class="label">Claims Merged (30d)</div>' +
'<div style="font-size:32px;font-weight:700;color:#3fb950">' + totalMerged + '</div>' +
'</div>' +
'<div class="card" style="text-align:center">' +
'<div class="label">Rejection Rate</div>' +
'<div style="font-size:32px;font-weight:700;color:' + (parseFloat(rejRate) > 30 ? '#f85149' : '#e3b341') + '">' + rejRate + '%</div>' +
'</div>' +
'<div class="card" style="text-align:center">' +
'<div class="label">Claims/Week (avg last 4w)</div>' +
'<div style="font-size:32px;font-weight:700;color:#58a6ff">' + claimsPerWeek + '</div>' +
'</div>';
// --- Per-agent table ---
if (sorted.length === 0) {
document.getElementById('agent-table').innerHTML =
'<tr><th>Agent</th><th>Merged</th><th>Rejection Rate</th><th>Last Active</th><th>Inbox</th></tr>' +
'<tr><td colspan="5" style="color:#8b949e;text-align:center">No evaluation data yet</td></tr>';
return;
}
// Helper: format relative time
function timeAgo(isoStr) {
if (!isoStr) return '<span style="color:#484f58">unknown</span>';
const diff = (Date.now() - new Date(isoStr).getTime()) / 1000;
if (diff < 3600) return Math.round(diff / 60) + 'm ago';
if (diff < 86400) return Math.round(diff / 3600) + 'h ago';
return Math.round(diff / 86400) + 'd ago';
}
let tableHtml = '<tr><th>Agent</th><th style="text-align:right">Merged</th>' +
'<th style="text-align:right">Rejection Rate</th>' +
'<th style="text-align:right">Last Active</th>' +
'<th style="text-align:right">Inbox</th></tr>';
for (const [name, a] of sorted) {
const color = agentColor(name);
const rr = a.evaluated > 0 ? ((a.rejected / a.evaluated) * 100).toFixed(1) + '%' : '-';
const rrColor = a.rejection_rate > 0.3 ? '#f85149' : a.rejection_rate > 0.15 ? '#e3b341' : '#3fb950';
// Agent state lookup (case-insensitive match)
const stateKey = Object.keys(agentState).find(k => k.toLowerCase() === name.toLowerCase()) || '';
const state = agentState[stateKey] || {};
const lastActive = timeAgo(state.last_active);
const inboxDepth = state.inbox_depth != null ? state.inbox_depth : '-';
const inboxColor = inboxDepth > 10 ? '#f85149' : inboxDepth > 5 ? '#d29922' : inboxDepth > 0 ? '#58a6ff' : '#3fb950';
tableHtml += '<tr>' +
'<td><span style="display:inline-block;width:8px;height:8px;border-radius:50%;background:' + color + ';margin-right:6px"></span>' + esc(name) + '</td>' +
'<td style="text-align:right;font-weight:600;color:#3fb950">' + (a.approved || 0) + '</td>' +
'<td style="text-align:right;color:' + rrColor + '">' + rr + '</td>' +
'<td style="text-align:right">' + lastActive + '</td>' +
'<td style="text-align:right;color:' + inboxColor + '">' + inboxDepth + '</td>' +
'</tr>';
}
document.getElementById('agent-table').innerHTML = tableHtml;
// --- Weekly trend chart ---
const allWeeks = new Set();
const agentNames = [];
for (const [name, a] of sorted) {
if (a.weekly_trend && a.weekly_trend.length > 0) {
agentNames.push(name);
a.weekly_trend.forEach(w => allWeeks.add(w.week));
}
}
const sortedWeeks = [...allWeeks].sort();
if (sortedWeeks.length > 0 && agentNames.length > 0) {
const trendMap = {};
for (const [name, a] of sorted) {
if (a.weekly_trend) {
trendMap[name] = {};
a.weekly_trend.forEach(w => { trendMap[name][w.week] = w.merged; });
}
}
new Chart(document.getElementById('trendChart'), {
type: 'bar',
data: {
labels: sortedWeeks,
datasets: agentNames.map(name => ({
label: name,
data: sortedWeeks.map(w => (trendMap[name] || {})[w] || 0),
backgroundColor: agentColor(name),
})),
},
options: {
responsive: true,
scales: {
x: { stacked: true, grid: { display: false } },
y: { stacked: true, title: { display: true, text: 'Claims Merged' }, min: 0 },
},
plugins: { legend: { labels: { boxWidth: 12 } } },
},
});
}
}).catch(err => {
document.getElementById('hero-metrics').innerHTML =
'<div class="card" style="grid-column:1/-1;text-align:center;color:#f85149">Failed to load: ' + err.message + '</div>';
});
// --- Agent Scorecard ---
fetch('/api/agent-scorecard')
.then(r => r.json())
.then(data => {
const cards = data.scorecards || [];
if (cards.length === 0 || cards.every(c => c.total_reviews === 0)) {
document.getElementById('scorecard-table').innerHTML =
'<tr><td colspan="7" style="color:#8b949e;text-align:center">No structured review data yet (review_records table is empty)</td></tr>';
return;
}
let html = '<tr><th>Agent</th><th style="text-align:right">PRs</th><th style="text-align:right">Reviews</th>' +
'<th style="text-align:right">Approved</th><th style="text-align:right">w/ Changes</th>' +
'<th style="text-align:right">Rejected</th><th style="text-align:right">Approval Rate</th></tr>';
const allReasons = {};
for (const c of cards) {
const arColor = c.approval_rate >= 80 ? '#3fb950' : c.approval_rate >= 60 ? '#d29922' : '#f85149';
html += '<tr>' +
'<td><span style="display:inline-block;width:8px;height:8px;border-radius:50%;background:' + agentColor(c.agent) + ';margin-right:6px"></span>' + esc(c.agent) + '</td>' +
'<td style="text-align:right">' + c.total_prs + '</td>' +
'<td style="text-align:right">' + c.total_reviews + '</td>' +
'<td style="text-align:right;color:#3fb950">' + c.approved + '</td>' +
'<td style="text-align:right;color:#d29922">' + c.approved_with_changes + '</td>' +
'<td style="text-align:right;color:#f85149">' + c.rejected + '</td>' +
'<td style="text-align:right;font-weight:600;color:' + arColor + '">' + c.approval_rate.toFixed(1) + '%</td>' +
'</tr>';
if (c.rejection_reasons) {
for (const [reason, cnt] of Object.entries(c.rejection_reasons)) {
allReasons[reason] = (allReasons[reason] || 0) + cnt;
}
}
}
document.getElementById('scorecard-table').innerHTML = html;
// Top rejection reasons across all agents
const sortedReasons = Object.entries(allReasons).sort((a, b) => b[1] - a[1]);
if (sortedReasons.length > 0) {
let rHtml = '<div style="font-size:12px;font-weight:600;color:#8b949e;margin-bottom:6px;text-transform:uppercase">Top Rejection Reasons</div>';
rHtml += sortedReasons.map(([reason, cnt]) =>
'<span style="display:inline-block;margin:2px 4px;padding:3px 10px;background:#f8514922;border:1px solid #f8514944;border-radius:12px;font-size:12px;color:#f85149">' +
esc(reason) + ' <strong>' + cnt + '</strong></span>'
).join('');
rHtml += '<div style="margin-top:8px;font-size:11px;color:#484f58">Target: 80% approval rate. Too high = too conservative, too low = wasting pipeline compute.</div>';
document.getElementById('scorecard-rejections').innerHTML = rHtml;
}
}).catch(() => {
document.getElementById('scorecard-table').innerHTML =
'<tr><td colspan="7" style="color:#8b949e;text-align:center">Failed to load scorecard</td></tr>';
});
// --- Latest Session Digests ---
fetch('/api/session-digest?latest=true')
.then(r => r.json())
.then(data => {
const digests = data.digests || [];
if (digests.length === 0) {
document.getElementById('digest-container').innerHTML =
'<div class="card" style="text-align:center;color:#8b949e">No session digests yet. Data starts flowing when agents complete research sessions.</div>';
return;
}
let html = '<div class="grid" style="grid-template-columns:repeat(auto-fit, minmax(320px, 1fr))">';
for (const d of digests) {
const color = agentColor(d.agent);
const dateStr = d.date || d.timestamp || '';
html += '<div class="card" style="border-left:3px solid ' + color + '">' +
'<div style="display:flex;justify-content:space-between;align-items:center;margin-bottom:8px">' +
'<strong style="color:' + color + '">' + esc(d.agent || 'unknown') + '</strong>' +
'<span style="font-size:11px;color:#484f58">' + esc(dateStr) + '</span>' +
'</div>';
if (d.research_question) {
html += '<div style="font-size:13px;font-style:italic;color:#c9d1d9;margin-bottom:8px">' + esc(d.research_question) + '</div>';
}
if (d.key_findings && d.key_findings.length > 0) {
html += '<div style="font-size:11px;color:#8b949e;text-transform:uppercase;margin-bottom:4px">Key Findings</div><ul style="margin:0 0 8px 16px;font-size:12px">';
for (const f of d.key_findings) html += '<li>' + esc(f) + '</li>';
html += '</ul>';
}
if (d.surprises && d.surprises.length > 0) {
html += '<div style="font-size:11px;color:#8b949e;text-transform:uppercase;margin-bottom:4px">Surprises</div><ul style="margin:0 0 8px 16px;font-size:12px">';
for (const s of d.surprises) html += '<li>' + esc(s) + '</li>';
html += '</ul>';
}
if (d.confidence_shifts && d.confidence_shifts.length > 0) {
html += '<div style="font-size:11px;color:#8b949e;text-transform:uppercase;margin-bottom:4px">Confidence Shifts</div>';
for (const cs of d.confidence_shifts) {
const arrow = cs.direction === 'up' ? '&#9650;' : cs.direction === 'down' ? '&#9660;' : '&#9654;';
const arrowColor = cs.direction === 'up' ? '#3fb950' : cs.direction === 'down' ? '#f85149' : '#d29922';
html += '<div style="font-size:12px;margin-left:16px"><span style="color:' + arrowColor + '">' + arrow + '</span> ' + esc(cs.claim || cs.topic || '') + '</div>';
}
}
// Expandable details
const detailId = 'digest-detail-' + Math.random().toString(36).substr(2, 6);
const hasDetails = (d.sources_archived && d.sources_archived.length > 0) ||
(d.prs_submitted && d.prs_submitted.length > 0) ||
(d.follow_ups && d.follow_ups.length > 0);
if (hasDetails) {
html += '<a style="color:#58a6ff;cursor:pointer;font-size:11px;display:block;margin-top:6px" ' +
'onclick="var e=document.getElementById(\\x27' + detailId + '\\x27);e.style.display=e.style.display===\\x27none\\x27?\\x27block\\x27:\\x27none\\x27">Details</a>';
html += '<div id="' + detailId + '" style="display:none;margin-top:6px;font-size:12px">';
if (d.sources_archived && d.sources_archived.length > 0) {
html += '<div style="color:#8b949e;font-size:11px">Sources: ' + d.sources_archived.length + '</div>';
}
if (d.prs_submitted && d.prs_submitted.length > 0) {
html += '<div style="color:#8b949e;font-size:11px">PRs: ' + d.prs_submitted.map(p => '#' + p).join(', ') + '</div>';
}
if (d.follow_ups && d.follow_ups.length > 0) {
html += '<div style="color:#8b949e;font-size:11px;margin-top:4px">Follow-ups:</div><ul style="margin:2px 0 0 16px">';
for (const fu of d.follow_ups) html += '<li>' + esc(fu) + '</li>';
html += '</ul>';
}
html += '</div>';
}
html += '</div>';
}
html += '</div>';
document.getElementById('digest-container').innerHTML = html;
}).catch(() => {
document.getElementById('digest-container').innerHTML =
'<div class="card" style="text-align:center;color:#8b949e">Failed to load session digests</div>';
});
</script>"""
return render_page(
title="Agent Performance",
subtitle="Who's contributing what?",
active_path="/agents",
body_html=body,
scripts=scripts,
timestamp=now.strftime("%Y-%m-%d %H:%M UTC"),
)
Reference in a new issue View git blame Copy permalink