Compare commits

..

2 commits

Author SHA1 Message Date
b57cb41e31 fix(mirror): Step 0b self-heals stuck submitted_by via cron retry
Step 4.5's submitted_by write inherits the same one-shot failure mode
that Phase 1's Step 0 sweep was designed to retire. On transient
GitHub API failure, link-only fallback writes github_pr and permanently
closes the retry window — submitted_by stays stuck on 'm3taversal'
(bot identity), and contribution_events never gets the author event.

Step 0b adds a second sweep alongside Step 0:
  SELECT ... WHERE github_pr IS NOT NULL
              AND (submitted_by IS NULL OR submitted_by = 'm3taversal')

Same idempotent cron-retry shape: SELECT empty when clean, per-row
GitHub API call + UPDATE only when stuck. Targets bidirectional
repos only (gh-pr-* branches don't exist for main_only mirrors).

Derives bidirectional GitHub repo from MIRROR_REPOS at sweep time
since Step 0 runs before sync_repo() sets GITHUB_REPO scope.

Doubles as future-proof safety net: any external PR landing during
the deploy window with submitted_by stuck on bot identity gets
self-healed on the next cron tick. No backfill script needed.

Per Ganymede line-level review of 1decf09.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 18:21:04 +01:00
1decf09598 fix(mirror): Step 4.5 writes submitted_by from GitHub user.login
Closes the systematic external-PR attribution gap diagnosed on FwazB PR #4066.
The Forgejo PR was being created via admin-token (m3taversal), so
prs.submitted_by ended up as the bot identity. record_contributor_attribution
treats m3taversal as a bot and finds no other signal for fork PRs (commit
authors are bot-rewritten by sync-mirror), so zero author events emit.

Mechanism — for gh-pr-* branches in the auto-create path:
1. After deriving GH_PR_NUM from branch name, GET /pulls/{N} from GitHub API
2. Extract user.login (e.g. "FwazB") from the PR's head author
3. Validate against GitHub username regex: ^[a-zA-Z0-9]([a-zA-Z0-9-]{0,37}[a-zA-Z0-9])?$
4. Lowercase to match contributor.py canonical handle storage
5. Include submitted_by in the same UPDATE that sets github_pr + source_channel

The regex doubles as the SQL-injection safety boundary (no parametric binding
in bash sqlite3). 14 cases tested locally including SQL injection probes —
all rejected, all real handles pass.

Failure modes:
- API call fails or returns no user.login → fall back to link-only UPDATE
  (existing behavior). Better than failing the whole step.
- Regex rejects malformed login → same fall-back. Preserves audit trail.

Smoke-tested against real FwazB PR #90 on VPS: extracts "FwazB", lowercases
to "fwazb", regex passes, would write submitted_by='fwazb'. Once deployed,
record_contributor_attribution will trust submitted_by as fallback when no
agent trailer found, emit author event with weight 0.30 → FwazB-class
contributors auto-attributed end-to-end with zero manual backfill.

Architectural decisions per Ship's Apr 28 sign-off:
- (a) Sweep stays zero-API: no per-row API calls in Step 0 self-heal.
  This Step 4.5 fix is create-time only. Existing rows (FwazB) already
  manually backfilled — no further population to backfill.
- (b) Skip _BOT_AUTHORS exception (#2 in original ticket): once submitted_by
  is correct, the bot filter doesn't fire on external PRs anymore.
- (c) Defer frontmatter rewriting: convenience, not load-bearing.
2026-04-28 18:15:13 +01:00

View file

@ -275,9 +275,26 @@ print('no')
fi fi
log "Auto-created PR #$PR_NUM on Forgejo for $branch" log "Auto-created PR #$PR_NUM on Forgejo for $branch"
# Step 4.5: Link GitHub PR to Forgejo PR in pipeline DB # Step 4.5: Link GitHub PR to Forgejo PR + populate submitted_by from GitHub user.login.
#
# Why submitted_by here: sync-mirror creates the Forgejo PR using the admin
# token (user=m3taversal), so the Forgejo PR's submitter is bot-identity, not
# the GitHub author (FwazB, etc). Without this fix, contribution_events emits
# zero author events for external PRs because record_contributor_attribution
# treats m3taversal as a bot and finds no other signal. Architectural fix per
# Ship + Cory after the FwazB PR #4066 attribution gap (Apr 28).
local GH_USER
GH_USER=""
if [[ "$branch" == gh-pr-* ]]; then if [[ "$branch" == gh-pr-* ]]; then
GH_PR_NUM=$(echo "$branch" | sed 's|gh-pr-\([0-9]*\)/.*|\1|') GH_PR_NUM=$(echo "$branch" | sed 's|gh-pr-\([0-9]*\)/.*|\1|')
# Look up GitHub PR author for submitted_by attribution.
local PAT_GHU
PAT_GHU=$(cat "$GITHUB_PAT_FILE" 2>/dev/null | tr -d '[:space:]')
if [ -n "$PAT_GHU" ] && [[ "$GH_PR_NUM" =~ ^[0-9]+$ ]]; then
GH_USER=$(curl -sf "https://api.github.com/repos/$GITHUB_REPO/pulls/$GH_PR_NUM" \
-H "Authorization: token $PAT_GHU" 2>/dev/null | \
python3 -c "import sys,json; print((json.load(sys.stdin).get('user') or {}).get('login',''))" 2>/dev/null || true)
fi
else else
local PAT local PAT
PAT=$(cat "$GITHUB_PAT_FILE" 2>/dev/null | tr -d '[:space:]') PAT=$(cat "$GITHUB_PAT_FILE" 2>/dev/null | tr -d '[:space:]')
@ -289,9 +306,24 @@ print('no')
fi fi
fi fi
if [[ "$GH_PR_NUM" =~ ^[0-9]+$ ]] && [[ "$PR_NUM" =~ ^[0-9]+$ ]]; then if [[ "$GH_PR_NUM" =~ ^[0-9]+$ ]] && [[ "$PR_NUM" =~ ^[0-9]+$ ]]; then
sqlite3 "$PIPELINE_DB" "UPDATE prs SET github_pr = $GH_PR_NUM, source_channel = 'github' WHERE number = $PR_NUM;" 2>/dev/null && \ # GitHub usernames: 1-39 chars, alphanumeric + hyphen, can't start/end with hyphen.
log "Linked GitHub PR #$GH_PR_NUM -> Forgejo PR #$PR_NUM" || \ # Strict regex is the SQL-injection safety boundary (no bash sqlite3 parametric binding).
log "WARN: Failed to link GitHub PR #$GH_PR_NUM to Forgejo PR #$PR_NUM in DB" if [[ "$GH_USER" =~ ^[a-zA-Z0-9]([a-zA-Z0-9-]{0,37}[a-zA-Z0-9])?$ ]]; then
# Lowercase for canonical handle storage (matches contributor.py normalize)
local GH_USER_LC
GH_USER_LC=$(echo "$GH_USER" | tr '[:upper:]' '[:lower:]')
sqlite3 "$PIPELINE_DB" "UPDATE prs SET github_pr = $GH_PR_NUM, source_channel = 'github', submitted_by = '$GH_USER_LC' WHERE number = $PR_NUM;" 2>/dev/null && \
log "Linked GitHub PR #$GH_PR_NUM -> Forgejo PR #$PR_NUM (author: $GH_USER_LC)" || \
log "WARN: Failed to link GitHub PR #$GH_PR_NUM to Forgejo PR #$PR_NUM in DB"
else
# Fall back to link-only when user.login lookup failed or was malformed.
# Still better than no link; record_contributor_attribution can use
# other signals (commit author trailer, etc) — author event may still
# not emit, but submitted_by stays NULL/whatever discovery set.
sqlite3 "$PIPELINE_DB" "UPDATE prs SET github_pr = $GH_PR_NUM, source_channel = 'github' WHERE number = $PR_NUM;" 2>/dev/null && \
log "Linked GitHub PR #$GH_PR_NUM -> Forgejo PR #$PR_NUM (author lookup failed)" || \
log "WARN: Failed to link GitHub PR #$GH_PR_NUM to Forgejo PR #$PR_NUM in DB"
fi
fi fi
done done
} }
@ -393,6 +425,50 @@ if [ -f "$PIPELINE_DB" ]; then
log "self-heal: linked Forgejo PR #$pr_num -> GitHub PR #$gh_pr_num" log "self-heal: linked Forgejo PR #$pr_num -> GitHub PR #$gh_pr_num"
fi fi
done done
# Step 0b: self-heal stuck submitted_by on linked gh-pr-* PRs.
# Step 4.5 fall-through writes github_pr but leaves submitted_by as 'm3taversal'
# (bot identity) on transient GitHub API failures. Without retry, contribution_events
# never gets the author event — same one-shot failure mode Step 0 was designed to
# retire. Same idempotent cron-retry shape: SELECT empty when clean, per-row API
# call only when stuck. Targets bidirectional repos only (gh-pr-* branches don't
# exist for main_only mirrors).
GH_REPO_BIDIR=""
for entry in "${MIRROR_REPOS[@]}"; do
read -r _f gh_repo _bare mode <<< "$entry"
if [ "$mode" = "bidirectional" ]; then
GH_REPO_BIDIR="$gh_repo"
break
fi
done
if [ -n "$GH_REPO_BIDIR" ] && [ -f "$GITHUB_PAT_FILE" ]; then
sqlite3 -separator '|' "$PIPELINE_DB" \
"SELECT number, branch FROM prs
WHERE branch LIKE 'gh-pr-%'
AND github_pr IS NOT NULL
AND (submitted_by IS NULL OR submitted_by = 'm3taversal');" \
2>/dev/null | while IFS='|' read -r pr_num branch; do
gh_pr_num=$(echo "$branch" | sed -n 's|^gh-pr-\([0-9][0-9]*\)/.*|\1|p')
[ -z "$gh_pr_num" ] && continue
PAT_S0=$(cat "$GITHUB_PAT_FILE" 2>/dev/null | tr -d '[:space:]')
[ -z "$PAT_S0" ] && continue
gh_user=$(curl -sf "https://api.github.com/repos/$GH_REPO_BIDIR/pulls/$gh_pr_num" \
-H "Authorization: token $PAT_S0" 2>/dev/null | \
python3 -c "import sys,json; print((json.load(sys.stdin).get('user') or {}).get('login',''))" 2>/dev/null || true)
# Regex matches Step 4.5: GitHub username spec (anchored, alnum + hyphen,
# no consecutive-hyphen check). SQL-injection boundary: char class excludes
# quotes/semicolons/backslashes, so single-quoted literal is safe.
if [[ "$gh_user" =~ ^[a-zA-Z0-9]([a-zA-Z0-9-]{0,37}[a-zA-Z0-9])?$ ]]; then
gh_user_lc=$(echo "$gh_user" | tr '[:upper:]' '[:lower:]')
# SQL-integer-safe: pr_num from INTEGER column, gh_user_lc regex-validated.
if sqlite3 "$PIPELINE_DB" \
"UPDATE prs SET submitted_by = '$gh_user_lc' WHERE number = $pr_num;" \
2>/dev/null; then
log "self-heal: set submitted_by=$gh_user_lc on Forgejo PR #$pr_num (GitHub PR #$gh_pr_num)"
fi
fi
done
fi
fi fi
for entry in "${MIRROR_REPOS[@]}"; do for entry in "${MIRROR_REPOS[@]}"; do