[agentrx-optimizer] Daily Workflow Optimization - 2026-10-22 #71053

Executive Summary

The AgentRx LLM pipeline (ir/static/dynamic/check/judge/report) could not run in this sandbox — the copilot endpoint it defaults to has no authentication available, and no Azure/TRAPI credentials are configured either (see report_missing_tool). All 6 stages failed at the ir step after 5 retries each, producing zero stage artifacts.

Falling back to the raw pre-downloaded run/audit telemetry (summary.json + per-run audit), the clearest, best-evidenced finding is: the "Checkout PR branch" step in the agent job is the single largest failure cluster of the last 24h — 7 of 35 failed runs (20% share, Wilson 95% CI 10–36%), spanning 5 different PR-review workflows and 3 different engines, and it fails in roughly 1 second, before the agent model is ever invoked.

AgentRx Evidence

• Critical step: agent job → Checkout PR branch step (id checkout-pr)
• Failure category: driver_exit (infrastructure/pre-agent gate failure, not an agent reasoning error)
• Frequency / impact: cluster failure-a7e3cc48252fc236, count 7/35 failed runs (20% share); these 7 runs are a strict subset of the 14-run "Handle agent failure" aftermath cluster (failure-e121990520ae929c) in the conclusion job — i.e. this one step accounts for half of all downstream conclusion-job failures observed
• Cross-run confirmation: per-run baseline comparison shows turns: before=1 → after=0 for the representative run — the model never got a turn, confirming the failure happens before agent invocation, consistent with the step's ~1s duration (too fast for a network/git timeout, consistent with a synchronous permission check failing fast)
• Affected workflows: design-decision-gate, mattpocock-skills-reviewer, ponytail-reviewer, pr-code-quality-reviewer, test-quality-sentinel — all PR-triggered review/analysis workflows
• Representative runs: §39709752322 (Test Quality Sentinel), §39674517281 (Matt Pocock Skills Reviewer), §39674517258 (Ponytail Reviewer)

[CopilotCLI] exit 1: Error: No authentication information found.
[ERROR] Stage failed: LLM IR conversion failed for trajectory 0 after 5 attempts.

Recommended Optimization

Gate the agent job's startup on the same actor-permission check that checkout_pr_branch.cjs's assertTrustedCheckoutRuntime() already performs internally, instead of starting the full job and failing ~1s into the checkout step.

• Why highest impact: today, pre_activation (~22s) and activation (~30–40s) jobs run to completion, paying real CI/action-minute cost, before the checkout step synchronously rejects the actor and fails closed. Moving (or duplicating, at low cost) the permission check into the job's if: condition — or short-circuiting immediately after pre_activation/activation for actors below the write-permission floor — would convert 7 of the last 24h's failed runs into fast no-ops, and would stop the failure from cascading into the conclusion job's "Handle agent failure" path (which currently also reports failure, doubling the visible red X's for the same root cause).
• Where to implement: actions/setup/js/checkout_pr_branch.cjs (assertTrustedCheckoutRuntime, lines ~268–394) and pkg/workflow/pr.go (generatePRReadyForReviewCheckout / ShouldGeneratePRCheckoutStep), which together control whether and how this step is compiled into each PR-review workflow's lock file.
• Fix type: adding precondition checks before expensive tools.

Validation Plan

• Re-run agentic-workflows logs/audit for design-decision-gate, mattpocock-skills-reviewer, ponytail-reviewer, pr-code-quality-reviewer, and test-quality-sentinel over the next 24h window and confirm the failure-a7e3cc48252fc236 cluster (or its successor signature) no longer appears, or that its driver_exit runs now terminate at/near pre_activation/activation instead of inside agent.
• Expected success metrics: the 7/35 (20%) failure share attributable to this cluster should drop toward 0; the "Handle agent failure" aftermath cluster count should drop from 14 toward 7 (its current non-overlapping remainder); total job-minutes spent on these 5 workflows for untrusted-actor PR events should decrease since pre_activation/activation cost is avoided once gated earlier.
• Separately, re-attempt the AgentRx pipeline once COPILOT_GITHUB_TOKEN/GH_TOKEN (or Azure/TRAPI credentials) are available to the setup step, and confirm ir/static/dynamic/check/judge/report all produce artifacts so future reports can cite judge-classified root causes directly instead of falling back to raw cluster data.

References

• §39709752322 — Test Quality Sentinel, representative "Checkout PR branch" failure
• §39674517281 — Matt Pocock Skills Reviewer, same failure signature
• §39717823226 — gh-aw work-queue run, highest-aic failed run in the 10-run trajectory sample (99.9 AIC)

Generated by ⚡ Daily AgentRx Trace Optimizer · claude · agent · 150.7 AIC · ⌖ 0.786 AIC · ⊞ 8K · ◷

• expires on Oct 28, 2026, 9:51 PM UTC-08:00