[cli-tools-test] windows workflow uses deprecated missing-tool key, failing strict dry-run compile gate #71046

Overview

Bounded exploratory testing found that the trusted windows workflow emits a lock but fails the strict dry-run gate because it still declares the deprecated missing-tool safe-output key. Two other selected workflows were rejected by dry-run safety checks; those rejections are coverage limits, not evidence of a compiler defect.

Finding and reproduction

• Repository: github/gh-aw; tested 2026-10-22.
• Source: .github/workflows/windows.md, safe-outputs.missing-tool.
• Tool call: agenticworkflows compile {"workflows":["windows"],"dry_run":true,"strict":true}.
• Expected: a maintained trusted workflow passes strict validation without deprecated declarations.
• Actual: the workflow entry reports valid: true, but the compiler batch reports compiler_warning and the overall dry-run gate is failed. Shellcheck passed. Consumers must inspect batch results, not just the workflow entry.
• Severity: Low; observed once. This blocks using this sample as a successful strict dry-run compile check.
• Suggested correction: migrate the source declaration to report-missing-tool through the normal authoring and validation flow. No source changes or publication were performed during this test.

Coverage and timings

Passed checks: status discovery returned names, engines, triggers, and dependencies in 2.9s. Five logs queries all used count: 3: last-day metadata (3 runs, 6.3s), weekly-issue-summary over seven days (1 run, 7.3s), copilot with a before-run cursor (3 runs, 7.8s), nonexistent workflow (actionable empty response, 0.3s), and future date (empty summary, 1.2s). Explicitly exposed summary files were readable; workflow, engine, timestamps, and count matched filters. Count-limited queries correctly disclosed partial coverage.

Two existing-run audits matched logs IDs, names, conclusions, and durations: §39273204856 succeeded, took 15.4s to audit, and reported tool usage, network counts, job timings, and one created discussion; §39724437444 failed, took 7.8s to audit, and identified agent and conclusion failures. These are existing-run observations, not execution of newly compiled locks. The successful run SHA was 88d48aeea757ac1a3b316aecac1e08cc59b3bbb0; the failed run SHA was 273e7e1773d364c9ca7c400676882f89f35604f8.

Blocked or unfinished required checks: all 3 targeted compile calls used dry_run: true and strict: true, but none passed the overall gate. Windows took 29.0s; Archie 27.6s; Weekly Issue Summary 28.3s (concurrent measurements, above the suggested 5s target; no isolated performance conclusion). Successful compilation and generated-lock validation for all three targets remain unverified. The Windows lock had a frontmatter hash, strict/dry-run metadata, jobs, and staged safe outputs; complete syntax and semantic validation was not independently established.

Not tested (expected exclusion)

• Synthetic malformed YAML, invalid tool configurations, and missing imports: restricted to trusted repository targets.
• Private raw logs, directory structure, and independent artifact completeness: outside session scope.
• Execution of newly compiled locks: dispatch and reruns prohibited.
• Historical and in-progress audit variants: omitted in favor of two representative completed runs.
• Redundant date variants and repeat-compilation hash comparison: optional expansion omitted after the three failed gates.

Other unverified details include independent completeness of tool/network detection, created-resource link validation, memory usage, and sub-agent error attribution. No repository-wide health claim is made.

Next actions

Migrate the deprecated Windows key and rerun the bounded compile check. Select future test targets compatible with dry-run restrictions without weakening those restrictions. Required compile coverage is reported incomplete separately.

References: Tester §39727564924, successful audit §39273204856, failed audit §39724437444.

Generated by 🧪 Daily Cli Tools Tester · codex · gpt60 · 124.2 AIC · ⌖ 39.3 AIC · ⊞ 20.7K · ◷

• expires on Oct 28, 2026, 9:29 PM UTC-08:00