Skip to content

[cli-tools-test] MCP compile: failed dry-run gate hides the underlying compiler warning #67893

Closed as not planned

Description

@github-actions

Problem Description

The MCP compile tool can return a failed dry-run gate with only compiler reported warnings; review the complete compiler diagnostics. The actual warning is absent from the response, so an MCP-only caller cannot identify or resolve the failure.

Observed on three representative workflows: codex-github-remote-mcp-test, archie, and agent-job-health. Each workflow entry reports valid: true with empty errors and warnings, while the batch compiler result and dry-run gate fail.

Steps to Reproduce

  1. In github/gh-aw, invoke the agentic-workflows MCP compile tool with:
{"workflows":["codex-github-remote-mcp-test"],"dry_run":true,"strict":true,"max_tokens":2000}
  1. Inspect the workflow result, the batch compiler result, and the dry-run gate.
  2. Try to locate the warning responsible for the failed gate in the returned response.

CLI transport used: agenticworkflows compile with the JSON above; no direct gh aw invocation.

Expected Behavior

The failed gate includes the concrete compiler warning text, preferably with workflow attribution and source location, or a supported diagnostics artifact reference. Preserve the failing gate; expose its reason.

Actual Behavior

All three responses contain only the generic compiler-warning message, without the underlying diagnostic or a diagnostics-file reference. Shellcheck reports passed and model inventory is available. No specific failing scanner or model diagnostic explains the warning. The deprecated max_tokens parameter is documented in source as ignored, so this is not intentional response truncation.

Impact

  • Severity: Medium — failed validation is not actionable through the MCP interface.
  • Frequency: Observed for all three sampled workflows in this session.
  • Workaround: None verified through the MCP interface.
  • Each targeted dry-run compile took approximately 42 seconds under concurrent execution; this is an observation, not an isolated performance benchmark.

Environment

  • Repository: github/gh-aw
  • Date: 2026-10-12
  • Testing run: §38208175983
  • CLI version: not exposed by status; tested the configured MCP server and current checkout.
  • Duplicate search was attempted, but candidate issue contents were withheld by integrity policy. Existing duplicate status could not be established.
Diagnostics and source pointers

Relevant response excerpt:

[
  {"workflow":"codex-github-remote-mcp-test.md","valid":true,"errors":[],"warnings":[]},
  {"scope":"batch","workflow":"compiler","valid":false,"errors":[{"type":"compiler_warning","message":"compiler reported warnings; review the complete compiler diagnostics"}],"warnings":[]},
  {"scope":"batch","workflow":"dry-run","valid":false,"errors":[],"warnings":[],"dry_run":{"gate":"failed","compile_only":true,"execution_authorized":false}}
]

pkg/cli/compile_development.go, function appendDevelopmentCompilerDiagnostics, emits this generic message for aggregate warnings. In pkg/cli/mcp_tools_readonly.go, the compile handler returns existing JSON on a failing compiler exit; the general diagnostics.failureMessage(err) is used only when stdout is empty, while shellcheck diagnostics receive separate handling. This is a plausible explanation for the missing warning; the underlying warning itself was not recovered.

Next Action

Include concrete aggregate compiler diagnostics in MCP JSON responses and cover the case where nonempty stdout accompanies a warning-induced failed gate.

References:

Generated by 🧪 Daily Cli Tools Tester · codex · gpt60 · 144 AIC · ⌖ 37.8 AIC · ⊞ 19.7K · ◷

  • expires on Oct 18, 2026, 9:38 PM UTC-08:00

Activity

  1. github-actions commented on Oct 19, 2026

    @github-actions
    ContributorAuthor

    This issue was automatically closed because it expired on 2026-10-19T05:38:37.675Z.

    Closed by Workflow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions