Repository navigation
[cli-tools-test] MCP compile: failed dry-run gate hides the underlying compiler warning #67893
Description
Problem Description
The MCP compile tool can return a failed dry-run gate with only compiler reported warnings; review the complete compiler diagnostics. The actual warning is absent from the response, so an MCP-only caller cannot identify or resolve the failure.
Observed on three representative workflows: codex-github-remote-mcp-test, archie, and agent-job-health. Each workflow entry reports valid: true with empty errors and warnings, while the batch compiler result and dry-run gate fail.
Steps to Reproduce
- In
github/gh-aw, invoke the agentic-workflows MCP compile tool with:
{"workflows":["codex-github-remote-mcp-test"],"dry_run":true,"strict":true,"max_tokens":2000}- Inspect the workflow result, the batch compiler result, and the dry-run gate.
- Try to locate the warning responsible for the failed gate in the returned response.
CLI transport used: agenticworkflows compile with the JSON above; no direct gh aw invocation.
Expected Behavior
The failed gate includes the concrete compiler warning text, preferably with workflow attribution and source location, or a supported diagnostics artifact reference. Preserve the failing gate; expose its reason.
Actual Behavior
All three responses contain only the generic compiler-warning message, without the underlying diagnostic or a diagnostics-file reference. Shellcheck reports passed and model inventory is available. No specific failing scanner or model diagnostic explains the warning. The deprecated max_tokens parameter is documented in source as ignored, so this is not intentional response truncation.
Impact
- Severity: Medium — failed validation is not actionable through the MCP interface.
- Frequency: Observed for all three sampled workflows in this session.
- Workaround: None verified through the MCP interface.
- Each targeted dry-run compile took approximately 42 seconds under concurrent execution; this is an observation, not an isolated performance benchmark.
Environment
- Repository:
github/gh-aw - Date: 2026-10-12
- Testing run: §38208175983
- CLI version: not exposed by
status; tested the configured MCP server and current checkout. - Duplicate search was attempted, but candidate issue contents were withheld by integrity policy. Existing duplicate status could not be established.
Diagnostics and source pointers
Relevant response excerpt:
[
{"workflow":"codex-github-remote-mcp-test.md","valid":true,"errors":[],"warnings":[]},
{"scope":"batch","workflow":"compiler","valid":false,"errors":[{"type":"compiler_warning","message":"compiler reported warnings; review the complete compiler diagnostics"}],"warnings":[]},
{"scope":"batch","workflow":"dry-run","valid":false,"errors":[],"warnings":[],"dry_run":{"gate":"failed","compile_only":true,"execution_authorized":false}}
]pkg/cli/compile_development.go, function appendDevelopmentCompilerDiagnostics, emits this generic message for aggregate warnings. In pkg/cli/mcp_tools_readonly.go, the compile handler returns existing JSON on a failing compiler exit; the general diagnostics.failureMessage(err) is used only when stdout is empty, while shellcheck diagnostics receive separate handling. This is a plausible explanation for the missing warning; the underlying warning itself was not recovered.
Next Action
Include concrete aggregate compiler diagnostics in MCP JSON responses and cover the case where nonempty stdout accompanies a warning-induced failed gate.
References:
Generated by 🧪 Daily Cli Tools Tester · codex · gpt60 · 144 AIC · ⌖ 37.8 AIC · ⊞ 19.7K · ◷
- expires on Oct 18, 2026, 9:38 PM UTC-08:00
Activity
github-actions commentedon Oct 19, 2026
This issue was automatically closed because it expired on 2026-10-19T05:38:37.675Z.