Model routing: accept none/max for pinned Codex and add routed Codex smoke

.github/workflows/smoke-codex-routed.md

@@ -0,0 +1,85 @@
+---
+private: true
+emoji: "🧭"
+description: Smoke Codex model routing with deterministic runner and proxy evidence checks
+on:
+  schedule: every 2 days
+  workflow_dispatch:
+  label_command:
+    names: ["smoke-routing", "smoke"]
+    events: [pull_request]
+    strategy: decentralized
+  bots: [Copilot]
+  status-comment: false
+  github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
+permissions:
+  contents: read
+  copilot-requests: write
+name: Smoke Codex Routed
+concurrency:
+  group: "smoke-codex-routed-${{ github.event_name == 'pull_request' && format('pr-{0}', github.event.pull_request.number) || github.run_id }}"
+  cancel-in-progress: true
+  job-discriminator: ${{ github.run_id }}
+strict: true
+engine:
+  id: codex
+  model-provider: github
+  model-routing:
+    goal: cost
+    mode: robust
+    allowed-models: [gpt-5.6-luna, gpt-5.6-terra, gpt-5.6-sol]
+imports:
+  - shared/smoke-test-brevity.md
+  - shared/reporting.md
+checkout: false
+tools:
+  github: false
+safe-outputs:
+  create-issue:
+    expires: 2h
+    group: true
+    close-older-issues: true
+    close-older-key: "smoke-codex-routed"
+    labels: [automation, testing]
+timeout-minutes: 10
+features:
+  gh-aw-detection: false
+sandbox:
+  agent:
+    id: awf
+post-steps:
+  - name: Assert model-routing evidence
+    if: always()
+    uses: actions/github-script@v9.0.0
+    env:
+      SMOKE_EXECUTION: ${{ steps.agentic_execution.outcome }}
+    with:
+      script: |
+        const path = require("node:path");
+        const { main } = require(path.join(process.env.RUNNER_TEMP, "gh-aw/actions/smoke_model_routing_assertions.cjs"));
+        await main({
+          core,
+          engine: "codex",
+          executionOutcome: process.env.SMOKE_EXECUTION,
+          allowedModels: ["gpt-5.6-luna", "gpt-5.6-terra", "gpt-5.6-sol"],
+          mainEndpoint: "/responses",
+          expectSelectedEffort: "max",
+          expectRequestedEffortMatchesSelection: true,
+        });
+---
+
+# Smoke Test: Codex Model Routing
+
+This run checks model routing, not the task. Keep it trivial and short.
+
+## Task
+
+Do not inspect the repository or run commands.
+
+## Output
+
+Create an issue titled **"Smoke Test: Codex Routed - ${{ github.run_id }}"** with:
+- ✅ for the task above
+- Run URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+
+The routing verdict comes from the post-step, which checks runner- and proxy-written evidence, not this report. It fails (H2) when the proxy selected an effort other than `max`, so a run that did not exercise `max` is never counted as coverage.

DEVGUIDE.md

@@ -984,16 +984,17 @@ This documentation provides:
    - Run the `pr-finisher` skill (automates final review/check/mergeability hardening) to get to green
    - For features that deeply impact the engine, add the `smoke` label and approve workflows
    - If no smoke run is queued after setting `smoke`, or additional changes require another smoke run, toggle the `smoke` label (remove and re-add), then approve workflows again
-   - For changes to model routing (`engine.model-routing`), add the `smoke-routing` label and approve workflows. It runs eight routed smokes on the PR's code:
+   - For changes to model routing (`engine.model-routing`), add the `smoke-routing` label and approve workflows. It runs nine routed smokes on the PR's code:
      - `smoke-copilot-routed`: Copilot CLI with routing and no sub-agents. It also runs the default threat-detection engine on a pinned model, and a detection-job post-step checks that detection completed on that model without routing (`D1`–`D3`)
      - `smoke-claude-routed`: Claude Code with routing on `model-provider: github`; AWF selects `/chat/completions` and the main agent must use `/v1/messages`
      - `smoke-pi-routed`: pi with routing and two cross-family declared sub-agents (Claude and GPT)
+     - `smoke-codex-routed`: Codex with routing on `model-provider: github` over `gpt-5.6-*` candidates; the main agent must use `/responses`, `H2` fails unless the proxy-selected effort is `max`, and `H3` fails unless every main-agent request the proxy recorded (`firewall.model_routing` `stage: request`) has `requestedEffort` equal to the selected effort
      - `smoke-copilot-sdk-routed`: Copilot SDK with routing and a Claude sub-agent under a GPT main agent
      - `smoke-copilot-routed-steering`: Copilot CLI with routing on one model and a 7-credit `max-ai-credits` budget; the main agent must stop on the AWF AI-credit steering notice and finish under budget
      - `smoke-copilot-routed-claude`: Copilot CLI with routing over a Claude-only pool in `mode: auto`; the main agent must use `/chat/completions`
      - `smoke-pi-routed-claude`: pi with routing over a Claude-only pool in `mode: robust`; the main agent must use `/v1/messages`
      - `smoke-copilot-sdk-routed-claude`: Copilot SDK with routing over `claude-haiku-4.5` (no reasoning effort) and a GPT sub-agent on `/responses`
-   - Each routed smoke's post-step (`actions/setup/js/smoke_model_routing_assertions.cjs`) checks runner- and proxy-written evidence, not the agent's reply. Every failed check logs its ID (`R1`–`R4`, `S1`–`S4`, `M1`, `P1`, `E1`, `E2`, `U1`–`U3`, and `T1`–`T4` for the steering smoke) with the observed model, endpoint, status, event or steering notice, so the step log is enough to diagnose a failure
+   - Each routed smoke's post-step (`actions/setup/js/smoke_model_routing_assertions.cjs`) checks runner- and proxy-written evidence, not the agent's reply. Every failed check logs its ID (`R1`–`R4`, `S1`–`S4`, `M1`, `P1`, `E1`, `E2`, `U1`–`U3`, `H2` for the Codex smoke, and `T1`–`T4` for the steering smoke) with the observed model, endpoint, status, event or steering notice, so the step log is enough to diagnose a failure
    - `U1`–`U3` check the unified session (`usage/aw_session.jsonl`) that audit, logs and footers read: `U1` no `session.collection_warning`; `U2` exactly one agent-phase `model.inventory` (with `copilotInventory`, a configured `copilot` endpoint with models); `U3` one successful `model.resolution` for each role in the smoke's explicit `resolutionRoles`. The unrouted engine smokes (`smoke-copilot`, `smoke-copilot-sdk`, `smoke-claude`, `smoke-claude-on-copilot`, `smoke-codex`, `smoke-pi`) run `E2` and `U1`–`U3` alone through `sessionContractMain`
    - A routed smoke opts in to `U1`–`U3` by setting `expectModelInventory: true`, `copilotInventory: true` or a `resolutionRoles` array (`[]` requires no roles); `false` flags do not opt in. `U2` runs when `expectModelInventory` or `copilotInventory` is true. If neither is true, the log explicitly reports `U2 not evaluated`. With `expectModelInventory: true`, missing inventory evidence fails gracefully as `FAIL U2 ... observed 0`; it does not throw. A missing session instead fails `E2` and logs `U1–U3 not evaluated`. Unavailable evidence paths also fail `E2`. Repeated `U1` warnings are grouped by `(code, path)` with a count and an example line when available; every distinct pair remains visible
    - `label_command` removes the `smoke-routing` label when the runs start. To run them again, re-add the label, then approve workflows again

actions/setup/js/awf_model_routing.cjs

@@ -45,7 +45,11 @@ function mapAWFRoutingEffort(engine, effort) {
   if (engine === "copilot") return { effort, error: null };
   const supported = {
     claude: new Set(["low", "medium", "high", "xhigh", "max"]),
-    codex: new Set(["minimal", "low", "medium", "high", "xhigh"]),
+    // Reviewed for the pinned Codex 0.164.0 (DefaultCodexVersion): its ReasoningEffort accepts every
+    // routing-contract value unchanged (openai/codex rust-v0.164.0 codex-rs/protocol/src/openai_models.rs).
+    // Per-model eligibility is AWF's candidate catalogue, not this client-capability list.
+    // Re-review on every Codex pin bump (see awf_model_routing.test.cjs).
+    codex: new Set(["none", "minimal", "low", "medium", "high", "xhigh", "max"]),
     pi: new Set(ROUTING_REASONING_EFFORTS),
   }[engine];
   if (engine === "pi" && effort === "none") return { effort: "off", error: null };

actions/setup/js/awf_model_routing.test.cjs

@@ -5,7 +5,8 @@ import os from "os";
 import path from "path";
 
 const require = createRequire(import.meta.url);
-const { ROUTING_REASONING_EFFORTS, mapAWFRoutingEffort, recordAWFModelRoutingOutcome, resolveAWFModelRoutingSelection } = require("./awf_model_routing.cjs");
+const { ROUTING_REASONING_EFFORTS, getAWFModelRoutingFailureCode, mapAWFRoutingEffort, recordAWFModelRoutingOutcome, resolveAWFModelRoutingSelection } = require("./awf_model_routing.cjs");
+const reflectFixtures = require("./fixtures/model_resolution_reflect.cjs");
 
 const reflectData = {
   endpoints: [{ provider: "github", configured: true, models: ["claude-opus-5", "gpt-5.6-sol"] }],
@@ -167,12 +168,49 @@ describe("awf_model_routing.cjs", () => {
   it.each([
     ["claude", "none"],
     ["claude", "minimal"],
-    ["codex", "none"],
-    ["codex", "max"],
   ])("rejects unsupported %s effort %s", (engine, effort) => {
     expect(mapAWFRoutingEffort(engine, effort).error).toContain(`"${effort}"`);
   });
 
+  it.each(["none", "max"])("maps a selected Codex %s effort unchanged", effort => {
+    // Derived fixture: recorded /reflect with an added selected routing block.
+    const selected = resolveAWFModelRoutingSelection(reflectFixtures.withRoutingSelection("gpt-5.6-luna", effort), true, ["/responses"], true);
+    expect(selected.error).toBeNull();
+    expect(selected.selection).toMatchObject({ wire_model: "gpt-5.6-luna", effort, endpoint: "/responses" });
+    expect(mapAWFRoutingEffort("codex", selected.selection.effort)).toEqual({ effort, error: null });
+  });
+
+  it("codex_routing_effort_contract_requires_pin_review", () => {
+    // Reviewed pair: Codex 0.164.0 accepts all seven routing-contract efforts unchanged
+    // (openai/codex rust-v0.164.0 codex-rs/protocol/src/openai_models.rs). Both values are
+    // hard-coded on purpose; a pin bump must re-review the Codex entry in mapAWFRoutingEffort.
+    const reviewedCodexVersion = "0.164.0";
+    const reviewedCodexEfforts = ["none", "minimal", "low", "medium", "high", "xhigh", "max"];
+    const versionConstants = fs.readFileSync(new URL("../../../pkg/constants/version_constants.go", import.meta.url), "utf8");
+    const pinned = versionConstants.match(/^const DefaultCodexVersion Version = "([^"]+)"/m)?.[1];
+    expect(
+      pinned,
+      `Review required: DefaultCodexVersion is ${pinned}, but the Codex routing effort list in mapAWFRoutingEffort was reviewed for ${reviewedCodexVersion}. Check the new Codex ReasoningEffort parser, then update the list and this test together.`
+    ).toBe(reviewedCodexVersion);
+    expect(reviewedCodexEfforts.filter(effort => mapAWFRoutingEffort("codex", effort).error === null)).toEqual(reviewedCodexEfforts);
+    for (const effort of reviewedCodexEfforts) expect(mapAWFRoutingEffort("codex", effort)).toEqual({ effort, error: null });
+    for (const effort of ["ultra", "persistent", "off", "custom"]) expect(mapAWFRoutingEffort("codex", effort).error).toContain(`"${effort}"`);
+  });
+
+  it("non_codex_routing_effort_mappings_are_unchanged", () => {
+    const contract = ["none", "minimal", "low", "medium", "high", "xhigh", "max"];
+    const claude = ["low", "medium", "high", "xhigh", "max"];
+    for (const effort of contract) {
+      expect(mapAWFRoutingEffort("claude", effort)).toEqual(claude.includes(effort) ? { effort, error: null } : { effort: null, error: `AWF model routing effort "${effort}" is not supported by the claude engine` });
+      expect(mapAWFRoutingEffort("pi", effort)).toEqual({ effort: effort === "none" ? "off" : effort, error: null });
+      expect(mapAWFRoutingEffort("copilot", effort)).toEqual({ effort, error: null });
+    }
+    for (const engine of ["claude", "codex", "pi", "copilot"]) expect(mapAWFRoutingEffort(engine, null)).toEqual({ effort: null, error: null });
+    expect(resolveAWFModelRoutingSelection(reflectFixtures.withRoutingSelection("gpt-5.6-luna", null), true, ["/responses"], true).selection?.effort).toBeNull();
+    expect(resolveAWFModelRoutingSelection(reflectFixtures.withRoutingSelection("gpt-5.6-luna", "ultra"), true, ["/responses"], true).error).toBe("AWF /reflect returned unsupported reasoning effort ultra");
+    expect(getAWFModelRoutingFailureCode("AWF /reflect returned unsupported reasoning effort ultra")).toBe("invalid_selection");
+  });
+
   it.each([
     [null, "required model-routing selection"],
     [{ routing: { status: "pending" } }, "pending"],

actions/setup/js/codex_harness.cjs

@@ -528,6 +528,18 @@ async function resolveCodexConfiguredModel(options) {
   return result.engineModel;
 }
 
+/**
+ * Classify a Codex routing selection error. An effort outside the routing contract
+ * (for example Codex-native "ultra") is rejected by the shared selection validator
+ * before the Codex effort mapper runs; report it as unsupported_effort, like a mapper
+ * rejection. Every other error keeps the shared classification.
+ * @param {string|null|undefined} error
+ * @returns {string}
+ */
+function getCodexRoutingFailureCode(error) {
+  return /^AWF \/reflect returned unsupported reasoning effort /.test(error || "") ? "unsupported_effort" : getAWFModelRoutingFailureCode(error);
+}
+
 /**
  * Resolve and apply AWF's selection to a Codex invocation.
  * @param {any} reflectData
@@ -539,7 +551,7 @@ function resolveCodexModelRouting(reflectData, args) {
   if (result.error || !result.selection) {
     recordAWFModelRoutingOutcome({
       status: reflectData?.routing?.status === "failed" ? "failed" : "rejected",
-      failure_code: reflectData?.routing?.failure_code || getAWFModelRoutingFailureCode(result.error),
+      failure_code: reflectData?.routing?.failure_code || getCodexRoutingFailureCode(result.error),
       detail: result.error,
     });
     return { selection: null, model: "", args, error: result.error || "AWF model routing selection is missing" };
@@ -1375,6 +1387,7 @@ if (typeof module !== "undefined" && module.exports) {
     removeCodexRoutingOverrides,
     applyCodexRoutingEffort,
     resolveCodexModelRouting,
+    getCodexRoutingFailureCode,
     getCodexModelEnvVar,
     resolvePostResultWatchdogIdleTimeoutMs,
     createMCPCallWatchdog,

actions/setup/js/codex_harness.test.cjs

@@ -52,6 +52,7 @@ const {
   removeCodexRoutingOverrides,
   applyCodexRoutingEffort,
   resolveCodexModelRouting,
+  getCodexRoutingFailureCode,
   resolveCodexConfiguredModel,
 } = require("./codex_harness.cjs");
 const reflectFixtures = require("./fixtures/model_resolution_reflect.cjs");
@@ -324,12 +325,57 @@ describe("codex_harness.cjs", () => {
       expect(applyCodexRoutingEffort(["exec", "--json"], "high")).toEqual(["exec", "-c", 'model_reasoning_effort="high"', "--json"]);
     });
 
-    it("rejects an effort that Codex does not support", () => {
-      const reflect = {
-        ...reflectData,
-        routing: { ...reflectData.routing, selection: { ...reflectData.routing.selection, effort: "max" } },
-      };
-      expect(resolveCodexModelRouting(reflect, ["exec"]).error).toContain('"max"');
+    // Derived fixtures: the recorded /reflect from run 38945917145 (routing: null) with an
+    // added selected routing block for gpt-5.6-luna, which advertises none..max there.
+    const runRoutedHarness = effort => {
+      const dir = makeHarnessTempDir("routed-effort-");
+      const { result, calls } = runHarnessFixture("process.exit(0);", {
+        args: ["--model", "fixed-model", "-c", 'model_reasoning_effort="low"'],
+        env: { GH_AW_MODEL_ROUTING: "1", GH_AW_MODEL_AGENT_CODEX: "fixed-model", GH_AW_LLM_PROVIDER: "github", GH_AW_TMP_DIR: dir },
+        reflectData: reflectFixtures.withRoutingSelection("gpt-5.6-luna", effort),
+      });
+      const outcomePath = path.join(dir, "agent", "awf-routing-outcome.json");
+      const outcome = fs.existsSync(outcomePath) ? JSON.parse(fs.readFileSync(outcomePath, "utf8")) : null;
+      return { result, calls, outcome };
+    };
+
+    it.each([
+      ["routed_codex_max_starts_with_selected_effort", "max"],
+      ["routed_codex_none_starts_with_selected_effort", "none"],
+    ])("%s", (_name, effort) => {
+      const direct = resolveCodexModelRouting(reflectFixtures.withRoutingSelection("gpt-5.6-luna", effort), ["exec", "--model", "fixed-model", "-c", 'model_reasoning_effort="low"', "--json", "-"]);
+      expect(direct.error).toBeNull();
+      expect(direct.args).toEqual(["exec", "-c", `model_reasoning_effort="${effort}"`, "--model", "gpt-5.6-luna", "--json", "-"]);
+
+      const { result, calls, outcome } = runRoutedHarness(effort);
+      expect(result.status, result.stderr).toBe(0);
+      expect(result.stderr).not.toContain("unsupported_effort");
+      expect(result.stderr).toContain(`inference routing: mode=awf-routed model=gpt-5.6-luna effort=${effort} endpoint=/responses`);
+      expect(calls).toHaveLength(1);
+      const efforts = calls[0].args.filter(arg => arg.startsWith("model_reasoning_effort="));
+      expect(efforts).toEqual([`model_reasoning_effort="${effort}"`]);
+      expect(calls[0].args).toContain("gpt-5.6-luna");
+      expect(calls[0].args).not.toContain("fixed-model");
+      expect(outcome).toMatchObject({ status: "selected", wire_model: "gpt-5.6-luna", endpoint: "/responses", effort, applied_effort: effort });
+      expect(outcome).not.toHaveProperty("failure_code");
+    });
+
+    it.each([
+      ["AWF /reflect returned unsupported reasoning effort ultra", "unsupported_effort"],
+      ["AWF model routing selection has none compatible endpoints", "unsupported_endpoint"],
+    ])("classifies Codex routing failure %s", (error, expectedCode) => {
+      expect(getCodexRoutingFailureCode(error)).toBe(expectedCode);
+    });
+
+    it("routed_codex_out_of_contract_effort_is_rejected", () => {
+      const direct = resolveCodexModelRouting(reflectFixtures.withRoutingSelection("gpt-5.6-luna", "ultra"), ["exec"]);
+      expect(direct.error).toBe("AWF /reflect returned unsupported reasoning effort ultra");
+
+      const { result, calls, outcome } = runRoutedHarness("ultra");
+      expect(result.status).not.toBe(0);
+      expect(result.stderr).toContain("unsupported reasoning effort ultra; refusing to start Codex");
+      expect(calls).toHaveLength(0);
+      expect(outcome).toMatchObject({ status: "rejected", failure_code: "unsupported_effort", detail: "AWF /reflect returned unsupported reasoning effort ultra" });
     });
 
     it("starts Codex with the routed model and effort from AWF reflection", () => {

actions/setup/js/engine_subagent_models.test.cjs

@@ -598,16 +598,17 @@ describe("Codex sub-agent models (daily-arxiv-researcher run 39394965127)", () =
     expect(run.logs.join("\n")).not.toMatch(/not applied/);
   });
 
-  it("fails closed for an alias effort Codex does not accept", async () => {
+  it("applies an alias effort Codex accepts", async () => {
     const run = codexLayout();
     run.env.GH_AW_MODEL_ALIASES = JSON.stringify({ fast: ["openai/gpt-5.4-mini?effort=none"] });
     stageInline(run, "codex", { "paper-screener": arxivAgents()["paper-screener"].replace("model: openai/gpt-5.4-mini", "model: fast") });
 
     const outcome = await apply(run);
 
-    expect(outcome.ok).toBe(false);
-    expect(run.logs).toContainEqual(expect.stringMatching(/codex sub-agent 'paper-screener': model 'fast' resolves with effort 'none': .*not supported by the codex engine.*Refusing to start Codex/));
-    expect(roleFiles(run)).toEqual([]);
+    expect(outcome.ok).toBe(true);
+    expect(outcome.resolutions).toEqual([{ name: "paper-screener", configuredModel: "fast", model: "gpt-5.4-mini", status: "resolved", params: { effort: "none" } }]);
+    expect(readRole(run, "paper-screener")).toMatchObject({ model: "gpt-5.4-mini", model_reasoning_effort: "none" });
+    expect(run.logs.join("\n")).not.toMatch(/not applied/);
   });
 
   it.each([