Skip to content

[agentic-token-optimizer] Package Specification Enforcer: cut ~60-80 AIC/run (context rebuild, model cost) #70539

Closed as not planned

Description

@github-actions

Target workflow

Package Specification Enforcer (.github/workflows/spec-enforcer.md). It is the highest-AIC workflow in the 7-day window that has no optimization in the last 14 days. Higher-ranked entries were excluded: Claude Code User Documentation Review was optimized on 2026-10-09, and Avenger on 2026-10-15.

Analysis period and runs

  • Window: the last 7 days (all-runs.json).
  • Runs analyzed: 1, §39415378641. It succeeded.
  • Prior daily snapshots from 2026-09-14 to 2026-10-02 show about 2–5 AIC per run, and 0 AIC on some days.
  • This run cost roughly 20–50x that baseline.

Cost profile

Metric Value
Total AIC 102.2
Avg AIC/run 102.2
Raw tokens 1,075,698
Model invocations 33
Peak input per invocation 56.5k tokens
Cumulative input 1.066M tokens
Working-set rebuild factor 18.9x
Rebuild excess tokens ~1.01M (94% of total)
Context growth 59.2x
Tool success rate 94.4% (1 failure)
Retries / loops 0 / 0
Duration 10.3 min

The engine and model are Codex with openai/gpt-5.6-sol. Cache efficiency could not be measured from the data. About 94% of the tokens are re-read context, so per-step context size is the main cost driver.

Ranked recommendations

  1. Cut context growth by capping per-package output (about 40–60 AIC/run).

    • Phase 2 Step 3 runs cat pkg/<package>/spec_test.go. It also reads the full README and runs grep ... | head -50. All of these stay in context for every later call.
    • Instead, have the main agent read only grep -n "^func Test" spec_test.go.
    • Pipe go test -v output through tail -n 60 before passing it to test-output-classifier.
    • Evidence: 33 invocations rebuilt a context that grew 59x.
  2. Reduce steps per package (about 15–25 AIC/run).

    • The prompt has many sequential bash calls per package: find, git log, grep, grep package, find spec_test, cat, build, test, and a cache write.
    • Combine them into one && chain per package.
    • Run go build only when go test fails to compile, since go test already compiles.
    • Evidence: 33 steps with a trajectory efficiency of 0.22.
  3. Reconsider the model choice (savings uncertain).

    • Earlier runs cost about 4 AIC. The current model: openai/gpt-5.6-sol pin is a plausible cause of the jump, but this is not verified. The audit data does not show an earlier model.
    • Compare against the previous model or a cheaper tier on 3+ runs before keeping the pin.
  4. Lower max-turns: 100 to about 40 as a runaway guard.

    • The run needed 33 steps, so there is headroom.
    • This is a safety net and not an expected saving.

Structural optimizations

  • Inline sub-agents: the workflow already has two (rotation-state-recoverer, readme-spec-extractor/test-output-classifier references). I recommend no additional ones.
  • Shared setup prefix: none warranted. The sections do not repeat opening calls.

Tool usage

  • The configured tools are bash: "*", edit, cache-memory, cli-proxy and github (default toolsets).
  • The agenticworkflows and github tools did not clearly appear in this run's usage data.
  • I would not remove any tool on one run, because the rule requires at least 5 runs. github toolsets: re-check once more runs are available.

Caveats

  • Only one run is in the window, so the recommendations are low-confidence.
  • A larger package batch (the 2–3 round-robin packages, or enforce_all) probably explains part of the cost.
  • Run §39415378641 is the only evidence cited.

References: §39415378641

Generated by Agentic Workflow AIC Usage Optimizer · copilot · auto · 22.7 AIC · ⊞ 11K · ◷

  • expires on Oct 27, 2026, 7:00 AM UTC-08:00

Activity

  1. github-actions commented on Oct 21, 2026

    @github-actions
    ContributorAuthor

    This issue is being closed as outdated. A newer issue has been created: #70904

    View newer issue


    This action was performed automatically by the Agentic Workflow AIC Usage Optimizer workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions