Skip to content

fix(codex): repair cache identity, quota handling, and usage accounting - #27

Merged
dannote merged 1 commit into
masterfrom
fix/codex-session-quota-accounting
Sep 11, 2026
Merged

dannote merged 1 commit into
masterfrom
fix/codex-session-quota-accounting

Conversation

@dannote

@dannote dannote commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Repair the Codex path and usage accounting without changing storage, deployment architecture, or historical counters:

  • Forward stable Chat/Responses prompt_cache_key and metadata session/thread identities, scoped to the authenticated API key.
  • Temporarily pin ReqLLM to dannote/req_llm@69f488da53fdd81452092cbaa5c05cdd4799f24b for the upstream Codex wire compatibility fixes: fix(codex): preserve session and prompt cache identity across transports agentjido/req_llm#996.
  • Preserve outer WebSocket quota status instead of discarding it while extracting the nested error. Recognize status-less quota codes and use future resets_at values for model-token cooldown/retry delay.
  • Normalize OpenAI cached input once before accounting; preserve total-input wire semantics when rendering internal usage structs. Raw ReqLLM usage maps are not incremented again.
  • Fix the provider-usage HTTP builder's incompatible combination of :finch and :connect_options, which raised before network dispatch.
  • Update protocol, token-pool, accounting, and provider regression tests and document the temporary pin and historical accounting boundary.

The required Igniter 0.8.4 security update follows the fresh upstream ReqLLM dependency constraint. No historical rows or lifetime counters are rewritten.

Verification

  • MIX_ENV=test mix ci: passed, including 504 tests (9 excluded), compile, formatting, strict Credo, Dialyzer, duplication budget, and architecture checks.
  • Quota regression test demonstrates that a wrapped 429 persists a model cooldown until the supplied reset time.
  • Deployed the exact commit through HostKit's build/drain/migration/readiness lifecycle, retaining the previous release for rollback.
  • Eight pure post-release checks passed.
  • Three minimal live calls succeeded: Astra buffered Chat, Astra streaming Chat, Sol streaming Responses. All returned HTTP 200 with terminal events and usage (61 total reported tokens).
  • Real application traffic completed successfully with separately recorded cached/uncached input.
  • Previously broken provider usage refresh now returns a fresh weekly snapshot.

Follow-up

Replace the temporary Git dependency once the upstream compatibility fix is released and validated. Session-aware connection reuse and turn-state propagation are separate work; this change does not introduce cross-request WebSocket sharing. Improved proxy accounting does not establish that private provider quota consumption has improved.

@dannote
dannote merged commit 35b0f6a into master Sep 11, 2026
dannote added a commit to mikehostetler/llm_proxy that referenced this pull request Sep 29, 2026
0.2.1 already used ReqLLM from Hex, so the temporary Git pin from elixir-vibe#27 and its
removal are not release-visible. Record the Codex fixes from elixir-vibe#27 that landed
after 0.2.1, and fold stream error projection into the streaming entry since
ReqLLM streaming did not exist before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant