You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Custom endpoints behind a gateway never get a context length, so Claude assumes 200k #754
On Codeman 1.47.0, a custom model endpoint that is a gateway in front of llama-swap (for example Bifrost) has no context length saved, so CLAUDE_CODE_MAX_CONTEXT_TOKENS is never set. Claude Code then prints Model "..." isn't recognized, context assumed to be 200k tokens and does not compact before a smaller server overflows. Codex, OpenCode and DeepSeek get no limit either.
Cause: modelContextLengths is only filled from llama-swap's /running launch command (and /props), which exists only for a loaded model and only when Codeman talks to llama-swap directly. A gateway has no /running. Discovery never reads the size from GET /v1/models, although current llama-swap sends context_length and context_window there (see llama-swap api.go) and Bifrost passes context_window through (Bifrost openai/models.go). OpenRouter-style servers send context_length.
Expected:
Discovery reads a positive integer context_length, else context_window, from each /v1/models entry. The llama-swap /running value keeps priority for a direct llama-swap endpoint.
An optional per-endpoint defaultContextLength is used for models whose size cannot be discovered, so a CLI never falls back to its own large default. Discovery must never overwrite it.
Regression tests for both deployments (direct llama-swap and a gateway).
Activity
feat(custom-model): discover context length from /v1/models and add a…
feat(custom-model): discover context length from /v1/models and add a…