Repository navigation
feat(custom-model): endpoint type selector for OpenAI or Anthropic protocol - #756
opticon454 wants to merge 16 commits into
Conversation
|
Thank you @opticon454 for splitting this out of #752. This PR adds an endpoint type to saved custom model endpoints so Claude Code, OpenCode, pi and omp can talk to a gateway's Anthropic Messages route, while the CLIs that cannot speak it refuse it with a clear message. The design question you raised (whether to hardcode the One thing needs fixing whichever way that goes: 1. pi and omp get a doubled
The command was Smaller items for the same revision:
CI is green, and typecheck, lint, format and the custom-model and CLI registry test files all pass locally. Once the URL question is answered, items 1 to 4 against that answer are what this needs before merge. |
|
Pushed dd83f0d, thanks for running real pi. Your result is right: the Anthropic SDK appends Before the review items, one scope change from the owner: the Anthropic endpoint type is now Claude Code only. The endpoints this tool points at are local models, never Claude models, so OpenCode, pi and omp have no Anthropic path any more and always use the OpenAI
Full |
|
Thank you @opticon454 for the quick revision. This PR adds an Round one is covered: the doubled Two points on this PR are open, and the code and docs should not record either as settled until they are:
One change applies whichever way these go: the code comments, docs and one test describe the Claude Code only rule as an "owner decision" ( Smaller items for the same revision:
Once both open points are settled, what remains is aligning the code, help text and docs with that answer; the rest of this revision is in good shape. |
…d trim the diff Third review round on PR Ark0N#756 (mechanical items): replace every 'owner decision' attribution for the Claude Code only rule with the reason itself (local models never speak a Claude-specific protocol); drop the 'remains under discussion' sentence from architecture-invariants.md; revert the opencode-json case to its exact upstream shape; delete the no-npm-override test for a field nothing sets. Open questions (/anthropic path form, scope, help text) untouched.
|
Pushed d66f4a7, thanks. The two open points stay open and the code does not record either as settled: I have not touched the The mechanical items are in: the "owner decision" wording is gone from the code, docs and test and the reason is stated instead (the endpoints serve local models, which never speak a Claude-specific protocol), the "remains under discussion" sentence is deleted, the Custom-model, CLI registry and render tests pass (506), plus typecheck, lint and format. |
|
Thank you @opticon454 for the quick turnaround on round two. This PR adds an Everything from my last comment is in: the "owner decision" wording is gone, the "remains under discussion" sentence is deleted, the Both design points are still open on my side, and the code and docs should keep not recording either as settled:
Two facts that bear on them: under the second form of point 1, the Anthropic option would give Claude Code the same If the PR stays, these apply whichever way the points are answered:
Once the help text is settled, the user manual needs the dropdown too: step 1 of "Adding an endpoint" in The two open points come first. Once they are answered, what remains is aligning the code, help text and docs with that answer, plus the items above. |
|
Pushed 1214acb, thanks. The two design points stay open and I have not touched the
The wiki dropdown docs wait on the help text, as you said. Full |
|
Thank you @opticon454 for round three. This PR adds an Round two is in: the scope is stated as code facts, DeepSeek sits with the env recipes, Both design points are still open on my side, unchanged from my last comment:
One more fact that bears on both. Discovery calls
So the two types give Claude Code a different URL only when the saved base is the bare gateway root, and the Small items for the same revision, whichever way the points go:
The two open points come first. Once they are answered, what remains is aligning the code, help text, docs and the wiki pages ( |
…thfully word the recovery warning Review round on PR Ark0N#756, small items 1 and 2. - The endpointType doc comment dropped the "local models are never Claude models" rationale: the code fact is that only claude's recipe declares supportsAnthropic and every other recipe returns unsupported for an Anthropic-type endpoint - customModelApplyErrorMessage moved from session-routes.ts next to applyCustomModelInjection (its natural home) and exported, so the two route sites and the recovery warning share one wording - The recovery warning no longer claims "is Anthropic-type" for every undefined from applyCustomModelInjection (an unsupported recipe, e.g. after a clis.json override, warned the same lie); it picks the sentence from endpoint.endpointType like the routes do - New test/custom-model-recovery-warning.test.ts pins both shapes: an OpenAI-type endpoint with antigravity's unsupported recipe warns without "Anthropic-type"; an Anthropic-type endpoint warns with it
|
Pushed 2278764. The two design points stay open and I have not touched the path handling, the help text or the discovery sentences. The table was useful: it shows the two types differ for Claude Code only when the saved base is the bare gateway root, which fits your reading that the two points are close to one question. I will come back on whether this PR stays.
Full |
|
Thank you @opticon454 for round four. This PR adds an Round three is in: the Both design points are still open on my side, unchanged:
Please keep holding off on the path handling, the help text at If the PR stays, two small items apply whichever way the points are answered:
The two open points come first. Once they are answered, what remains is aligning the code, help text, docs and the wiki pages ( |
…a-swap checks Review round on PR Ark0N#756, both small items. - The swap route and the quick-start customModel path now run buildCustomModelInjection right after the endpoint lookup, next to the existing unsupported-recipe check, and return the shared customModelApplyErrorMessage when the kind is unsupported. A non-claude CLI pointing at an Anthropic-type endpoint never reaches the GET /running probe or a requiresConfirmation answer, so the caller is no longer asked to approve evicting another session's model for an apply that cannot happen (API-only path; the Run menu already hides the rows) - Route tests in both files pair the refusal with a /running mock that WOULD ask for confirmation (ready model + conflicting session) and pin that /running was never fetched - test/custom-model-recovery-warning.test.ts stops the WebServer it constructs, in line with every other test that builds one
|
Pushed e5fa234, thanks. Both design points stay as they were; I have not touched the path handling, the help text, the discovery sentence or the wiki pages, and I will answer your question about whether this PR stays separately.
Full |
|
Thank you @opticon454 for round five. This PR adds an Round four is in:
Typecheck, lint, format, the frontend syntax check, the browser-exclude check, the CLI catalogue check, the custom-model, render and route test files and FULL_GATE_DRAFT pass here. Both design points are still open on my side, unchanged:
Nothing else is needed on the branch until these are answered, so please keep holding off on the path handling, the help text at Once the two points are answered, what remains is aligning the code, help text, docs and the wiki pages ( |
…sed run-menu rows, share the refusal wording, document endpointType - renderConfigFile now hands the pi-ai lineage (pi, omp) the same withAnthropicRootSuffix base Claude Code gets; their SDK appends /v1/messages itself, so the previous /anthropic/v1 form doubled the /v1 (live-confirmed against real pi). OpenCode alone keeps withAnthropicV1Suffix; the doc comments are corrected to match. - server.ts publishes supportsAnthropic per CLI on window.__codemanCustomModelClis, computed by running buildCustomModelInjection against a synthetic Anthropic probe endpoint (never an id list); session-ui.js skips those rows for an Anthropic-type endpoint. - The duplicated apply-time refusal is one shared helper with the wording the review asked for, and both routes gained an app.inject refusal test (codex against an Anthropic-type endpoint). - endpointType documented in api-reference POST/PUT, custom-model-endpoints Adding an endpoint, and architecture-invariants Custom Model Endpoint Profiles. The /anthropic path-segment design question stays untouched.
…only Owner decision: local models are never Claude models. OpenCode, pi and omp lose their anthropic recipe branches (no @ai-sdk/anthropic override, no anthropic-messages api, no /anthropic/v1 base helper); every configDir and configContentEnv recipe refuses an Anthropic-type endpoint, env recipes still gate on supportsAnthropic (claude's alone). withAnthropicRootSuffix and claude's behaviour are untouched. Run-menu flag and shared refusal message stay as they were.
…d trim the diff Third review round on PR Ark0N#756 (mechanical items): replace every 'owner decision' attribution for the Claude Code only rule with the reason itself (local models never speak a Claude-specific protocol); drop the 'remains under discussion' sentence from architecture-invariants.md; revert the opencode-json case to its exact upstream shape; delete the no-npm-override test for a field nothing sets. Open questions (/anthropic path form, scope, help text) untouched.
…hen recovery refuses
…thfully word the recovery warning Review round on PR Ark0N#756, small items 1 and 2. - The endpointType doc comment dropped the "local models are never Claude models" rationale: the code fact is that only claude's recipe declares supportsAnthropic and every other recipe returns unsupported for an Anthropic-type endpoint - customModelApplyErrorMessage moved from session-routes.ts next to applyCustomModelInjection (its natural home) and exported, so the two route sites and the recovery warning share one wording - The recovery warning no longer claims "is Anthropic-type" for every undefined from applyCustomModelInjection (an unsupported recipe, e.g. after a clis.json override, warned the same lie); it picks the sentence from endpoint.endpointType like the routes do - New test/custom-model-recovery-warning.test.ts pins both shapes: an OpenAI-type endpoint with antigravity's unsupported recipe warns without "Anthropic-type"; an Anthropic-type endpoint warns with it
…a-swap checks Review round on PR Ark0N#756, both small items. - The swap route and the quick-start customModel path now run buildCustomModelInjection right after the endpoint lookup, next to the existing unsupported-recipe check, and return the shared customModelApplyErrorMessage when the kind is unsupported. A non-claude CLI pointing at an Anthropic-type endpoint never reaches the GET /running probe or a requiresConfirmation answer, so the caller is no longer asked to approve evicting another session's model for an apply that cannot happen (API-only path; the Run menu already hides the rows) - Route tests in both files pair the refusal with a /running mock that WOULD ask for confirmation (ready model + conflicting session) and pin that /running was never fetched - test/custom-model-recovery-warning.test.ts stops the WebServer it constructs, in line with every other test that builds one
|
Rebased onto 1.48.0 ( Typecheck, lint and format are clean. Full |
e5fa234 to
bc20c7d
Compare
|
Thank you @opticon454 for the rebase onto 1.48.0. This PR adds an The conflict resolution is right in the code: boot recovery keeps its warning on top of the new The rebase left a few things behind:
Both design points are still open on my side, unchanged:
Please keep holding off on the path handling, the help text at The two open points come first. Once they are answered, what remains is aligning the code, help text, docs and wiki pages with that answer, plus the items above. |
…ents and rewrap the endpoint docs after the rebase
|
Pushed 5da33ef, thanks. All the leftovers from the rebase are done:
Typecheck, lint and format are clean, and the six custom-model test files pass (220 tests). The two design points are untouched, and I have not touched the path handling, help text, discovery sentence or wiki pages. |
|
Thank you @opticon454 for clearing up the rebase leftovers. This PR adds an All of the last round is in: DeepSeek sits in the Both design points are still open on my side, unchanged:
Please keep holding off on the path handling, the help text at One fact bears on both points, and you have the gateway to check it: does If the PR stays, these small items apply whichever way the points are answered:
The two open points come first. Once they are answered, what remains is aligning the code, help text, docs and wiki pages with that answer, plus the items above. |
|
Thank you @opticon454 for the latest revision. This PR adds an All of the last round is in: the PUT reset is stated in Both design points are still open on my side, unchanged:
Please keep holding off on the path handling, the help text at On the question from my last comment, Bifrost's source at
You have the gateway, so please run If the PR stays, these small items apply whichever way the points are answered:
If The two open points come first. Once they are answered, what remains is aligning the code, help text, docs and wiki pages with that answer, plus the items above. |
… tidy a comment and a doc line
|
Pushed 0bf45b7, thanks. The three small items are in:
On the gateway question: I have not been able to get a successful |
|
Thank you @opticon454 for this revision. This PR adds an All of the last round is in: Both design points are still open on my side, unchanged:
One measurement for both: Claude Code 2.1.318 against a local logging server sent Please keep holding off on the path handling, the help text at If the PR stays, these small items apply whichever way the points are answered:
If The two open points come first. Once they are answered, what remains is aligning the code, help text, docs and wiki pages with that answer, plus the items above. |
…how endpoint type in list
|
Applied the three small items in
Typecheck, format and the custom-model route, editor, run-menu and recovery test files pass. I have not run the full suite. The two design points and the |
|
Thank you @opticon454 for this revision. This PR adds an All of the last round is in: both routes refuse the mismatch on the existing build result with the extra build gone, an Anthropic-type endpoint is marked on its list row, and Both design points are still open on my side, unchanged:
Please keep holding off on the path handling, the help text at Two things are still outstanding from your side: the If the PR stays, these small items apply, and none of them needs doing before the two points are answered:
Nothing else is needed on the branch until the two points are answered. |
…mments, rewrap doc
|
Applied items 2 to 4 in
Item 1 ( Both design points and the gateway comparison are still open on my side. Bifrost still returns 502 on the listing routes while one of its providers is down. |
|
Thank you @opticon454 for this revision. This PR adds an Items 2 and 3 from my last comment are in: One leftover from item 4: Both design points are still open on my side, unchanged:
Please keep holding off on the path handling, the help text at Two things are still outstanding from your side: the If the PR stays, these small items apply, and none of them needs doing before the two points are answered:
Nothing else is needed on the branch until the two points are answered. |
|
Done in
Items 1 and 3 stay on hold with the two design points. The Bifrost listing comparison is still blocked by the 502 from a provider behind it. |
|
Thank you @opticon454 for this revision. This PR adds an All of the last round is in: Both design points are still open on my side, unchanged:
One fact bears on both. The Claude Code setup docs of three other Anthropic-protocol servers use a base with no Please keep holding off on the path handling, the help text at Two things are still outstanding from your side: the If the PR stays, these small items apply, and none of them needs doing before the two points are answered:
Nothing else is needed on the branch until the two points are answered. Once they are answered, what remains is aligning the code, help text, docs and wiki pages with that answer, plus the items above. |
|
Items 3 and 4 are done:
Items 1 and 2 stay on hold with the two design points. The Bifrost listing comparison is still blocked by a 502 from a provider behind it. |
|
The gateway answers now. Both listings were read with GET and the same bearer key (Bifrost, one provider behind it: llama-swap, 50 models).
So on this gateway the Anthropic route lists no context figure, and discovery has to stay on the OpenAI listing for it, which matches the current behaviour. The id on the Anthropic route is the bare model name, while the OpenAI route needs the prefix. I did not send a My read on the design points is still to come; I will post it once I have decided. |
|
Follow-up to the listing comparison: I sent
So on this gateway the Anthropic route accepts either id form, and an id taken from the OpenAI list works there as is. I ran this with plain HTTP requests, not through Claude Code, so I have not shown Claude Code reaching it end to end. |
|
Thank you @opticon454 for this revision. This PR adds an Items 3 and 4 from my last comment are in: Both design points are still open on my side, unchanged:
Please keep holding off on the path handling, the help text at Two things are still outstanding from your side: the If the PR stays, these small items apply, and none of them needs doing before the two points are answered:
Nothing else is needed on the branch until the two points are answered. |
|
End-to-end run with Claude Code 2.1.318 against the Bifrost gateway (
What this says for the two design points:
Not covered: these runs used I have no view posted yet on the two design points; this is the evidence for them. |
|
One correction to my end-to-end comment: I wrote that saving the base with |
|
Measured evidence for the first design point, using Codeman's own discovery route and Claude Code 2.1.318 against the Bifrost gateway (no guessing from the code). 1. Claude Code needs
So the figure is honoured whenever a 2. Where Codeman gets that figure. I ran Codeman's real
Discovery reads 3. Conclusion on this gateway. With the base at the root, Claude Code needs |
|
Closing this one. I tested Claude Code (2.1.318) against eight servers and only Bifrost needs Claude Code against each server. "Root" means
So llama-swap, llama.cpp, vLLM, Ollama and LiteLLM all answer the Anthropic protocol natively at the root, which is where Claude Code already posts. Bifrost serves it only under its own Context size. Claude Code only honours
The last row is a separate gap, not this PR: those four list a figure under other names ( For Bifrost in the meantime: a second endpoint saved as |
|
Thanks for measuring this before it shipped. The eight-server table settled it better than any review could have: only Bifrost needs |
Context
A saved custom endpoint can only be treated as OpenAI-compatible. Gateways such as Bifrost also expose the Anthropic Messages protocol, which Claude Code, OpenCode, pi and omp speak natively.
Split out of the custom-endpoint PR (#752) at the maintainer's request, because it has an open design question.
What changed
endpointType(openaioranthropic) on a saved endpoint, with a dropdown in the add and edit form. Omitted meansopenai, so nothing saved earlier changes.supportsAnthropic). Every other CLI refuses an Anthropic-type endpoint with one shared message, and drops out of the Run menu for it.ANTHROPIC_BASE_URLis the gateway's/anthropicroot (the SDK appends/v1/messages) instead of the raw saved base URL the OpenAI type uses. Model discovery still uses the gateway's OpenAI/v1/models.Open questions
Both are open on the maintainer's side and the code does not treat either as settled:
withAnthropicRootSuffixhardcodes an/anthropicpath segment, which is Bifrost's routing layout and not part of the Anthropic protocol. The alternative is to treat the saved base URL as the Anthropic base and only normalise the/v1.ANTHROPIC_BASE_URLat the raw saved base URL, so the only thing this option changes is appending/anthropic.Verification
typecheckis clean.npm testat8366e928: 689 files passed, 14,133 tests passed, 12 skipped, 1 failed. The failure istest/frontend-public-tooling.test.ts(public asset check), which needs gitignored spreadsheet assets absent from my test checkout; it does not touch this change.