Skip to content

feat(local-model): run a local coding model on the Mac Codeman runs on - #784

Merged
Ark0N merged 51 commits into
masterfrom
feat/local-model-recommendation
Nov 8, 2026
Merged

Ark0N merged 51 commits into
masterfrom
feat/local-model-recommendation

Conversation

@Ark0N

@Ark0N Ark0N commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

Codeman can now set up a coding model that runs on the Mac the server runs on. It reads the hardware, recommends a model and a coding agent that fit it, and on request downloads and checks both, saves the model as a Custom Model Endpoint Profile (id local-ollama) and offers to start a session of that agent on it. No account or API key is involved, and the model server runs with its cloud features switched off (OLLAMA_NO_CLOUD=1).

Ships in Codeman 1.50.0, with the macOS app (#763).

All of it is experimental: the routes and their payloads may change in any release (docs/versioning-policy.md). Apple silicon macOS only. On a server that is not a Mac the button and the Settings row stay hidden, and an install request is refused (400). An Intel Mac, a macOS older than 14 or a Mac with too little memory gets the window, which says why nothing is offered. Neither the routes nor the button sit behind a setting. In multi-user mode every route is admin only (403), and a non-admin sees neither surface.

What the user gets

  • "Run a local model" at the top of the start screen, above the launchers and in the size of the primary button: white with dark ink (dark on the light skins, where white would vanish). It reads "Local model" once one is installed and appears only after the server has answered for a Mac. Its tooltip says that "this Mac" is the machine Codeman runs on, which is not always the device the page is open on.
  • App Settings > Models > Model on this Mac: one row with the state in a few words and a button that opens the window.
  • One window for the whole flow. The chooser shows a memory map of what this Mac can give a model, filled with the selected model's weights, context and brief peak, and the models that fit as a radio list (the pick first, then the others by download size, largest first), each with its download size, a speed gauge and at most one tag: "Tested on a Mac like this", "Best fit", "Best fit, tight", "Tight", "Slow" or "Small model". The coding agent is chosen there too. The confirm screen lists the download (model, runtime and, when it must be installed, the coding agent with its literal install command), the total and where it goes. The other screens follow the job: progress, paused, failed (the server's sentence and the one action that helps), installed (Start a session, Stop or Start, Remove) and "nothing fits".
  • On a Mac without Xcode or the Command Line Tools, the confirm and installed screens say that coding agents use git and Python and that macOS will offer to install its developer tools the first time an agent needs one. fix(macos): never run Apple's placeholder git, which opens the developer tools window #779 keeps Codeman itself from running the placeholder git, but the agent runs git and python3 on its own.

The open window polls GET /api/local-model once a second; the start screen and App Settings ask it at most once a minute. There is no SSE event. The job runs in the server, so the window can be closed. Every new string has a zh-CN entry.

How it works

The recommendation (read only)

GET /api/local-model answers { hardware, recommendation, catalogUpdated, install }. It downloads, installs and changes nothing; its one write is bookkeeping (a job a restart interrupted is marked paused).

  • Probe (src/local-model/hardware.ts): one bounded spawn of /usr/sbin/sysctl with seven keys (memory, chip, arm64, Rosetta, iogpu.wired_limit_mb, macOS version, model), the free disk of the data dir through statfs behind the bounded path probe, node:os as the fallback when the spawn fails, cached for 60 s. It never runs system_profiler, never throws, and spawns nothing off macOS.
  • Catalogue (src/config/local-model-catalog.ts): a compiled constant checked by a Zod schema, with no override file. Eight models, one or more for every Apple silicon memory size from 8 to 512 GB, chosen from a web survey of current coding models (benchmarks, licences, tool-call support in the runtime's renderers and parsers). Every row says which runner a Mac uses for it (runner: mlx | llamacpp, required), and its size is what a Mac really downloads (the registry's manifest list, read the way the runtime reads it). Three coding agent rows (OpenCode, Claude Code, and pi, which is gated and never offered); a table of chip memory bandwidths. Model ids are lowercase name:tag without a slash, so they can only name a model of the runtime's default registry.
  • Budget: before the runtime exists it is an estimate: the wired limit when one is set, else 2/3 of memory below 36 GB and 3/4 from 36 GB, less 1 GiB, and never more than memory less a reserve (6 GiB up to 18 GB, 8 GiB up to 48 GB, 12 GiB above). Once the runtime has run, the device total it prints at start replaces the estimate (budgetSource: "measured"), still bounded by the reserve.
  • Need: per row and per runner, never borrowed. A llama.cpp row: its weights, a q8_0 K/V cache for the whole context, its fixed recurrent state, the copies llama-server keeps beside the cache (8 context checkpoints, and the rollback copies a multi-token-prediction tag needs per drafted token), 1 GB for the runner and the bounded 1 GiB prompt cache. An MLX row: its weights, the K/V cache and recurrent state, and the runner's fixed 8 GiB prefix-cache budget. Only qwen3.6:35b-a3b (MLX) is measured, on an M5 Pro with 48 GB; every other figure is computed from the model's config.json, marked estimated, and re-derived by test/local-model-catalog.test.ts from a table of architectures. The runtime now starts llama.cpp models with OLLAMA_KV_CACHE_TYPE=q8_0, OLLAMA_FLASH_ATTENTION=1, LLAMA_ARG_CACHE_RAM=1024 and LLAMA_ARG_CTX_CHECKPOINTS=8 (runtimeEnv()); the MLX runner reads none of them.
  • Fit: need over budget up to 0.85 is Good, up to 0.98 Tight, and above that the model is not offered. Fit decides whether a row is listed, not its place.
  • Ranking: a pair tested on a Mac like this one first, then speed (ok, unknown, slow below 10 tokens per second; estimated from the chip's bandwidth, and unknown for a chip not in the table or a model without a speed figure, never guessed), then model rank, coding agent rank and the larger context. A coding agent counts when it is not gated, is installed or has a stock install command without sudo or brew, works at that context and accepts the model id. A download must leave 10 GiB free on the disk; the best-ranked model left out for that reason is named in a note.
  • Verdicts: ok; limited when no model of 12 billion parameters or more fits (a statement about the Mac, judged before the disk filter); none with a reason (not-macos, intel, macos, memory, no-harness, disk) and one sentence.
  • "Recommended for this Mac" appears only for an ok pick that has a tested entry for exactly this chip, this memory size and this coding agent. The catalogue has one: Qwen3.6 35B-A3B with OpenCode on an Apple M5 Pro with 48 GB. Any other ok pick reads "Best fit for this Mac" with "Codeman has not run this on a Mac like this yet."; a limited answer has a headline of its own.

What each Mac gets (estimated budget, every coding agent available; pinned for every size from 8 to 512 GB in test/local-model-recommend.test.ts, the rule tests run on a frozen copy of the first catalogue in test/fixtures/local-model-rules-catalog.ts):

Memory Budget What it gets
8 GB 2.1 GB Nothing. The smallest model a coding agent can work with needs 10.2 GB at 65,536 tokens
16 GB 10.4 GB Nothing. MiMo V2.6 9B at 65,536 tokens needs 10.2 GB, 98.7% of the budget, just past the 98% a tight fit may reach (it is 95% of the 10.7 GB a 16 GB Mac mini measured). On the edge either way: only a run on such a Mac settles it
18 GB 11.8 GB A small model (limited): MiMo V2.6 9B at 131,072 tokens, 11.4 GB, tight. Its base, Qwen3.5 9B, is the one alternative
24 GB 16.1 GB The same, with room. On a base M2 or M3 (100 GB/s) it computes to 9.3 tokens per second: slow
32 GB 21.8 GB The same. North Mini Code 1.0 needs 23.2 GB at 65,536 tokens; once the runtime measures about 3/4 of the memory, as it did on a 16 and a 48 GB Mac, the Mac gets it at 131,072 tokens, or Qwen3.8 27B at 65,536 tokens on a Max chip
36 GB 27.9 GB Qwen3.8 27B at 131,072 tokens on an M4 or M5 Max (27.0 GB, 97%); on a slower chip, where the dense 27B computes to under 10 tokens per second, Qwen3.6 35B-A3B as a GGUF build (27.1 GB, 97%). Both tight
48 GB 37.6 GB Qwen3.6 35B-A3B (MLX) at 131,072 tokens: the pair we tested, "Recommended" on an Apple M5 Pro with 48 GB. On a Max chip Qwen3.8 27B at 262,144 tokens (32.1 GB, tight). The test was a short task: in a long session MLX may keep up to 8 GiB more, which brings this pair to the edge of the budget, and past it while a long prompt is processed
64 GB 50.5 GB Qwen3.8 27B at 262,144 tokens on a Max chip; Qwen3.6 35B-A3B (MLX) on a Pro chip
96 GB 76.2 GB Qwen3.8 27B at 262,144 tokens; on an M3 Max, Qwen3.6 35B-A3B (MLX)
128 GB 102 GB Ornith 1.5 122B-A10B (MLX) at 131,072 tokens, 97.4 GB, tight
192, 256, 512 GB 154 GB+ Ornith 1.5 122B-A10B at 262,144 tokens

Every pick runs with OpenCode. Set aside on purpose, with the reason in the catalogue header: Bonsai 2 27B (no agent-loop evidence), Ornith 1.5 397B and 9B/35B (no tool parser in the runtime), Flint (a tool-argument bug), LFM2.5 and Qwen3.8-Flash-Next (licences), Laguna (open macOS issues), gpt-oss:20b (weak at agentic coding), and the plain MLX tags at 36 GB and below (the 8 GiB prefix budget).

From a Claude Code session: cloud plus local

The codeman skill (skills/codeman, mirrored into plugins/codeman) can hand work to the installed model, so a Claude Code session plans and reviews in the cloud while the Mac does the bulk:

  • local_model asks GET /api/v1/local-model once and prints one line of JSON: installed and running or not, the coding agent, the endpoint and the model, or the reason there is nothing to use (not a Mac, nothing installed, install unfinished, blocked, admin only, no local model support on an older server). It never downloads or installs anything.
  • spawn_worker <name> local (also name:local in spawn_workers) starts the runtime the way the window's Start does when it is stopped, then quick-starts the installed agent with customModel (the same launch as the window's Start a session). A confirmation the server asks for fails the spawn with empty stdout; the skill never confirms. Readiness for OpenCode is its composer bar, measured live with OpenCode 1.3.0; a worker that never draws it is deleted.
  • sendmark <sid> <prompt> [timeoutSec] gives any worker without an end-of-turn signal a task: the prompt asks for the answer in codeman-answer-<token>.md plus a completion marker, and the helper returns once the marker shows or the answer file has settled (OpenCode's streamed redraws can split the marker, which a live run showed), reading at most 256 KB and refusing a symlink.
  • Tests: test/agent-skill-local-model.test.ts runs the real preamble against a fake server (34 tests, under bash 5 and bash 3.2).

Keeping the catalogue current

.github/workflows/local-model-catalog.yml runs scripts/check-local-model-catalog.mjs every Monday (and on demand): for every row it reads the registry the way the pinned runtime does on a Mac, and reports a tag that is gone, a size that drifted, or a runner that changed, plus a newer runtime release. It opens or updates ONE issue ("Local model catalogue: weekly check"), with contents: read and issues: write only. Its first live run found two rows of the first catalogue with the wrong Mac size, which the rebuild fixed. A model change still ships only in a release: the catalogue is compiled in on purpose.

The install

POST /api/local-model/install with { modelId, cli, context, acknowledgedBytes } starts one job of five steps (src/local-model/install-job.ts):

Step What it does
runtime downloads, checks and unpacks the pinned runtime, chooses a port, starts it, reads the measured budget and judges the choice again. A model that no longer fits ends the job here with does-not-fit and the next model to try, before any of the model is downloaded
model pulls the model through the runtime (POST /api/pull, parsed piece by piece: a 23 GB pull is about 1.4 million lines), retried up to five times
verify one small chat that loads the model; GET /api/ps must show all of it on the GPU at the chosen context; one tool call on /v1/chat/completions whose arguments parse
agent installs the coding agent with the CLI registry's stock command; skipped when it is installed
connect saves the endpoint and switches the endpoints setting on (below)

Before anything is downloaded, the request must name a model and context the current recommendation offers and a coding agent offered for that row (else 400), and acknowledgedBytes must be the server's own total for it (else 409 size-changed with the new total). A saved endpoint called local-ollama that this feature did not write is a 400 at this point too. One job at a time, and one installed model: another install answers 409 until Remove. Every step can be entered again, so Resume and Try again continue where the job stopped. Cancel interrupts the step in flight and pauses; in connect it counts only before the first write, after which both writes finish and the job ends installed. A job a server restart interrupted comes back paused and never continues without a click. Every failure is an error code with one sentence and one next step.

The other verbs are POST /api/local-model/cancel, POST /api/local-model/resume ({ newPort? }), POST /api/local-model/runtime ({ action: "start" | "stop", newPort? }) and DELETE /api/local-model ({ removeModel? }). Stop, a port change and Remove answer 409 with the session names while a live session uses the model.

Downloads, and how each is checked

  • Runtime: Ollama 0.40.15, ollama-darwin.tgz from Ollama's GitHub release, 185,971,671 bytes. The URL, size and sha256 are a compiled constant (OLLAMA_RUNTIME_PIN in src/local-model/runtime.ts) that no file, setting, environment variable or request can change. The archive is hashed while it streams, continued with a Range request at most three times, refused when longer than the pin, and compared before anything is unpacked; a mismatch deletes it. /usr/bin/tar unpacks it into a temporary folder, which is then renamed, and the archive is deleted.
  • Model: pulled by the runtime from its own registry under the catalogue's name:tag. Codeman does not hash the model itself, and the catalogue pins the name, not a digest. Once the check passed, Codeman stores the digest the runtime reports and compares it again at boot and before every agent start on the endpoint.
  • Coding agent: the compiled stock install command of a catalogue row, never a clis.json override (for OpenCode, curl -fsSL https://opencode.ai/install | bash). The window shows that literal command on the confirm screen before it sends the install; the server cannot tell whether an API caller saw it. Codeman verifies nothing about what that script fetches, the same as the install button in App Settings > CLIs, whose runner this step now shares (src/cli-install-runner.ts: an environment without CODEMAN_*, its own process group, a five-minute bound, one install per CLI at a time). It does not need cliManagementEnabled, and it writes the admin audit entry cli_install with via: "local-model".

What runs, and what is written where

  • The runtime is a non-detached child of the Codeman server, running as the server's user. Its environment is an allowlist: PATH, TMPDIR, LANG, the proxy variables, a private HOME, and fixed OLLAMA_* values (loopback host, model folder, context length, OLLAMA_NO_CLOUD=1, a 10-minute keep-alive, one request and one loaded model at a time). No CODEMAN_* variable and none of the user's own OLLAMA_* settings reach it. It listens on 127.0.0.1, on a port chosen once by binding from 11435 (11434 is left to a runtime the user may run). It is started at boot once a model is installed and was not stopped, restarted at most three times after a crash, and sent SIGTERM at shutdown. A leftover from a server that was killed is recognized by the command line of its stored pid (/bin/ps -o command=: a binary under Codeman's runtime folder) and adopted or replaced. Any other listener on the port is port-in-use and is never signalled.

  • The other programs this feature runs, all as the server's user: /usr/sbin/sysctl, /usr/bin/tar, /usr/bin/xcode-select -p (whether the developer tools are there; never git or python3), /bin/ps, and the coding agent's installer.

  • On disk, under the data dir: local-model.json (the job's state, mode 0600, holds no path, not part of config backups) and local-model/ with runtime/ollama-0.40.15/ (about 579 MB, plus the archive while it downloads), models/ (23.1 GB for the 35B model) and home/ (the runtime's private HOME, where it keeps its key pair; the user's own ~/.ollama is never created or touched). Every path comes from src/local-model/paths.ts, never from the state file.

  • Beyond those, Codeman writes two things, both undone by Remove:

    1. One record in custom-model-hosts.json: id local-ollama, label "This Mac", baseUrl http://127.0.0.1:<port>, the one model, and the chosen context as defaultContextLength (the runtime states none, so this is what reaches the coding agent).
    2. customModelEndpointsEnabled: true in settings.json (off by default). The value before is kept, and Remove switches it off again only when this feature switched it on and no other endpoint is saved. Switching it on also starts the existing five-minute discovery sweep for the other saved endpoints.

    The connection is claimed in the state file before either write, so a server killed between them leaves nothing the state file does not know about. The admin audit log gets local_model_install, local_model_remove and cli_install entries.

  • The coding agent's installer writes wherever it installs (OpenCode: under ~/.opencode). Remove leaves the agent installed.

  • Remove ends the runtime before any file goes, then deletes the runtime and its private home, and the model when asked, only inside local-model/ (and refuses a local-model folder that is a symbolic link). In the window "Also delete the downloaded model" starts ticked; the API keeps the model unless removeModel is true.

Starting a session on it

"Start a session" goes through the existing custom model launch (the Run menu's endpoint entry) with the installed coding agent on local-ollama. Before a new agent process is started on that endpoint, the server checks that the runtime child it started is alive, that the model's digest is the stored one, and that the model asked for is the one installed. The check sits on every path that starts an agent: a new session with the model (POST /api/quick-start), a model applied to a running session (POST /api/sessions/:id/custom-model), the attach that creates a missing pane or respawns a dead one (from a click on the tab or boot recovery), restartCli(), and the reboot-restore rebuild, which skips the entry as local-model-not-ready and leaves it on offer. An attach to an agent that still runs, and a respawn cycle that types into it, are not launches and are not asked. A refusal is a LaunchRefusedError and never falls back to a direct PTY.

Changes outside the feature

  • src/custom-model-hosts-writer.ts: every write of custom-model-hosts.json (the four writing /api/model-endpoints routes, the discovery sweep and this feature's connect step) now goes through one serialized writer with a random temp name, mode 0600. Before, two writers at once could tear the file or drop each other's record: ten creates sent together did not all land. A file that does not parse is now refused with 409 instead of being read as an empty list and replaced, and deleting an unknown id no longer rewrites the file. The id local-ollama is reserved: a new endpoint may not take it, and while the record is the managed one these routes neither edit nor delete it and the sweep skips it. All of this is visible on the stable /api/v1/model-endpoints routes: creating an endpoint with that id, or editing or deleting the managed record, now fails with INVALID_INPUT. A guard test fails any other writer in src/.
  • src/cli-install-runner.ts: the CLI install runner moves out of cli-registry-routes.ts unchanged, so both callers share it; the route's tests are unchanged. New in it: an abort signal that kills the process group, stockInstallCommand() and tryBeginCliInstall().
  • src/session.ts: setLaunchGuard() and LaunchRefusedError, asked before every new agent process. WebServer.setupSessionListeners() sets the guard for every session, and it has nothing to say for a session that is not on the managed endpoint.
  • src/web/server.ts: the routes, the service's boot() at start-up and shutdown() beside stopDeepSeekWeb() (both skipped in test mode).
  • src/web/sensitive-path.ts: the runtime's private home and any .ollama/id_ed25519 are sensitive paths.
  • Docs: the guide docs/local-models.md, the API reference, the versioning policy, security architecture §10d and .github/SECURITY.md (the port without a login), plus the endpoint pages, the wiki and both CLAUDE.md files.

Map of the new files

File What it holds
src/types/local-model.ts, src/types/local-model-install.ts the payload, every sentence the server may send (the zh-CN dictionary is keyed by them), the install states, steps and error codes
src/config/local-model-catalog.ts the catalogue and its schema
src/local-model/hardware.ts the probe
src/local-model/recommend.ts recommend() and modelFitsBudget(), pure
src/local-model/harness-facts.ts the one module here that reads the CLI registry
src/local-model/runtime.ts the pin, the download, the environment, the port, the supervisor, removal
src/local-model/ollama-client.ts the runtime's HTTP API, the pull parser, the checks
src/local-model/install-job.ts the state tables, the pure reducer, the view and the service (2,235 lines)
src/local-model/connect.ts the two writes, their undoing, the digest comparison
src/local-model/state-store.ts, paths.ts, format.ts, developer-tools.ts the state file, the folders, the one size formatter, the xcode-select -p check
src/web/routes/local-model-routes.ts the six routes
src/web/public/local-model-ui.js the button, the Settings row and the window (styles in styles.css and mobile.css, strings in i18n.js)

About 31,000 lines are added: 18,600 of them tests, 9,000 server code, 3,100 frontend.

Testing

Unit and integration tests (no Mac, no network, no runtime process; the end-to-end test does run the real /usr/bin/tar):

  • The probe parser on the captured sysctl output of two real Macs, the catalogue schema, recommend() across memory sizes, chips and every none reason, the pull parser and the checks, the state file, the job's reducer, and the window over the real markup in JSDOM.
  • A fake runtime (test/fixtures/fake-ollama.ts) on a port the system picks, answering in the shapes measured on a Mac.
  • test/local-model-install-e2e.test.ts runs the real routes, service, download, hash check, /usr/bin/tar, supervisor, pull, checks, connection and state file against that fake: the whole install and its removal, a runtime download that arrives damaged, a model download that fails twice and then finishes, one that keeps failing, a model that calls no tool, Cancel during the download, a server restart during the download, and a runtime left behind by a server that was killed.
  • Launch guard tests for both session routes, the dead-pane respawn, restartCli() and reboot restore.
  • The full gate on the branch head (be8fc298): 16,013 tests in 732 files passed, 12 skipped. test/local-model-ui.browser.test.ts, which is outside the gate, passes 6 of 6 in Chromium.

On a real Mac (a MacBook Pro with an M5 Pro chip and 48 GB, macOS 27.2), with a server from a test branch that merged this branch with the other 1.50.0 work:

  • GET /api/local-model, on an earlier head of this branch with the same rules, answered Qwen3.6 35B-A3B at 131,072 tokens with OpenCode, 36.0 of 37.6 GB estimated, tight, tagged "Tested on a Mac like this", OpenCode not installed.
  • The start screen button and the chooser were seen in the macOS app on that Mac.
  • The install request the button sends (qwen3.6:35b-a3b, OpenCode, 131,072, acknowledgedBytes 23,291,770,627) was sent through the API, not clicked: the runtime (about 0.2 GB) in 5 s, the model (23.1 GB) at 53 to 74 MB/s, verify, the OpenCode install, connect. The state was installed after 376 s, with the model loaded on the GPU and the endpoint on port 11435.
  • The chooser's start-a-session request (quick-start with OpenCode and customModel on local-ollama), also sent through the API, started an OpenCode session on the local model, and it answered a prompt.

By hand, before this feature existed, with the same runtime build and the same model on that MacBook; the catalogue's measured figures and its one tested entry come from this run. The runtime reported a device total of 37.4 GiB and generated 67.7 tokens per second. The model held 36,000 to 38,000 bytes per token of context and 2.5 to 3 GB beside the weights, and took 5.3 GiB (5.7 GB) more for a moment while an 89,000-token prompt was processed (the catalogue uses 37,000 bytes, 3 GB and 5 GB). A four-step task with tools made 4 tool calls, none malformed. On a Mac mini with an M4 and 16 GB the same runtime reported 11.8 GiB, and the recommendation tests use that figure: that Mac gets no model with either budget.

Limits

  • Nothing but the 48 GB pair is measured: the rebuilt rows are computed. Measuring the shortlist on a real Mac (the 16 GB and 18 GB 9B first, then the llama.cpp settings on Metal) can promote rows to measured or move the 16 GB answer, which today misses a tight fit by 0.7%. The skill's local worker path has run against a fake server and a live OpenCode on Linux, not yet on a Mac with the model.

  • The tested 48 GB pair was measured on single long prompts. A runtime issue reports this model's MLX build growing up to 8 GiB over a long agent session, which would pass the measured budget on that Mac; a 30-request session settles it.

  • The runtime's port has no login. Any program on the Mac, and any page served from localhost, can use the model server and change its models, as with any model server of its kind. Codeman limits what that buys (one model in the endpoint record, the sweep leaves it alone, the digest check before an agent starts) but does not authenticate the port. Listed in .github/SECURITY.md.

  • Not yet done on a real Mac: clicking Install in the window (the real install went through the API), Cancel and Resume, Stop and Start, Remove, a Codeman restart with a model installed (the start at boot, adopting or ending a leftover runtime, SIGTERM at shutdown with live sessions), the launch refusals and a change of the runtime pin. Only the tests above cover these. Nobody has tried Safari.

  • Only OpenCode has been run on the local model. Claude Code is offered as the second coding agent (the catalogue marks its first turn as slow, for its long fixed prompt) and has not been run against this runtime.

  • The runtime stops with Codeman. A session that is mid-answer on the local model loses that answer at a Codeman restart; the runtime comes back with the server and the first request loads the model again. An open session keeps calling the port while the runtime is stopped, which is why Stop and a port change are refused while a session uses the model.

  • An agent that already runs when Codeman restarts is not checked again, since Codeman does not start it.

  • The runtime does not enforce the context length: in the hand run it accepted a 167,012-token prompt at 131,072.

  • A loaded model keeps its memory for ten minutes after the last request (23.2 GB for the 35B model on the MacBook).

  • The memory figures rest on one measured model. They may be too high for the 9B rows, which is what leaves 16, 18 and 24 GB Macs without a model, and too low for the two dense 27B rows. Only the check after the download (all of the model on the GPU) catches a row judged too kindly. The speed estimates come from one published run on an M1 Max.

  • The model is pinned by name, not by digest: an install gets what the registry serves under that tag that day, and from then on the digest is held.

  • There is no way to update the runtime in place. An installed model keeps the runtime version it was installed with; a later pin takes effect after Remove and a new install.

  • After does-not-fit the window offers only a model that needs less memory, not the same model at a smaller context.

  • The OpenCode the job installs is not on the login shell's PATH: Codeman finds it, opencode typed in a terminal does not.

  • A config-backup restore can take the endpoint record away while the model stays installed. Codeman puts it back at its next start, not at the restore.

  • The runtime archive is fetched with Node's built-in fetch, which ignores HTTP_PROXY by default; the model pull goes through the runtime, which is given the proxy variables. Neither has been tried behind a proxy.

  • On a phone, "this Mac" is the server. mobile.css makes the window full-screen, and the start screen button is reached only with the phone overview switched off. This PR reports no run of it on a phone or in npm run test:mobile.

  • For reviewers: harness-facts.ts repeats the launch path's model-id acceptance test, which session-routes.ts keeps inline at two sites, so a change to that rule must be made in all three until they share a helper. install-job.ts holds both the pure tables and the IO service.

Landing

Base master. The branch sits directly on the current master and merges cleanly. It is already in the 1.50.0 integration branch; when the release lands, master fast-forwards to it and this PR shows as merged.

It depends on no other PR of the batch. Against the others it has two small text conflicts, both resolved on the integration branch by keeping both sides: config/test-suites.ts with #763 (each adds a browser test to the list) and the state files line of CLAUDE.md with #778. #783 also changes the start screen and merges with this one without a conflict.

No changeset on the branch: the 1.50.0 release changeset covers it.

claude added 28 commits November 6, 2026 14:56
…ecommendation

The contract three builders code against in parallel. Nothing is recommended
yet: the route answers 501 and the catalogue is empty.

- src/types/local-model.ts: HardwareFacts, the catalogue rows, HarnessFact,
  Candidate, Recommendation, LocalModelStatus, and LOCAL_MODEL_LABELS, the one
  list of sentences the server may put into the payload (the zh-CN dictionary
  is keyed by them).
- src/local-model/format.ts: formatGb, the one place a byte count becomes a
  size a person reads (decimal GB), and memClassGb.
- src/config/local-model-catalog.ts: the Zod schema (model ids are name:tag,
  lowercase, no slash) and an empty stock catalogue. No override file.
- Skeletons with their final signatures: hardware.ts, recommend.ts,
  harness-facts.ts, routes/local-model-routes.ts (injectable dependencies),
  public/local-model-ui.js with its script tag.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…mendation

The core of the read-only recommendation: what this Mac is, what the catalogue
knows, and which model fits. Nothing is downloaded, installed or written.

- hardware.ts: one bounded async spawn of /usr/sbin/sysctl with seven keys,
  without -n, parsed as "key: value" lines by the pure parseSysctl(). Free disk
  of the data dir through statfs behind the bounded path probe. Fallbacks from
  node:os, a 60 s cache, never throws, never system_profiler. A wired limit of
  0 means "not set".
- local-model-catalog.ts: seven models, three harness rows (pi gated) and the
  chips table as a compiled constant, validated by a Zod schema on first use.
  No override file. Model ids are name:tag in lowercase without a slash; every
  model carries paramsB and an (empty) tested list. Chip names match on the
  end of the chip string, so 'M4' is not the chip of 'Apple M4 Pro'.
- recommend.ts: device budget (the wired limit when above 0, else 2/3 of
  memory below 36 GB and 3/4 from 36 GB, less the reserve), need = weights +
  KV cache + 1 GiB + prompt cache, fit good up to 0.85 and tight up to 0.98,
  speed ok, unknown or slow (a chip in no chips row is unknown), and the
  ranking. Verdicts ok, limited (best model under 12 billion parameters) and
  none with its reason. "Recommended" only for a pick with a tested entry for
  this Mac, which no row has today.
- types/local-model.ts: comments brought in line with the code (chip matching,
  budgetShare rounding, a budget of 0 on an Intel Mac).
- cli-registry-no-id-branching.test.ts: the guard also catches a comparison of
  a `cli` field with a CLI id (plus one stray blank line Prettier refused).

Tests: test/local-model-format.test.ts, test/local-model-hardware.test.ts
(the captured output of two real Macs), test/local-model-catalog.test.ts,
test/local-model-recommend.test.ts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The read-only route behind the recommendation, and the one module that reads
the CLI registry for it.

- harness-facts.ts: one fact per catalogue harness row, resolved in the CLI
  registry at call time: the registry's label, `installed` from the existing
  availability probe (asked once per build), `installable` for a stock CLI
  whose install command for this platform needs no administrator rights, and
  `acceptsModelId`, the test a custom model launch applies to the model id.
  A row whose CLI is unknown, disabled or has no custom model mechanism gets
  no fact. No CLI id is compared with a literal.
- routes/local-model-routes.ts: answers { hardware, recommendation,
  catalogUpdated } in the usual envelope and reformats nothing. Admin only in
  multi-user mode (403), checked before anything is probed. A failed probe is
  an answer (200, the hardware's `source` says what was read); a 500 is left
  for a fault in the catalogue, the harness facts or recommend(). It writes no
  file and reads no setting.

Tests: test/local-model-harness-facts.test.ts,
test/routes/local-model-routes.test.ts (app.inject(), no port; the temp HOME
is byte-identical before and after the requests).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… this Mac

The read-only face of GET /api/local-model: what the Mac the server runs on
can run as a local coding model.

- local-model-ui.js: one builder paints both containers from the same
  payload: the hardware line, the server's headline, the pick with its numbers
  (context, download, memory need against the budget, speed), the server's
  notes, the other models that fit with their Good, Tight, Slow or Speed
  unknown labels, and one line saying that setting it up from Codeman is the
  next step and is not built yet.
- The welcome card ships hidden and shows only once the server answered for a
  Mac. It stays hidden for not-macos, for a refused request (a non-admin in
  multi-user mode) and after "Hide", a per-device localStorage key. The App
  Settings group "Model on this Mac" (Models) carries the loading and failure
  sentences and is the way back after a Hide.
- Every dynamic string goes in through textContent, both containers are
  data-i18n-skip and the module translates its own sentences, repainting on a
  language change. Sizes are printed as sent. Labels and harness names come
  from the payload: the module names no CLI and the word "Recommended" is not
  in it.
- One request a minute at most, shared while in flight; a failed refresh keeps
  the last answer on screen.
- i18n.js: zh-CN entries for the module's strings and for every server label.
- frontend-cli-no-id-branching.test.ts scans the new module.

Tests: test/local-model-ui.test.ts (JSDOM over the real markup),
test/local-model-i18n.test.ts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A "Local model recommendation" section in the API reference: the read-only
route, its admin gate in multi-user mode, why a failed probe is a 200, and
every field of { hardware, recommendation, catalogUpdated }, including that a
size travels as bytes and as decimal GB text.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The label list was "frozen for the parallel build" and the card's test said
the route answers 501: both were true while the pieces were written side by
side and are not any more. Comments only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…sing

Review fixes for the read-only recommendation and its card.

Recommendation (pure):
- `limited` is now a statement about the Mac: no model with 12 billion
  parameters or more fits. It is judged on everything that fits, before
  the disk filter and whatever the pick is. A 32 or 36 GB Mac whose pick
  is a 9B model no longer reads "This Mac can run a small model" above a
  list of 20B, 27B and 35B models, and knowing the chip no longer lowers
  the verdict. A small pick under `ok` carries a note that says so.
- A model that fits the memory but has no room on the disk is no longer
  dropped in silence: the best-ranked one is named in a note with the GB
  that are missing. A full disk cannot turn a 48 GB Mac into a
  small-model Mac.
- "Models fit, but not at a context a harness needs" has its own
  sentence with the budget and the need, worded as an estimate. The
  16 GB Mac mini misses by 1.3 points (10.3 of 10.4 GB) and was told
  flatly that its memory is too little, the sentence of an 8 GB Mac.
- The installed memory is named by the size the Mac is sold as (48, not
  51.5): the hardware line and the memory sentence. Every computed size
  stays decimal GB from the one formatter. An unknown memory size is no
  longer reported as "0.0 GB".
- Slow is graded on the unrounded speed: 9.95 to 9.9996 tokens per
  second printed as 10 and read ok.
- The disk sentence names the 10 GiB that must stay free, so its
  numbers add up for the reader.

Card and Settings group:
- The Settings group "Model on this Mac" is hidden on a server that is
  not a Mac, and until the first answer says it is one.
- Only `403 FORBIDDEN` is a refusal. `403 PASSWORD_CHANGE_REQUIRED`
  (an admin who must change the password) is a failed request and is
  asked again, so that admin no longer loses the card until a reload.
- Each fit and speed label carries what it measures as its tooltip and
  accessible name, an alternative with a speed estimate prints it, and
  the count beside "Other models that fit" is gone (it was the number
  of rows, capped at four, not the number of models that fit).
- The line saying nothing can be set up yet sits right under the pick,
  in the colour of every other line (it was 3.9:1), and says whose step
  it is.
- The dismissal key is named for this stage, so hiding the read-only
  card does not hide the install offer later.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- docs/api-reference.md: GET /api/local-model is labeled experimental
  until the step that sets a model up ships, so its payload is not a
  stable /api/v1 contract one stage before that step changes it. The
  section now describes the verdict as judged, the three sentences of
  the memory reason, the notes, the speed rounding and which size to
  show as the installed memory. The os-fallback description was wrong:
  the free disk space is reported there too.
- docs/versioning-policy.md: the route is listed under the experimental
  surfaces.
- docs/wiki/Settings-Reference.md: App Settings > Models > Model on this
  Mac, in the words of a user.
- CLAUDE.md and src/web/CLAUDE.md: the invariants the code relies on
  (read-only, the probe, harness rows as data, the meaning of limited,
  the word Recommended, one unit, labels from the payload, when the
  card and the group show).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… runtime)

The types, the module skeletons and the test fake that let the runtime, the
install job and the window be built at the same time.

Types:
- src/types/local-model-install.ts (new): the runtime pin, the install
  states, steps and error codes with one sentence and one next step each,
  the progress, the `install` object of GET /api/local-model, the bodies and
  409 answers of the install verbs, the state file. `LocalModelStatus` moves
  here; `install` is optional until the route emits it.
- src/types/local-model.ts: the catalogue row's Mac memory figures, the
  measured speed of a tested entry, the `measured` budget source,
  `RecommendOptions`, and the row tag, the coding agents and the install
  total of a candidate. All optional until the catalogue and `recommend()`
  carry them, so the read-only stage keeps compiling and its tests stay
  green.

Modules with final signatures and bodies that throw:
runtime.ts (the compiled pin is final), ollama-client.ts, developer-tools.ts,
state-store.ts, install-job.ts (the state, step and transition tables are
final), connect.ts, custom-model-hosts-writer.ts, cli-install-runner.ts.
Nothing imports them yet, so the server is unchanged.

Implemented: paths.ts (every folder of the feature, from dataPath()), and
`modelFitsBudget()` in recommend.ts, the judgement the job makes with the
measured budget before it downloads a model. `recommend()` takes the options
parameter and does not read it yet.

Test fake: test/fixtures/fake-ollama.ts, a runtime on a port the system
picks, in the shapes measured on a Mac: version, tags, ps, the pull stream
(MLX and GGUF lines, an error line, a cut stream, a resumed pull), chat with
and without a tool call, the model list, a runtime archive with Range
support, and a fake child process that logs the device total.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rebuilt log line

- `LocalModelService.launchRefusal(endpointId)`: what a session launch asks
  before it starts on the managed endpoint. It answers the error that
  refuses the launch (the runtime child is not alive, another program holds
  the port, the runtime keeps stopping, the model's digest changed) or
  nothing. New error code `runtime-not-running` with its sentence, for a
  runtime the person stopped: only a refused session start says it.
- The fake runtime's MacBook log line is rebuilt from the two values the
  measurement notes quote, so it is `INFERENCE_COMPUTE_LINE_MACBOOK` now.
  Only the Mac mini line is a verbatim capture and keeps `REAL_`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ed Mac first

The catalogue rows carry the figures measured on a Mac (the MLX build of
qwen3.6:35b-a3b is 23,105,798,956 bytes, about 37 KB held per context token,
about 3 GB of runner overhead, about 5 GB more for a moment while a long
prompt is processed). A model's need is now its peak. Rows that were not
measured use the same per-token and overhead figures and say `estimated`.

qwen3.6:35b-a3b gets the tested entry for an Apple M5 Pro with 48 GB and
OpenCode, with its measured 67.7 tokens per second, so that row may say
"Tested on a Mac like this".

The ranking changes with it: a pair tested on a Mac like this one comes
first, then speed (ok, unknown, slow), then model rank, harness rank and the
larger context. The fit grade decides whether a row is listed and what it
says, it no longer sorts, so a model is offered at the largest context it
fits at. recommend() takes the budget the runtime measured
(`budgetSource: measured`), and modelFitsBudget() judges one model against it.

Candidate rows now always carry `needSource`, `harnesses`, `installBytes`,
`installGb` and, where one applies, `tag`. `kvBytesPerToken` and the prompt
cache tiers are gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… client

runtime.ts: the runtime is pinned in a compiled constant (version, URL, size,
sha256). The archive is streamed to a .partial file while it is hashed,
continued with a Range request, retried at most 3 times, and the hash is
compared before anything is unpacked; a mismatch deletes the file. It is
unpacked with /usr/bin/tar into a temporary folder and renamed.

The supervisor runs the runtime as a non-detached child with an environment
built from an allowlist by the pure runtimeEnv(), on a port chosen once by
binding. It restarts the child at most 3 times, stops it with SIGTERM, adopts
or replaces its own orphan on the port and answers port-in-use for any other
listener. parseInferenceComputeTotal() reads the device total the runtime
logs when it starts. Under vitest nothing spawns or fetches.

ollama-client.ts: the pull is parsed incrementally by a pure parser (a 23 GB
pull is about 1.4 million lines), with up to 5 retries on an error line or a
cut stream. verifyModel() loads the model with one tiny chat, checks what the
runtime reports (all on the GPU, the chosen context) and makes one tool
probe; each failure is a code.

developer-tools.ts: only /usr/bin/xcode-select -p, bounded, darwin only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The runner that executes a CLI's stock install command (the environment
without CODEMAN_*, the npm prefix redirect, the bounded process group, the
one install per CLI at a time) moves from cli-registry-routes.ts to
src/cli-install-runner.ts, so the local model install can use the same one.
The route re-exports installEnv and npmGlobalPrefixWritable and behaves as
before; its tests are unchanged.

New in the shared module: an abort signal that kills the process group,
stockInstallCommand() (the compiled stock command, never the clis.json one)
and tryBeginCliInstall(), the one install slot per CLI for both callers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The four endpoint routes and the 5-minute discovery sweep each read
custom-model-hosts.json, changed their copy and wrote it back through one
shared temp file name. Two of them at once could tear the file or drop each
other's record: ten creates sent together did not all land.

Every write now goes through mutateCustomModelHosts() (one promise chain, a
random temp name, mode 0600), with each check inside the same step as its
write. A file that does not parse is refused with 409 and never replaced.
A delete of an unknown id no longer rewrites the file. A guard test fails
any other writer in src/.

The id `local-ollama` is reserved for the model Codeman runs on this Mac: a
new endpoint may not take it, and while the record is the managed one it is
not edited, deleted or re-discovered from these routes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…an pause

POST /api/local-model/install, cancel, resume and runtime, and DELETE
/api/local-model. The GET gains an `install` object the window polls. All of
it is experimental and admin only in multi-user mode.

One job of five steps (runtime, model, verify, agent, connect) driven by a
pure reducer over explicit state, step and transition tables. One job at a
time (409). The consent is bound to the size shown: `acknowledgedBytes` must
be the server's own total (409 size-changed). A job a server restart
interrupted comes back paused and never continues without a click. Every
failure is a code with one next step.

The model is judged against the budget the runtime measured before it is
downloaded. A missing coding agent is installed with its stock command
through the shared runner, audited. Connect writes one endpoint record under
the reserved id through mutateCustomModelHosts() and switches
customModelEndpointsEnabled on through mutateSettingsFile(), remembering the
flag's value. The model's digest is stored and read again at boot and before
every session start on that endpoint; a changed model refuses the launch.

Remove answers 409 with the session names while a live session uses the
endpoint, undoes the two writes first, restores the flag only when this
feature switched it on and no other endpoint is saved, and deletes only
inside local-model/. The coding agent stays.

The server starts the runtime at boot when a model is installed and stops
it at shutdown beside stopDeepSeekWeb().

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e card

The read-only card is gone. The welcome screen has one button, "Run a local
model" ("Local model" once one is installed), and App Settings > Models has
one row; both open the same window.

The window follows `install.state`: choose a model (a radio group, with a
memory bar filled to what the selected model needs), confirm the sizes and
the folder, watch the download and the steps, and end on "Start a session".
Paused, failed, installed (running or stopped), Remove and "no model fits"
are screens of the same window. It polls the GET once a second while open.

Every dynamic string goes in through textContent, labels and sentences come
from the payload, no CLI id is compared, and every new string has a zh-CN
entry.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
docs/local-models.md is the guide: the steps, the layout on disk, what is
written outside the folder, the failures and their next steps. The API
reference documents the install object and the five verbs, and how the list
is ordered. The rules for the server side are in CLAUDE.md and
architecture-invariants.md, the user-facing pages in the wiki.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The real routes, service, recommendation, runtime download and unpack,
supervisor, pull, checks, connection and state file, joined together. Only
the runtime (a fake HTTP server the injected spawn "starts"), the hardware,
the coding agent's installer and the waits between retries are stand-ins.

Walks: the whole install and its removal (the folder is gone, the settings
flag is restored); a runtime download that arrives damaged; a model download
that fails twice and then succeeds; one that keeps failing; a model that
calls no tool; Cancel during the model download; a server restart during the
model download, which comes back paused and waits for a click.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ep its version

The supervisor reports the child's pid the moment it is spawned, before it
answers, so a server killed during a start still leaves a pid that names the
runtime as ours. stop() takes that stored pid (previousPid) and ends a
runtime an earlier server left behind when its command line is a binary
under our runtime folder, so no caller deletes the folder under a live
process. start() takes a version: an installed model keeps the runtime
version it was installed with when a later build pins another one.

removeLocalModelFiles() refuses a local-model folder that is itself a
symbolic link (every check compared against the real path of the root, so
a link in its place passed whatever it pointed at). removePartialModelBlobs()
deletes what an interrupted download left in the model store, which the
runtime answers "not found" for and never deletes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…k refused, say why a start failed

The connect step records the connection in the state file BEFORE it writes
the endpoint record and the settings flag (the flag's earlier value read by
inspectLocalEndpoint()), so a state write that fails or a server killed
between the two never leaves a record of ours that the state file does not
know; a record under the reserved id that says exactly what ours would say
is ours. changePort() writes the state first and puts the old port back when
the endpoint move is refused.

A job whose end could not be written (a full disk) is kept in memory: the
view shows it, Resume and a new install start from it, the next verb writes
it, and Cancel never answers a 500. A first install on a full disk is a 507
with the sentence. "installing" in the file while this process runs no job
always reads as paused or that failure.

An installed model whose runtime does not start says so (runtime-start-failed
with Start as the next step, and the launch check says it too); the stored
runtime version is started while its folder is unpacked. Stop, Remove and
the boot of anything that is not a running installation end a runtime an
earlier server left behind before any file is deleted. Stop and "Use another
port" answer 409 sessions-open while a live session uses the model. A pull
that failed because the runtime is gone is runtime-stopped, not a network
problem. After "does not fit" only a model that needs less memory is offered.
A replaced job's partial blobs are deleted. Every 400 carries its sentence as
data.message, and the six inline sentences are labels with zh-CN entries.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… on the managed endpoint

Only the two session routes asked launchRefusalMessage(). The attach that
finds no pane, the respawn of a dead pane (a click on the tab, a respawn
start, the boot recovery), restartCli() and the reboot-restore rebuild
started an agent on the endpoint unasked. Session gains setLaunchGuard(),
asked right before a NEW agent process is started and never for an attach to
an agent that runs; WebServer.setupSessionListeners() sets it for every
session; a refusal is a LaunchRefusedError that never falls back to a direct
PTY. The reboot-restore route asks before it builds and skips the entry with
its own reason, left on offer.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…for the consent

The window is one frame for the whole flow: a fixed height on desktop, App
Settings' boxed close button, the buttons in a foot of their own outside
the part that scrolls, so a click never moves the next button away and
nothing is cut off; the short screens are centred in it on desktop and sit
above the buttons on a phone.

The chooser lists the pick first and then the other models by download
size, largest first, a column the reader can see. A pick that only just
fits says "Best fit, tight". The memory figure is an estimate and says so
once, in whole GB; its legend names the peak, and the installed line uses
the same words beside what the model takes now, so the two figures are not
read as a contradiction. The legend, the column headers and the foot's
promise are set in --text-dim, the bar's track in --text-muted so its free
part can be seen, and only the download figure keeps tabular figures.

The consent screen is the download as a table: the chosen row, the runtime,
the coding agent with its installer's command as the row's second line, the
total in the strong weight, then where it goes in one line. The download bar
means the model download alone and is gone once the model is in; the screen
that ends an install shows "Start a session" alone; an error on an installed
model is the headline with a grey dot; an error's first sentence is the
headline; Remove is quiet everywhere but red in its own question. The
no-recommendation reasons are one sentence each, and the untested note is
in Codeman's voice. A refusal the server names is printed from its sentence.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…tors, and the port without a login

The runner header says what each caller passes (the job: the compiled stock
command; the route: the merged entry, behind cliManagementEnabled).
docs/cli-registry.md names both executors in its trust boundary. The
security architecture gains a section on the pinned runtime and its hash,
the loopback port without a login and the digest check, and the install
path without cliManagementEnabled; SECURITY.md lists the port as a known
limitation. The guides and the rules name exactly the paths the launch
guard covers, the session that keeps calling a stopped runtime's port, the
claim before the connect writes, and the outcome kept in memory.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…of the main one

The owner asked for it much bigger and in the middle. The button keeps the
chip's markup and takes the primary button's height, radius and weight, with
an accent outline and its glyph in a disc, so a model on this Mac is offered
as a main way in rather than as a footnote under the launchers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…utton first

The owner's verdict on the chooser: nice, but not nice enough, too much just
blue. He also wants "Run a local model" on top of the start screen, above
Run Claude Code.

The memory bar is now a map: the model, its context and the brief peaks side
by side on the track of what this Mac can give a model, each with its size in
a legend, and the verdict on the fit beside the figure. recommend() sends what
the need is made of as `Candidate.memory`.

Colour now carries meaning: green for good and tested, amber for tight, orange
for slow, teal for the context. Violet is the feature's own colour, on its
start screen button and its main action. Every row has a size tile and a speed
gauge scaled to the fastest row in the list. The light skins get deeper values
of the same colours, and the window is 40 px taller to hold it without
scrolling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ncher

The violet button was the one thing on the start screen outside the app's
blue, and it carried white text and a see-through disc where its neighbour
has dark ink and a solid disc. It is now white with dark ink and a solid
disc, the same on the Install button, and the chooser's selected row, tile
and model bar follow. Light skins turn it over to dark, since a white button
vanishes on a pale page. The context takes the app's blue.

The tokens are named for what they mean now (--lm-brand, --lm-context).

Brings test/local-model-ui.browser.test.ts up to the redesign it had fallen
behind: it is outside the gate, so the button's 46 px, the 12 px bar and the
memory panel's edges had gone unnoticed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The codeman skill's preamble learns the model Codeman installs on this Mac:

- local_model asks GET /api/v1/local-model and prints one JSON line: the
  installed model, the coding agent that runs it and its endpoint, or the
  reason there is none (not a Mac, nothing installed, an unfinished
  install, a blocked install, admin only, a server without the routes).
  Exit 0 only when a worker can start now or after a runtime start.
- spawn_worker <name> local (and name:local in spawn_workers) starts a
  stopped runtime the way the window's Start button does, quick-starts the
  installed agent with customModel, and waits for its composer (OpenCode's
  bar, the claude path for a claude agent, LOCAL_READY_MARK otherwise). A
  confirmation question from quick-start fails the spawn with empty stdout;
  nothing is ever confirmed on the person's behalf.
- sendmark <sid> <prompt> drives any agent worker without an end-of-turn
  signal: the exactly-once input body, a split marker through wait-output,
  then the answer read back from a file the worker wrote in its folder
  (regular files only, 256 KB at most), with one exit code per outcome.

The preamble stamp moves to 1.50.0 so every cached copy is rewritten.
The tests run the real preamble against a stubbed server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
claude added 22 commits November 8, 2026 07:48
SKILL.md gets a subsection after the DeepSeek Harness workers: why to hand
work to the model on this Mac (bulk work with no per-token bill, files
that stay on the machine, a second opinion, a worker that needs no
network), the copy-paste block (local_model, spawn_worker local, sendmark,
delete_session), and the limits as the code has them: a person installs
the model, one request at a time, slower than the cloud, markers not
hooks, a spawn can start a runtime the person stopped, admin only and
experimental.

The description now names the local model, so "use the local model"
reaches the skill. Section 2 gains a row, the hook-less row points at
sendmark, and reference/verbs.md gets 5.16 with every local_model reason,
the spawn steps and the sendmark exit codes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/local-models.md says how a Claude Code session drives the model on
this Mac through the skill (local_model, spawn_worker local, sendmark)
and what stays true: the skill never installs a model or answers a
confirmation, one request at a time, and Stop and Remove wait for the
workers it opened.

The README gets the local model as a feature with that pairing, and an
example prompt in the skill's table; the wiki's skill page gets the same
row and a short paragraph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…week

scripts/check-local-model-catalog.mjs reads every catalogue tag the way
Ollama 0.40.15 on Apple silicon reads it: it asks for the manifest list,
takes the mlx build when the tag offers one (else llamacpp, else ggml),
and compares that build's pull total (layers plus config) with the row's
`bytes`. It also reports a tag that is gone, a build whose `requires` is
newer than the pinned runtime, a pinned digest that moved, a newer Ollama
release, and a pinned ollama-darwin.tgz that is gone or changed. It never
requests a layer, bounds every request, retries nothing and validates
everything that reaches the report. Exit 0 clean, 1 drift, 2 failed.

The plain Docker manifest of a list tag is the llamacpp build, not what a
Mac pulls. Read the right way, the first run finds two rows sized from the
wrong build: qwen3.5:9b (a Mac downloads 7,639,229,431 bytes, +16.6%) and
qwen3.8:27b (18,174,721,847, +2.4%).

.github/workflows/local-model-catalog.yml runs it Mondays at 06:00 UTC
and on dispatch. The check job installs with --ignore-scripts and holds a
read-only token; a separate job with issues: write and no checkout opens
or updates the one "Local model catalogue: weekly check" issue on drift,
comments on it on a clean week when it is open, and is skipped when the
check failed. Actions are pinned by commit.

docs/local-models.md gains "Keeping the catalogue current".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…irst

A row can pin its build only as `digest`, the value /api/tags reports
after the pull, and LocalModelEntrySchema must gain that optional field
before the weekly check can compare it. The script header and the docs
now say so, so the field cannot arrive under another name and be ignored.

Until it exists, the report's digest table names the build a Mac pulls
for each row and asks for the rows with measured figures first: their
memory figures describe one build, and without a digest a rebuild of
about the same size goes unnoticed. A switch between the mlx and the gguf
build is still caught by the size check (2% or more apart).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
agentSkillEnabled now reads an absent key as ON, the way
workspaceHooksEnabled does: only an explicit false turns it off. A new
session therefore finds the codeman skill in its folder without a
setting first, and with it the verbs that hand work to other CLIs and to
the local model on this Mac. Settings shows the switch on for the same
absent key.

The injection itself is unchanged: add-only, marker-owned, never through
a symlink, never for remote sessions or a plain shell. Turning the
setting off still removes nothing already written.

The quick-start test now pins the new default (injected on an absent
key, nothing after an explicit false), and the docs that said "default
off" say when that changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sendmark's comment and verbs.md 5.5 listed six external modes by name,
and test/agent-skill-mode-lists.test.ts refuses any partial list of them
(the one a ninth CLI would silently miss). Both now say what the class
is: a hook-less claude, a worker on the local model, and every external
agent mode but deepseek.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured on opencode 1.3.0 in a private tmux socket, with a fake
OpenAI-compatible endpoint streaming the reply a character or two at a
time: the marker was on the screen, but the stripped client stream read
D, spinner, O, spinner, N, ... because opencode redraws its spinner
between the streamed pieces, so wait-output never matched it. sendmark
then waited its whole timeout and answered 2 for a reply that had
finished, with the answer file already written (opencode's write tool
wrote it without asking).

The marker wait now runs in rounds of at most 10 s, and between rounds
the answer file is a way out too: a non-empty regular file whose
checksum held still for one round is read, with matched false. The
marker stays the fast path. The answer-file wait after a marker no
longer spins when the exit wait gets no answer, and treats an ended
wait as the worker gone.

The same run measured the readiness mark: with a fresh home and a
custom provider, the first composer bar came with the composer and a
prompt pasted with its Enter at that moment was submitted, so the docs
say so instead of "not measured". The example keeps its input in a
subfolder, apart from the case's CLAUDE.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Codeman Agent Skill setting is now on by default, and the per-case
copy it writes lists twice next to the Claude Code plugin (codeman and
codeman:codeman), the same as a codeman skill install would. The README
and the wiki now say so and tell plugin users to turn the setting off.

The README's local model bullet said the session "sends nothing off the
machine". The server keeps the runtime on loopback with its cloud
features off, but opencode itself is not configured offline (its model
catalogue and update checks are untouched), so the bullet now says what
the code does: the prompts and files go to a model on this Mac.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The opencode readiness and marker measurements were made on Linux, not
on a Mac, and the docs now say so. A worker whose pane ends after it
wrote its answer file but before the marker matched answers 3 with no
answer: the rc 3 row now says the file may still be in its folder. The
example's input folder is named input, so the prompt reads cleanly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l request

This reverts commit 0aa34da, which made the codeman agent skill default on here as well. That change lives in feat/agent-skill-default-on, together with the self-ignoring copy and the reference-folder symlink check it needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e Mac

The skill-on-by-default change ships in its own pull request with the self-ignoring copy, so this branch keeps the old default and its wording. The local models guide no longer says nothing leaves the machine: the prompts go to the local model, while the coding agent may still check for updates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pt cache for llama.cpp

runtimeEnv() now also sets OLLAMA_KV_CACHE_TYPE=q8_0, OLLAMA_FLASH_ATTENTION=1
and LLAMA_ARG_CACHE_RAM=1024. Read in the v0.40.15 source of the pinned
runtime: the first becomes --cache-type-k/-v of llama-server (17/32 of the
f16 cache), the second --flash-attn on (a quantized V cache needs it, and
"auto" or "off" would fail such a load), and llama-server inherits the third
from ollama serve's environment, which bounds its host prompt cache to
1 GiB instead of letting it grow by gigabytes in a session. The MLX runner
reads none of them.

The catalogue's llama.cpp rows are computed for these values, so the two
change together. Not yet measured on a Mac.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ow with its own figures

The catalogue had seven Qwen-family rows that all carried the memory
figures measured on one model (Qwen3.6 35B-A3B on the MLX runner), so a
9B or a dense 27B was judged with figures that do not describe it, two
rows had the wrong Mac download size, and every Mac below 32 GB got
"none".

Now eight rows, each with the runner a Mac really uses for its tag (a new
`runner` field: `mlx` or `llamacpp`) and memory figures of its own:

- the 35B-A3B MLX row keeps its measured figures and the tested pair;
- every llama.cpp row is computed from its model's config.json: a q8_0
  K/V cache for the whole context (what the runtime now sets), its
  recurrent or sliding-window state, 1.0 GB of runner and the 1 GiB host
  prompt cache;
- the one other MLX row (Ornith 1.5 122B) is computed the same way,
  scaled by what the measured row showed, plus the runner's 8 GiB of
  prefix snapshots.

Sizes are the registry's Mac pull totals (2026-11-08, lowercase ids
verified there). What each memory size gets: 8 GB none; 16 to 32 GB a 9B
(limited); 36 GB Qwen3.8 27B or the 35B-A3B as GGUF; 48 GB the tested
pair (or the 27B on a Max chip); 64 and 96 GB the 27B or the 35B-A3B;
128 to 512 GB Ornith 1.5 122B. Nothing but the one row was measured.

The catalogue test recomputes every computed figure from a table of
architectures and lets two rows share figures only when they share the
architecture and the runner. The rule tests of recommend() now run on a
frozen copy of the 2026-11-06 catalogue, so a weekly catalogue update
changes only the per-memory table and the real-Mac tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…comes from

docs/local-models.md gains "Which model fits which Mac": the answer of the
stock catalogue for every memory size from 8 to 512 GB, why the chip
changes it at 36 to 96 GB, and why nothing is offered below 65,536 tokens.
"How much memory a model needs" explains the two runners (llama.cpp
allocates its K/V cache at load, MLX grows it and keeps up to 8 GiB of
prefix snapshots), the three runtime settings the llama.cpp rows assume,
and per row where its memory and speed figures come from. The check after
the download now says that for an MLX build it proves only the weights,
and "What to know" says most figures are calculated, not measured.

CLAUDE.md: every row names its runner and carries its own figures, and
the llama.cpp rows and runtimeEnv() change together.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ss's launch check

The new GGUF rows carry lowercase quantization tags (`-q4_k_m`); pin that
the launch path's model id test (`cliAcceptsCustomModelId()`) accepts every
stock id for every harness the catalogue offers, so a row the chooser shows
can always be started.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…at the check proves

The 48 GB row now says the test was a short task and that MLX may keep up
to 8 GiB more in a long session, which brings the pair to the edge of the
budget. The check after the download says what it proves for each runner,
from the runtime's source: for a GGUF build both of its figures are the
buffers llama-server reports allocating, so the check covers the whole
load whatever the cache type; for an MLX build only the weights. And the
docs say where a new row's architecture goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Q8 27B row counts a prediction layer its GGUF may or may not carry
(not checked, careful side), and the macOS notice on the runtime's page is
about Laguna XS 2.1; Laguna S 2.1 is set aside for its open runtime issues.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and bound its checkpoints

The llama.cpp memory rule of the catalogue left out two allocations that the llama.cpp the
runtime bundles (b12111, the runtime's LLAMA_CPP_VERSION) makes for every GGUF row:

- Context checkpoints. For a model whose state cannot be rolled back (the recurrent state of
  the Qwen-family rows, the sliding window of North Mini Code), llama-server keeps host copies
  of that state, two or three per request, up to 32 per slot by default
  (tools/server/server-context.cpp create_checkpoint). For the 27B that is 32 x 157 MB = 5 GB,
  none of which was counted. The runtime now sets LLAMA_ARG_CTX_CHECKPOINTS=8 (read by
  llama-server from its environment like LLAMA_ARG_CACHE_RAM; 8 was llama.cpp's own default
  before it went to 32), and every llamacpp row counts 8 copies of its state.
- Multi-token prediction. The two -mtp tags and qwen3.8:27b-q8_0 carry a prediction layer
  (nextn_predict_layers 1, read from the first bytes of each GGUF; the 9B tags and North Mini
  Code have none). The runtime then always passes --spec-type draft-mtp --spec-draft-n-max 4,
  so llama.cpp keeps 4 rollback copies of the recurrent state (need_n_rs_seq(),
  llama-memory-recurrent.cpp) and the prediction layer's cache in the draft context's cache
  type, f16 by default. The rows counted one state and that layer at q8_0.

What moves (test/local-model-recommend.test.ts, "every memory size", re-pinned):

- 16 GB: limited -> none. MiMo 9B at 65,536 tokens needs 10.2 GB, 0.987 of the 10.4 GB
  estimate. With the budget the Mac mini measured (10.7 GB after the reserve) it is offered,
  0.954, tight; that test is kept. With 4 checkpoints instead of 8 it would be 0.966 on the
  estimate: this size sits on the edge, and only a run on such a Mac settles it.
- 18 GB: the same pick, 0.964 instead of 0.928.
- 36 GB: the same picks, 0.966 (27B on M4/M5 Max) and 0.976 (35B-A3B GGUF on M3).
- 48 GB on a Max chip: the 27B at 262,144 tokens is tight (0.853) instead of good.
- 32 GB with a measured budget of 0.74: North Mini Code at 131,072 tokens (was 262,144), the
  27B at 65,536 on a Max chip (was 131,072).
- Every MLX row, and everything from 64 GB up, is unchanged.

test/local-model-catalog.test.ts recomputes the new rule from the architecture table and holds
the catalogue to LLAMA_ARG_CTX_CHECKPOINTS. The catalogue header, runtime.ts, the types,
docs/local-models.md, CLAUDE.md and architecture-invariants say the same. Read in the source,
not measured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pulls

The catalogue that names each row's runner (feat/local-model-catalog-v2: `runner` is `mlx` or
`llamacpp`, and a row's memory figures follow that runner's rule) needs the weekly check to
notice when a tag starts serving a Mac another runner: a tag that gains an mlx child, or a
list that loses it. The size check usually catches that, but not when the two builds come out
about the same size, and the figures are wrong either way.

classifyModel() now raises a `runner` finding when the row names a runner and the build a Mac
selects runs on another one. catalogueRunner() maps the registry's names into the catalogue's:
`mlx` stays `mlx`; `llamacpp`, and `ggml` (what Ollama calls a GGUF manifest that names no
runner), are `llamacpp`, because Ollama 0.40.15 serves every GGUF build through llama-server.
A catalogue without the field (the one on this branch today) is never held to a runner.

Run live on 2026-11-08 against the v2 catalogue's eight rows: exit 0, every runner matches.
Tests: the mapping, the finding (both directions, ggml, an unknown runner, a row without the
field), the report text, and runCheck against the fake registry with a plain GGUF manifest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rule

The weekly check's report and docs ask for an optional `digest` on each catalogue row, but the
catalogue keeps no digest on purpose: test/local-model-catalog.test.ts ("holds no download
data") refuses a `sha256` anywhere in it. Say so where the digest contract is described, so a
maintainer reads it as a decision for the catalogue, not a quick schema change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The previous commit counted 4 rollback copies of the recurrent state for every GGUF with a
prediction layer, the runtime's default `draft_num_predict`. A tag can set its own in its
params layer, and the runtime then passes that as `--spec-draft-n-max`, which is what
`need_n_rs_seq()` turns into rollback copies. Read on 2026-11-08:

- qwen3.6:35b-a3b-mtp-q4_k_m: draft_num_predict 2, so 2 copies (was counted as 4).
- qwen3.8:27b-mtp-q4_k_m: draft_num_predict 4.
- qwen3.8:27b-q8_0: none in its params, so the runtime's 4.

The 35B-A3B GGUF row's overhead drops by 132 MB: 0.971 instead of 0.976 on a 36 GB M3 Pro or
M3 Max, the same pick. test/local-model-catalog.test.ts now carries the drafted tokens per row
instead of a yes/no prediction layer.

Also states what was verified about llama.cpp's checkpoint default: 8 from b6800 to b8200,
32 from b8250 together with a checkpoint every 8,192 prompt tokens (common/common.h at those
tags), in place of an inferred reason. Checked in the b12111 source as well: the draft context
of a Qwen-family MTP model caches the prediction block alone (`mtp_on_hybrid_qwen`, layers at
or above n_layer()), and the main context leaves it out, so the row counts that layer once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Ark0N
Ark0N merged commit 8a54ded into master Nov 8, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants