fix(cli): a running server notices a CLI the installer just added

.github/workflows/ci.yml

@@ -80,6 +80,35 @@ jobs:
             CLI_DETECT_DONE=""; detect_all_clis
             offer_ai_cli_install >/dev/null 2>&1
             echo "bash $BASH_VERSION: skipping the AI CLI install menu continues"
+            # --ai-cli answers the same menu with no terminal: `none` carries on, and a
+            # name runs that one entry (bash 3.2 cannot expand an empty offer list under
+            # set -u, which is the guard the loop over it has to get right).
+            has_tty() { return 1; }
+            AI_CLI_CHOICE=none
+            offer_ai_cli_install >/dev/null 2>&1
+            AI_CLI_CHOICE=codex
+            k=0; while [[ $k -lt ${#CLI_INSTALL_CMD_TRUSTED[@]} ]]; do
+              CLI_INSTALL_CMD_TRUSTED[$k]=""
+              k=$((k + 1))
+            done
+            offer_ai_cli_install >/dev/null 2>&1
+            echo "bash $BASH_VERSION: --ai-cli answers the menu, also with nothing to offer"
+            # What the desktop app runs: preflight on stdout, and only JSON there. These
+            # child runs must not inherit CODEMAN_INSTALL_SH_LIB, or they stop at the
+            # sourcing guard and print nothing. Each check ends in `|| exit 1`, because
+            # bash 3.2 does not let a false [[ ]] trip set -e.
+            out=$(env -u CODEMAN_INSTALL_SH_LIB bash /w/install.sh preflight --no-admin)
+            [[ "$out" == "{\"schema\":1,"* && "$out" == *"]}" ]] || { echo "preflight printed: $out"; exit 1; }
+            json_string "a\"b\\c" >/tmp/json-out
+            [[ "$(cat /tmp/json-out)" == "\"a\\\"b\\\\c\"" ]] || { echo "json_string printed: $(cat /tmp/json-out)"; exit 1; }
+            echo "bash $BASH_VERSION: preflight prints JSON"
+            # How the desktop app adds an agent: a real run of the subcommand, which stops
+            # on the missing npm before it could install anything.
+            if out=$(env -u CODEMAN_INSTALL_SH_LIB bash /w/install.sh agent codex --no-admin --progress 2>&1); then
+              echo "agent ran with no npm to run"; exit 1
+            fi
+            [[ "$out" == *"@@codeman phase agent"* && "$out" != *"@@codeman phase done"* ]] || { echo "agent printed: $out"; exit 1; }
+            echo "bash $BASH_VERSION: agent stops without saying done"
           '
           # Issue #382: the dsh identity probe builds an OPTIONAL `timeout` prefix as an
           # array, and on stock macOS there is no `timeout`, so the array is empty and the

CLAUDE.md

@@ -319,7 +319,7 @@ One module per domain in `src/web/routes/` (plus a barrel; `ls src/web/routes/`
 
 ## State Files
 
-All in `~/.codeman/`: `state.json` (sessions, settings, respawn, orchestrator, cron jobs/runs, owner tab layouts, case default groups), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` + `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart), `docker-env-applied.json` (Compose deployment only: sha256 of the Dockerfile + compose file the running container was built from, written by `Start-Codeman.sh`, read by the self-updater's environment gate), `docker-build-source.json` (Compose deployment only: the checkout's HEAD commit and `package-lock.json` hash the `codeman-node-modules`/`codeman-dist` volumes currently reflect, written by both `Start-Codeman.sh` and a successful in-place self-update, compared to detect and refresh a volume left stale by an externally-triggered rebuild), `linked-cases.json`, `webviews.json` (saved web-tab dashboard URLs), `remote-hosts.json` + `remote-cases.json`, `docker-hosts.json` + `docker-cases.json` + `docker-exports/`, `subagent-window-states.json` + `subagent-parents.json` (subagent window layout, GET/PUT `/api/subagent-window-states`/`-parents`), `hook-secret` (per-instance), `users.json` (multi-user, mode 0600) + `admin-audit.jsonl`, `intents.json` (Read My Mind intent profiles, mode 0600), `agent-inbox.json` (agent inbox snapshot, only while `agentInboxPersistEnabled` is on, mode 0600), `edit-history/<id>.json` (Edit history panel, one store per session, every diff an agent wrote, mode 0600), `webhook.json` (webhook notifications: enabled/service/scope plus the ntfy/Slack/Discord URL, a bearer secret, so mode 0600, kept out of `settings.json` and never returned by `/api/webhook`, which is admin-only in multi-user mode), `discord.json` (Discord bridge: bot token, a bearer secret, so mode 0600, never returned by `/api/discord`, admin-only in multi-user mode) + `discord-bridge.json` (sessionId to Discord thread id), `config-backup-id` (the random id config backups record as each manifest's `origin`, so instances sharing a backup folder keep to their own snapshots; mode 0600, never tracked or restored, and deleting it unlists this instance's earlier snapshots), `backups/config/` (the default config-backup folder, moved by `configBackupDir`: `cfg-YYYYMMDD-HHMMSS/` snapshots, 0700 with 0600 files, each with a `manifest.json` of SHA-256s, the origin and the retention in force (`keepCount`/`keepDays`), which a pass falls back to when `settings.json` lacks a retention key), `certs/` (self-signed TLS for `--https`), `.env` (CODEMAN_USERNAME/PASSWORD fallback for the `codeman attach` CLI), `install.log` (installer step output, written by `install.sh`'s `run_step`) `tailscale-rename` (the node name before `install.sh` renamed it, so uninstall can offer it back) and `runtime/` (the Node.js and tmux `install.sh` downloaded on its no-admin route, linked from `~/.local/bin`; all three installer-route only). Transient: `self-update-runner.sh`. Multi-user case spaces live OUTSIDE the data dir at `~/codeman-users/<username>/cases` (shared across instances like `~/codeman-cases`, override `CODEMAN_USER_SPACES_DIR`).
+All in `~/.codeman/`: `state.json` (sessions, settings, respawn, orchestrator, cron jobs/runs, owner tab layouts, case default groups), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` + `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart), `docker-env-applied.json` (Compose deployment only: sha256 of the Dockerfile + compose file the running container was built from, written by `Start-Codeman.sh`, read by the self-updater's environment gate), `docker-build-source.json` (Compose deployment only: the checkout's HEAD commit and `package-lock.json` hash the `codeman-node-modules`/`codeman-dist` volumes currently reflect, written by both `Start-Codeman.sh` and a successful in-place self-update, compared to detect and refresh a volume left stale by an externally-triggered rebuild), `linked-cases.json`, `webviews.json` (saved web-tab dashboard URLs), `remote-hosts.json` + `remote-cases.json`, `docker-hosts.json` + `docker-cases.json` + `docker-exports/`, `subagent-window-states.json` + `subagent-parents.json` (subagent window layout, GET/PUT `/api/subagent-window-states`/`-parents`), `hook-secret` (per-instance), `users.json` (multi-user, mode 0600) + `admin-audit.jsonl`, `intents.json` (Read My Mind intent profiles, mode 0600), `agent-inbox.json` (agent inbox snapshot, only while `agentInboxPersistEnabled` is on, mode 0600), `edit-history/<id>.json` (Edit history panel, one store per session, every diff an agent wrote, mode 0600), `webhook.json` (webhook notifications: enabled/service/scope plus the ntfy/Slack/Discord URL, a bearer secret, so mode 0600, kept out of `settings.json` and never returned by `/api/webhook`, which is admin-only in multi-user mode), `discord.json` (Discord bridge: bot token, a bearer secret, so mode 0600, never returned by `/api/discord`, admin-only in multi-user mode) + `discord-bridge.json` (sessionId to Discord thread id), `config-backup-id` (the random id config backups record as each manifest's `origin`, so instances sharing a backup folder keep to their own snapshots; mode 0600, never tracked or restored, and deleting it unlists this instance's earlier snapshots), `backups/config/` (the default config-backup folder, moved by `configBackupDir`: `cfg-YYYYMMDD-HHMMSS/` snapshots, 0700 with 0600 files, each with a `manifest.json` of SHA-256s, the origin and the retention in force (`keepCount`/`keepDays`), which a pass falls back to when `settings.json` lacks a retention key), `certs/` (self-signed TLS for `--https`), `.env` (CODEMAN_USERNAME/PASSWORD fallback for the `codeman attach` CLI), `install.log` (installer step output, written by `install.sh`'s `run_step`) `tailscale-rename` (the node name before `install.sh` renamed it, so uninstall can offer it back), `cli-installed` (empty, touched by `install.sh` when it installs a CLI: a running server reads only its mtime, and a change ends that CLI's negative-cache backoff) and `runtime/` (the Node.js and tmux `install.sh` downloaded on its no-admin route, linked from `~/.local/bin`; all four installer-route only). Transient: `self-update-runner.sh`. Multi-user case spaces live OUTSIDE the data dir at `~/codeman-users/<username>/cases` (shared across instances like `~/codeman-cases`, override `CODEMAN_USER_SPACES_DIR`).
 
 **Generated top-level dirs** (all gitignored — don't edit or commit): `dist/` (esbuild output), `out/`, `coverage/`, `test-results/`, `tmp/`, `screenshots-echo-diag/`. The committed gesture bundle (`src/web/public/gesture/gesture-codeman.js`) IS tracked, but its runtime wasm/model assets (`src/web/public/gesture/wasm/`, `*.task`) are fetched and gitignored.
 
@@ -384,6 +384,6 @@ Two constraints worth knowing before you touch them: the env-derived PTY buffer
 
 ## Scripts & Tunnel
 
-**`install.sh`** (repo root) is the public `curl | bash` installer: it installs Node/tmux/git/build tools, clones to `~/.codeman/app`, builds, and offers a systemd/launchd service, asking every question BEFORE the unattended build (log in `~/.codeman/install.log`). Network access is Tailscale / LAN / local-only; re-runs preserve the existing binding AND password (`read_existing_binding()`). ⚠️ Compose the hand-start env only in `start_command_hint`/`export_bind_env`, and call `stop_background_helpers` before `exec`. ⚠️ Tailscale rename is opt-in (default NO, never under `--yes`/non-interactive); NEVER `tailscale serve reset`, touch a mapping it did not create, run `tailscale funnel` or advertise a Service (`test/install-sh-invariants.test.ts`). ⚠️ **No-admin route** (`--no-admin` / `CODEMAN_NO_ADMIN`, taken by itself when the account has no administrator rights): Node.js (nodejs.org, checked against `SHASUMS256.txt`) and tmux (`tmux/tmux-builds`, digests pinned in `tmux_build_sha256`: bump `TMUX_BUILD_VERSION` and all four together) go into `~/.codeman/runtime`, and without a working git the source is an archive that updates by a re-run (built beside the old tree, swapped in after the build). Nothing in that section may run `sudo`, Homebrew or `xcode-select --install`, a download is verified before it is unpacked, the route is never decided by `sudo -v`, `/usr/bin/git` on a Mac is never executed before `xcode-select -p` proves the developer tools, and `no_admin_blocker` stops BEFORE the first download for what it cannot supply (a Linux toolchain, macOS below 15). ⚠️ Stay **bash 3.2** clean (no `declare -A`, `mapfile`, namerefs, `${x,,}`, here-strings, empty-array expansion under `set -u`). ⚠️ Execute only commands from the generated CLI block (`CLI_INSTALL_CMD_TRUSTED`, `npm run generate:cli-catalog`). → [architecture-invariants#installsh-the-public-installer](docs/architecture-invariants.md#installsh-the-public-installer)
+**`install.sh`** (repo root) is the public `curl | bash` installer: it installs Node/tmux/git/build tools, clones to `~/.codeman/app`, builds, and offers a systemd/launchd service, asking every question BEFORE the unattended build (log in `~/.codeman/install.log`). Network access is Tailscale / LAN / local-only; re-runs preserve the existing binding AND password (`read_existing_binding()`). ⚠️ Compose the hand-start env only in `start_command_hint`/`export_bind_env`, and call `stop_background_helpers` before `exec`. ⚠️ Tailscale rename is opt-in (default NO, never under `--yes`/non-interactive); NEVER `tailscale serve reset`, touch a mapping it did not create, run `tailscale funnel` or advertise a Service (`test/install-sh-invariants.test.ts`). ⚠️ **No-admin route** (`--no-admin` / `CODEMAN_NO_ADMIN`, taken by itself when the account has no administrator rights): Node.js (nodejs.org, checked against `SHASUMS256.txt`) and tmux (`tmux/tmux-builds`, digests pinned in `tmux_build_sha256`: bump `TMUX_BUILD_VERSION` and all four together) go into `~/.codeman/runtime`, and without a working git the source is an archive that updates by a re-run (built beside the old tree, swapped in after the build). Nothing in that section may run `sudo`, Homebrew or `xcode-select --install`, a download is verified before it is unpacked, the route is never decided by `sudo -v`, `/usr/bin/git` on a Mac is never executed before `xcode-select -p` proves the developer tools, and `no_admin_blocker` stops BEFORE the first download for what it cannot supply (a Linux toolchain, macOS below 15). ⚠️ Stay **bash 3.2** clean (no `declare -A`, `mapfile`, namerefs, `${x,,}`, here-strings, empty-array expansion under `set -u`). ⚠️ Execute only commands from the generated CLI block (`CLI_INSTALL_CMD_TRUSTED`, `npm run generate:cli-catalog`). ⚠️ Codeman Desktop drives it with no terminal (`--no-admin --progress --ai-cli <id>|none`, `install.sh preflight`, `install.sh agent <id>` to add one agent next to a running Codeman, `packages/desktop/src/gui_install.rs`): `cli_offerable` is the one rule for what can be installed, `preflight` stdout is JSON only, `agent` says `done` only when THAT agent landed, and the `progress_phase` order is pinned against `connect.html`, with `done` last. → [architecture-invariants#installsh-the-public-installer](docs/architecture-invariants.md#installsh-the-public-installer)
 
 Other key scripts: `scripts/tmux-manager.sh` (safe tmux mgmt), `scripts/tunnel.sh [quick|named] start|stop|status|url` (quick = random trycloudflare URL, default; `named setup|enable` = fixed-hostname tunnel via `scripts/codeman-tunnel-named.service`; bare `start|stop|url` still means quick), `scripts/run-beta.sh` (isolated beta instance), `scripts/build-agent-image.mjs` (docker base image), `scripts/self-update.sh` (detached updater). Production services: `scripts/codeman-web.service`, `scripts/codeman-tunnel.service`. **Always set `CODEMAN_PASSWORD`** before exposing via tunnel.

docs/architecture-invariants.md

@@ -1303,6 +1303,8 @@ On this route Node.js comes from nodejs.org (`latest-v<major>.x/SHASUMS256.txt`
 
 ⚠️ `no_admin_blocker` runs before the first download and stops for what this route cannot supply: a C/C++ toolchain on Linux (node-pty 1.1.0 compiles there; a node-pty with Linux prebuilds is what would lift this, and it is a dependency decision, not an installer one), musl, a platform nodejs.org or tmux-builds has no build for, and macOS below 15 (the tmux build's deployment target). ⚠️ Nothing between the "No-admin route" header and "End of the no-admin route" may run `sudo`, `ensure_sudo`, `run_as_root`, Homebrew or `xcode-select --install`; `run_as_root` itself refuses while `NO_ADMIN=1`, so a missed call site fails instead of prompting.
 
+**Driving the installer from a program** (`--ai-cli`, `--progress`, `install.sh preflight`, 2026-11-06). Codeman Desktop installs with no terminal: it runs `install.sh --no-admin --service --progress --ai-cli <id>|none` with `CODEMAN_NONINTERACTIVE=1` and its stdin closed, and asks the two questions in its own window (`packages/desktop/src/gui_install.rs`). `--ai-cli` answers the agent menu, so the one thing `CODEMAN_NONINTERACTIVE` used to decide by itself (the first entry of the catalogue) is the user's. ⚠️ `cli_offerable` is the ONE rule for what can be installed here: the menu, `--ai-cli` and `preflight`'s `installable` all read it, so the app's list and the menu cannot disagree. A name that is not offerable here warns and carries on, and never falls back to another CLI; a typo dies in `parse_flags` before anything runs. `install.sh agent <id>` installs one CLI and nothing else, for Codeman Desktop's *Add a coding agent*: no service, no source, no build, so it may run next to a live server, and an agent already there is a success that runs nothing. ⚠️ It calls `run_offered_cli`, checks `CLI_FOUND_PATH` of that one entry itself and only then prints `done`: `install_offered_cli`'s gate is fatal only when NO CLI is left, which is right for a first install and would report success here, where another CLI is usually present. ⚠️ The app passes `--no-admin` to both `preflight` and `agent`, because that flag decides what `cli_offerable` allows; pinned in `gui_install.rs`. ⚠️ The app runs it from a folder of its own (`Run::Agent`), so `ps` never reads an agent run as "Codeman is being installed". A running server remembers a missing CLI for up to 5 minutes (`cliResolveRetryDelayMs`), and Run reads that memory, so `mark_cli_installed` touches `cli-installed` in the data dir and `resolve()` leaves the backoff when that file's mtime changes. ⚠️ That check is one `stat` inside the backoff branch and buys one run of the chain. Never replace it with a re-walk of PATH during the backoff: that puts a synchronous stat of every PATH entry on the request path (#516) and executes candidates already refused. The app never restarts the service for this. Only `install.sh` writes the marker, always under `~/.codeman`, so a CLI installed by hand, or another `CODEMAN_INSTANCE`, waits out the backoff as before, and so does a server older than this. ⚠️ `preflight` prints only JSON on stdout and changes nothing; `schema` moves only when a field changes meaning, because the app ships on its own and refuses a number it does not know. Fields may be added freely. ⚠️ `progress_phase` writes to stderr, not stdout, because a function can run inside a `$( )` that captures stdout. The phases must stay in the order of `PHASES` in `connect.html` (pinned by test), `done` must stay the last thing `main()` reports, since a run that ends without it failed, and an unknown phase is ignored by the app, so a new phase needs no new app. ⚠️ `error()` carries the text as `@@codeman error`, and the app shows the FIRST one: the ones after it are consequences (`Installation failed. Partial...`).
+
 ⚠️ **git is judged by whether it works, never by `command -v`.** On a Mac without the Xcode Command Line Tools `/usr/bin/git` is a stub that opens Apple's install dialog, so `check_git` judges that path by `xcode-select -p` and never executes it; any other git is asked for `--version`. `scripts/git-hooks.mjs` has the matching guard (no `.git`, no `git` call), because postinstall runs it on every `npm install`. Without a working git the source is an archive (`fetch_source_archive`, GitHub's `archive/<ref>.tar.gz` or `CODEMAN_TARBALL_URL`) and the install directory carries the marker `.install-source`, which `detectInstallKind` reads as `archive` (the name is pinned across both files by `test/install-sh-invariants.test.ts`). ⚠️ An archive install is replaced as a WHOLE, never patched in place: nothing says which files a newer version dropped, and a stale `src/*.ts` would be compiled. The new tree is fetched and built in `<install dir>.new` and `finish_source_swap` moves it into place only after `npm run build` succeeded, so a failed update leaves the running install untouched; a re-run on a machine that has gained a working git stages a clone instead, which turns the install back into a git one. ⚠️ `download`/`download_to_stdout` resolve `DOWNLOADER` themselves (`require_downloader`): only `main()` used to set it, so a subcommand that downloads died on `set -u` with the message swallowed by the step log.
 
 ## Agent inspection and guarded submission