Skip to content

feat: Claude advisor tool support (per session, App Settings default, skill workers) - #530

Merged
Ark0N merged 2 commits into
masterfrom
feat/advisor-model
Oct 4, 2026
Merged

Ark0N merged 2 commits into
masterfrom
feat/advisor-model

Conversation

@Ark0N

@Ark0N Ark0N commented Oct 4, 2026

Copy link
Copy Markdown
Owner

Claude Code has an experimental advisor tool (docs): the session's main model consults a second, stronger model at decision points, before committing to an approach, when an error keeps coming back, and before it declares a task done. Claude decides when to call it. A typical setup is a Sonnet main model with an Opus or Fable advisor, which costs less than running the stronger model the whole time.

Until now Codeman had no way to turn it on for a session. This PR adds it as a per-session setting with an App Settings default, and lets agents using the codeman skill give their workers one.

What changes

Sessions and API

  • New advisorModel field on POST /api/sessions, POST /api/quick-start and POST /api/ralph-loop/start. Accepted values: fable, opus, sonnet, or a full model id in one of those families (claude-opus-5-5). Haiku is refused because it can call an advisor but can never act as one.
  • Stored on the session and persisted to state.json, so respawns, boot restore and reboot restore all keep it.
  • Remote and docker quick-starts refuse it with a 400, exactly as they already refuse effort (neither crosses ssh or into the container).

UI

  • App Settings, Models, a new Advisor segment under Thinking effort: Default / Sonnet / Opus / Fable. Synced across devices as claudeAdvisorModel.
  • Run, resuming a past conversation and the Ralph wizard all send it. Default sends nothing, so whatever the user picked with /advisor in Claude Code stays in charge.

Codeman skill

  • spawn_worker and spawn_workers build the quick-start body themselves, so an agent could not give a worker an advisor without hand-rolling the call and losing the readiness checks. Setting CODEMAN_WORKER_ADVISOR=opus now adds it for every claude worker they start; other modes ignore it.
  • A bad value fails the spawn with the server's own error. An older server silently drops the unknown field, so spawn_worker reads it back and warns on stderr instead of handing back a worker without the advisor it asked for.
  • The preamble changed, so CODEMAN_PREAMBLE is bumped to 1.33.4 (stale cached copies get rewritten). SKILL.md's heredoc and the plugin mirror are synced.

Why --settings and not --advisor

Claude Code accepts the advisor both as a --advisor <model> flag and as an advisorModel settings key. They fail differently (measured on 2.1.289):

claude --advisor haiku claude --settings '{"advisorModel":"haiku"}'
result Error: The model "haiku" cannot be used as an advisor., exit 1 starts normally

The flag also exits for a Fable advisor before the account's usage-credit consent, or a model outside an org's availableModels. In Codeman that would mean a dead pane on every spawn and respawn. The settings key degrades instead: Claude just doesn't attach an advisor it can't use. So the value rides the launch's one --settings JSON object, merged with ultracode and the plan-usage statusLine exporter (Claude Code reads only one --settings flag). A launch without an advisor produces a byte-identical command to before. Both launch paths are covered: the tmux template and the direct-PTY fallback.

It stays a soft default: /advisor switches it or turns it off inside the running session.

Testing

  • New test/advisor-model.test.ts: the allowlist (including quote and injection attempts), the merged --settings object read back through a real shell, the resume variant, the direct-PTY args, the unchanged command without an advisor, persistence, and all four schemas. test/agent-skill.test.ts and test/plugin-manifest.test.ts pass with the bumped preamble.
  • Full npm test gate: 441 files, 8531 tests passed. Typecheck, lint, format and the frontend checks are clean.
  • End to end on an isolated instance (CODEMAN_INSTANCE, its own tmux socket and cases dir), with real Claude Code 2.1.289:
    • The settings segment saves, survives a reload, and Run creates a session with advisorModel: "opus".
    • The pane's claude process started with --settings {"advisorModel":"opus","statusLine":...}, and Claude's own /advisor picker in that pane showed Opus 5.5 ✔.
    • Killing that claude and restarting the server: boot restore found the dead pane, respawned it, and the new process still carried the advisor.
    • The real preamble.sh: CODEMAN_WORKER_ADVISOR=fable spawn_workers adv-a adv-b gave both workers fable, a worker spawned without the variable got none, and CODEMAN_WORKER_ADVISOR=haiku spawn_worker failed cleanly with the INVALID_INPUT message. The old-server warning was checked against a 1.33.3 server with a read-only GET.

Notes

  • The advisor needs the Anthropic API (not Bedrock or Vertex) and uses extra tokens when Claude consults it. An advisor ranked below the session's main model is accepted but not attached by Claude Code, so on an Opus 5.5 session only Opus or Fable do anything.
  • No changeset here; it goes in with the release.
  • Found along the way, not fixed here: when two sessions are created in the same millisecond while statusline-exporter.sh needs rewriting (a fresh data dir, or after a release that changes it), both writers use the same temp name (pid.Date.now().tmp), one rename fails with ENOENT, and that session silently falls back from tmux to a direct PTY. Seen once during the spawn_workers run above; the fix is a random temp suffix like the skill writer in the same file already uses. Worth its own small PR.

claude added 2 commits October 4, 2026 19:05
…s default

Claude Code's advisor tool (code.claude.com/docs/en/advisor) lets the session's
main model consult a second, stronger model at decision points: before
committing to an approach, on a recurring error, and before declaring a task
done. Codeman can now start claude sessions with one.

- `advisorModel` field on POST /api/sessions, /api/quick-start and
  /api/ralph-loop/start (fable, opus, sonnet or a full model id in those
  families; haiku cannot advise and is refused). Stored on the session and
  persisted, so respawn, boot restore and reboot restore keep it. Remote and
  docker quick-starts refuse it, as they refuse effort.
- App Settings, Models, "Advisor" segment (Default / Sonnet / Opus / Fable),
  synced as `claudeAdvisorModel`. Run, resume and the Ralph wizard send it.
  Default sends nothing, leaving the CLI's own /advisor choice in charge.
- Carried as the `advisorModel` key in the launch's single --settings JSON,
  merged with ultracode and the statusLine exporter, never the --advisor
  flag: `claude --advisor haiku` exits 1 at launch, which would leave a dead
  pane on every respawn, while the settings key degrades to no advisor. A
  launch without an advisor is byte-identical to before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…dvisor

spawn_worker builds its quick-start body itself ({caseName, mode,
parentSessionId}), so an agent driving the skill had no way to give a worker
the advisor without hand-building the call and losing the readiness ladder,
hooks vetting and trust-dialog fallback. Setting CODEMAN_WORKER_ADVISOR
(fable / opus / sonnet) now adds `advisorModel` for every claude worker that
spawn_worker or spawn_workers starts; other modes ignore it.

A refused value fails the spawn with the server's INVALID_INPUT message. A
server without advisor support drops the field silently (the schema is not
strict), so spawn_worker reads it back and says so on stderr.

The preamble changed, so CODEMAN_PREAMBLE is bumped to 1.33.4 and stale
cached copies are rewritten instead of silently ignoring the variable.
SKILL.md's heredoc and the plugin mirror are synced.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Ark0N
Ark0N merged commit b8038a5 into master Oct 4, 2026
2 checks passed
Ark0N pushed a commit that referenced this pull request Oct 4, 2026
…dvisor (#514, #515, #530 landing)

Maintainer merge-time fixes for the three PRs that landed together on the
session create / launch / persistence path.

#514 findings (bot verdict merge-with-fixes):
- minor, fixed: SessionState.model was published and persisted for every
  mode, so a codex/opencode cron session reported the app-wide Claude
  default it never ran on. toState() now emits it only where the new
  cliTakesSessionModel() holds (registry capability model.source ===
  'claude-settings-file', no CLI id branch). POST /api/sessions uses the
  same helper for its non-claude refusal, so refusal and publication cannot
  drift. Recovery then hands back undefined for other modes on its own.
- nit, fixed: the `model` schema admitted a leading dash (and '.', '[').
  The first character must now be a letter or digit; still a subset of the
  registry's model-claude pattern, so nothing accepted is refused at launch.
- nit, fixed (reject, the consistent choice): `model` with
  attachRemoteSession was silently dropped. Now a 400 INVALID_INPUT, as
  #514 does for non-claude CLIs and quick-start does for remote cases.
  advisorModel (#530) gets the same refusal there. effort and envOverrides
  keep their older silent ignore on that branch so no existing caller breaks.

#515 finding (bot verdict merge, one nit):
- nit, fixed: the types/session.ts @fileoverview described CodexConfig as
  (model, resumeSessionId); it now lists reasoningEffort, bypass,
  animations and renderMode too.

Audit of the merged combination (not reviewed before):
- The conflict resolutions in session.ts (toState), types/session.ts,
  reboot-restore-routes.ts, server.ts (restoreMuxSessions), CLAUDE.md and
  skills/codeman/reference/endpoints.md (+ plugin mirror) keep both sides
  correctly; nothing was lost or doubled.
- A claude session with both `model` and `advisorModel` launches with
  `--model <id>` and ONE merged `--settings` JSON (ultracode + advisorModel,
  or advisorModel beside `--effort <level>`), on the tmux template
  (including the resume || new variant and with the statusLine exporter)
  and on the direct-PTY fallback. Both values (and effort) survive
  restoreMuxSessions onto a dead pane, a reboot restore into a fresh pane,
  and restartCli/dead-pane respawn via _buildRespawnPaneOptions.
- quick-start and ralph-loop take no per-session `model` (matching #514's
  scope, POST /api/sessions only) and launch on the app-wide default, which
  toState now persists for claude, so recovery stays consistent.
- No defect found in the combination beyond the findings above. Noted, not
  changed: advisorModel is still published for any mode a caller sends it
  with (launch-inert there; the UI and skill send it for claude only).

Tests: test/session-model-recovery.test.ts pins the pair through both
recovery shapes for effort ultracode/high/none, the recovery constructors'
fields, the tmux-manager builder hop, and the codex/opencode/shell
non-publication; test/advisor-model.test.ts pins the launch lines and a
real direct-PTY Session's pty.spawn argv; the route test covers flag-shaped
models, attach refusals and the published fields. Docs: SessionState.model
docstring, the reboot-restore-registry header, the golden test comment and
the CLAUDE.md model/advisor bullets.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ark0N pushed a commit that referenced this pull request Oct 4, 2026
…#520, #521, #522, #523, #524, #530, #531)

One consolidated minor changeset with the Thanks block first; the four contributor changesets (#520, #521, #522, #523) are folded into it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants