Repository navigation
fix(test): make a bare vitest invocation use the isolated CI config - #660
Conversation
`npx vitest run` with no --config found no config file, so test/setup.ts never ran and HOME was not redirected: the suite wrote fixtures (clis.json with a test entry, users.json, webhook.json, intents.json, .env, settings.json values) into the developer's real ~/.codeman, and a running Codeman then loaded them on its next restart. A root vitest.config.ts re-exports config/vitest.ci.config.ts, so the default invocation is the isolated one; npm test and the browser/mobile/perf scripts keep passing their own --config. The shared temp-HOME teardown also retries on ENOTEMPTY, which failed whole files intermittently after every test had passed.
CLAUDE.md and Contributing said raw npx vitest skips the config; state what happened when it did (fixtures written into the real ~/.codeman) and that the root vitest.config.ts now makes a bare run isolated, with the test that pins it.
maxRetries alone was not enough: on a machine with a real agent CLI installed, a test's code path probes or spawns `claude`, which flushes its own ~/.claude.json and ~/.claude/backups/ into the temp HOME a moment after the test ends, so rmSync raced it and failed the whole file with ENOTEMPTY after every test had passed (about one run in three for session-routes-claude-model; CI has no CLI). Retry the removal for up to ~5s, and if it never goes quiet leave the folder to the exit backstop instead of failing a file that passed. 12 consecutive runs of that file: no failure.
safeRmHomeTree and isUnderTestHome answered "is this path under process.env.HOME?". With test/setup.ts skipped (a bare vitest) HOME is the developer's real home, so every real file passed, which is how the real linked-cases registry was deleted on 2026-10-10. They now also require HOME to be the throwaway codeman-vitest-* directory setup.ts creates, and the shared route harness throws when it is not, so every route test refuses to start against a real ~/.codeman rather than just the one file that checked.
|
Thanks for this, @opticon454. It closes a nasty trap: a bare A few small things I will apply at merge time, so no action is needed unless you want to do them yourself:
Smaller wording items: the Thanks again for tracking down both the root cause and the |
…count Merge-time fix for #660. afterAll runs under vitest's 10 s hookTimeout, and each failed removal attempt costs rmSync's own retry sleeps (50 + 100 + 150 ms) plus the 250 ms wait, so 20 attempts overran it: with a writer still filling $HOME/.claude/backups, the file failed with "Hook timed out in 10000ms" after its test passed, and the "leave it to the exit backstop" branch was never reached. The loop now stops after about 5 s. Checked with a temporary (uncommitted) test that leaves a child process writing there for 20 s: before, the file failed at 11 s with the hook timeout; after, it passes in under 6 s. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ndbox HOME Merge-time fix for #660. CLAUDE.md and architecture-invariants still said a raw npx vitest skips setup.ts, contradicting the updated Testing note: both now say it resolves the root vitest.config.ts (and why that file must stay). The architecture-invariants entry for safeRmHomeTree adds the sandbox-HOME requirement (isSandboxedHome, TEST_HOME_PREFIX), the same rule for isUnderTestHome, the route-harness assert and the 2026-10-10 incident. CLAUDE.md's repo-root preamble lists vitest.config.ts with its reason, so a tidy-up does not move it into config/ and reopen #640. The assertSandboxedHome JSDoc no longer claims every route test is covered: only the ones built on createRouteTestHarness are. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… the isolated CI config Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Merged and shipped in 1.44.0. Thank you @opticon454! This closes a trap that had already cost real state once. The release gate even proved the point: the one file that failed on a branch without your change failed on exactly the At merge I bounded the teardown retry by a deadline of about 5 s instead of an attempt count, so a stuck writer can no longer push |
What
A bare
npx vitest runwrites test fixtures into the developer's real~/.codeman.Why
There is no root vitest config: the configs live in
config/andnpm testpasses--config. A bare invocation therefore has nosetupFiles,test/setup.tsnever runs,HOMEis not redirected and the suite works against the real data dir. On a machine that runs Codeman this left aclis.jsoncontaining a test entry, a fixtureusers.json,webhook.json,intents.json,.env,settings.jsonvalues and a.secret-fixturefile; the running instance loaded them on its next restart, which looks exactly like "my settings were reset".How
vitest.config.tsre-exportsconfig/vitest.ci.config.ts, so the default invocation is the isolated one.npm testand the browser, mobile and perf scripts keep passing their own--config.exitbackstop instead of failing a file that passed. On a machine with a real agent CLI installed, a test's code path probes or spawnsclaude, which flushes its own~/.claude.jsonand~/.claude/backups/into the temp HOME after the test ends;rmSyncthen raced it and failed the whole file withENOTEMPTYafter every test had passed (about one run in three forsession-routes-claude-modelon unmodified master; CI has no such CLI). Twelve consecutive runs of that file now pass.CLAUDE.mdandContributingstate that a barenpx vitest runis isolated now, and what it used to do.Tests
test/test-env-isolation.test.tspins that the root config exists and loadstest/setup.ts. Checked by running a test that used to write into the data dir with a barenpx vitest runand confirming nothing appeared there. Changeset included.Fixes #640