Skip to content

feat(company-monitoring): add bounded portfolio onboarding - #6232

Merged
koala73 merged 9 commits into
mainfrom
codex/issue-6005-company-monitoring-onboarding
Aug 9, 2026
Merged

koala73 merged 9 commits into
mainfrom
codex/issue-6005-company-monitoring-onboarding

Conversation

@koala73

@koala73 koala73 commented Aug 5, 2026 •

Copy link
Copy Markdown
Owner

Summary

This is the account and onboarding slice after the merged viability and dark-service foundations. Company Monitoring can now maintain exactly one entitlement-bound account root and a bounded portfolio without activating any public product surface. Later corpus, scorer, provider, and runtime stages can build on durable account, company, claim, replay, removal, and purge semantics.

The implementation keeps bulk onboarding state-free. It does not add import-job or transfer state. Each normalized row is committed in its own transaction, and account-local replay metadata makes uncertain retries deterministic while rejected and no-op rows remain eligible for reevaluation.

Guarantee Behavior
Entitlement lifecycle The first authenticated Company Monitoring feature use provisions the root from canonical entitlement state. Billing writes stay independent. An hourly bounded reconciler converges existing roots after lapses and renewals.
Account safety Owner fences prevent deleted accounts from being recreated. A lapsed account receives an explicit 24-hour grace and can reactivate only before destructive purge starts. A dedicated fence secret and its previous-key ring keep terminal fences discoverable during key rotation.
Portfolio bounds Direct creates and imports share one atomic 500-company cap. Imports accept at most 100 normalized rows and preserve count parity under concurrency.
Replay Committed rows return their original result, changed retries conflict, and rejected or no-op rows reevaluate against current state. Individual and account purge paths scrub all five replay identity and fingerprint fields.
Removal and purge Removal hides once, decrements once, clears claims, increments the snapshot once, and schedules resumable bounded payload purge work. An hourly bounded reaper recovers stalled purge phases.
API keys Optional scopes and account binding are validated against the live account lifecycle. Scoped keys are rejected by unrelated generic API and MCP validators, and lifecycle changes invalidate the production cache namespace.

All Company Monitoring feature flags and public routes remain dark. This PR does not add blind-corpus or scorer work, provider imports, runtime monitoring, import-job state, or transfer state.

Fixes #6005

Related: #6002, #6003, #6004

Deployment gate

  • Provision COMPANY_MONITORING_OWNER_FENCE_SECRET in the production Convex environment before merge and deployment. There is intentionally no fallback to the Dodo identity-signing secret.
  • For a later rotation, pre-stage the current secret in COMPANY_MONITORING_OWNER_FENCE_PREVIOUS_SECRETS, deploy that keyring, then replace the current secret. Duplicate previous entries fail closed.
  • Keep all Company Monitoring public feature flags dark; this persistence slice does not authorize runtime or provider activation.

Review hardening

  • Removed Company Monitoring synchronization from billing writes. Lazy feature access provisions from stored entitlement state, and the hourly reconciler converges existing roots. Owner-fence configuration errors now fail Company Monitoring operations closed without rolling back independent entitlement writes.
  • Made the reconciliation budget fair across entitled and lapsed roots, and recovered the one nonterminal owner root when old fence history was dropped.
  • Added the explicit 24-hour ordinary-lapse grace, terminal-deletion bypass, bounded hourly purge reaper, exact replay-metadata scrub, and a final entitlement recheck before reactivation.
  • Added explicit Convex argument validators, removed five unused indexes, retained only the protected terminal owner-deletion operation, and added the single reaper index.
  • Corrected Convex cache invalidation to target the production cache namespace and added parity coverage for the self-contained Edge API-key scope mirror.

Type of change

  • Bug fix
  • New feature
  • New data source / feed
  • New map layer
  • Refactor / code cleanup
  • Documentation
  • CI / Build / Infrastructure

Affected areas

  • Map / Globe
  • News panels / RSS feeds
  • AI Insights / World Brief
  • Market Radar / Crypto
  • Desktop app (Tauri)
  • API endpoints (/api/*)
  • Config / Settings
  • Other: Convex persistence, entitlement lifecycle, onboarding, purge, and API-key authorization

Checklist

  • Tested on worldmonitor.app variant — N/A; no public route or enabled flag exists in this slice
  • Tested on tech.worldmonitor.app variant (if applicable) — N/A; no public route or enabled flag exists in this slice
  • New RSS feed domains added to api/rss-proxy.js allowlist (if adding feeds) — N/A
  • No API keys or secrets committed
  • TypeScript compiles without errors (npm run typecheck)

Documentation Alignment Checklist

  • Claim ledger attached or linked — N/A; this PR does not publish documentation claims
  • All required Audit Council role signoffs attached — N/A
  • Generated docs regenerated from proto where applicable — N/A; no proto or generated documentation changed
  • Fixture-backed examples recomputed — N/A
  • Redis writers/readers enumerated for every documented key — N/A; no Redis key documentation changed

Validation

  • Final focused Convex verification: 5 files and 232 tests passed after the last rebase onto origin/main.
  • Company Monitoring contract and Edge API-key verification: 82 tests passed. Server API-key validation: 33 tests passed.
  • npm run typecheck, npm run typecheck:api, Convex typecheck, contract-test typecheck, and repository lint passed. Repository lint reported only pre-existing warnings outside this change.
  • The final pre-push gate passed frontend, API, and Convex typechecks; architectural and policy checks; edge bundling; 107 server tests; 28 changed contract tests; 249 Edge tests; and version sync.
  • Full Compound Engineering review found six in-scope lifecycle, replay, scheduling, typing, and environment-contract gaps. All six were repaired. Independent cross-model review and validator passes confirmed the material findings.

Post-Deploy Monitoring & Validation

  • Watch Convex mutation, action, and scheduled-function failures, Upstash pipeline failures, account-root cardinality, company-count parity, and purge cursors for 24 hours after deployment.
  • Inspect [subscriptionHelpers], [cacheActions], COMPANY_MONITORING_ACCESS_DENIED, REPLAY_CONFLICT, and COMPANY_LIMIT_REACHED events.
  • Healthy means no duplicate roots, count drift, account-binding mismatch, stale scoped-key cache entries, or stuck purge generation. Public Company Monitoring traffic must remain absent while flags stay dark.
  • Duplicate roots, count drift, cross-account access, stale authorization after lifecycle change, or non-advancing purge cursors are rollback triggers. Keep all flags dark, revert the deployment, and repair affected rows before retrying.
  • Validation window: 24 hours after deployment. Owner: WorldMonitor maintainers.

Screenshots

N/A. This slice adds no public UI or enabled runtime surface.


Compound Engineering

@vercel

vercel Bot commented Aug 5, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
worldmonitor Ready Ready Preview Aug 9, 2026 11:33am

Request Review

koala73 added 9 commits August 9, 2026 15:27
COMPANY_MONITORING_OWNER_FENCE_SECRET is a hard requirement at every
entitlement write — companyMonitoringOwnerFenceCandidates throws when it
is unset, and that call sits inside upsertEntitlements — but it was
documented nowhere. Its sibling DODO_IDENTITY_SIGNING_SECRET has carried
an .env.example entry since it landed, so deploying this branch without
the new var set in the Convex dashboard would throw on every Dodo webhook
with nothing pointing an operator at the cause.

Also documents COMPANY_MONITORING_OWNER_FENCE_PREVIOUS_SECRETS, including
the ordering constraint the rotation comment in identitySigning.ts states
but no operator-facing file did: append the current value and deploy
BEFORE rotating, because a dropped historical key orphans the tombstones
signed with it.

Both are Convex-runtime variables. .env.example is documentation; the
Convex deployment env is what actually satisfies the check.

Claude-Session: https://claude.ai/code/session_015TeSnvgRFWNVPk3kaY6prH
…ling

syncCompanyMonitoringAccountFromEntitlement runs unguarded inside
upsertEntitlements — the one entitlement-write path — and reaches
companyMonitoringOwnerFenceCandidates, which throws for pure configuration
reasons: an unset or whitespace-padded COMPANY_MONITORING_OWNER_FENCE_SECRET,
or a malformed COMPANY_MONITORING_OWNER_FENCE_PREVIOUS_SECRETS keyring.

Convex mutations are atomic, so that throw rolled back the subscription
row, the entitlement row, and the webhook idempotency record, and the
handler returned 500. Dodo retries, but a config fault is not transient —
it fails identically forever, so a dark feature's env typo stopped billing
for every user until a human edited a variable.

Fixed by making the config path fail by RETURN rather than by THROW:

- identitySigning.ts gains tryCompanyMonitoringOwnerFenceCandidates, a
  non-throwing wrapper. The try/catch deliberately lives there, around a
  PURE function, not at the call site: Convex has no savepoints, so
  catching around code that has already written would commit that partial
  state.
- accounts.ts resolves the fence BEFORE canonicalEntitlement, which
  performs the transaction's first write (its same-value serialization
  patch). Bailing there commits nothing at all. A misconfigured keyring now
  logs and skips the root; the next entitlement write creates it, and
  syncStoredEntitlement remains as a manual repair seam.

Deliberately unchanged: ACCOUNT_OWNER_FENCE_CONFLICT and
ACCOUNT_OWNER_BINDING_MISMATCH still throw and still abort the write —
those are genuine data-integrity signals, not config typos. terminalize
keeps the throwing accessor, so explicit owner/account deletion still
fails loudly off the billing path.

The previous test asserted the opposite ("fail atomically", entitlements
left empty) and is inverted here, since it encoded the behaviour that made
a missing env var a billing outage.

Verification:
- npm run test:convex — 68 files, 1319 tests, all pass
- mutation: restoring the throwing call turns exactly the 3 new tests red
  and leaves the other 22 green, including the owner-deletion test that
  proves terminalize was not weakened
- npm run typecheck:api — passes, incl. the Convex export audit
- biome check on all four changed files — clean

Claude-Session: https://claude.ai/code/session_015TeSnvgRFWNVPk3kaY6prH
Company Monitoring is a segment feature, but syncCompanyMonitoringAccountFromEntitlement
ran from upsertEntitlements — the one entitlement-write path every subscriber
traverses. That charged all of them for its queries, an extra entitlements
write, and 1+N HMACs, and let any fault inside it roll back a paying customer's
entitlement. It also could not short-circuit for non-users: a deleted-owner
tombstone is discoverable only by fence hash, so the fence had to be computed
before the code could tell whether the caller was relevant at all.

Two properties showed the coupling bought nothing it looked like it bought:
activeAccountForOwner re-derives entitlement state live on every read, so the
root is a cache the read path does not trust; and the 24h ORDINARY_LAPSE_PURGE_GRACE_MS
discards the immediacy, so a scheduled scan is equivalent.

Provisioning is now lazy and reconciliation is pull-based:

- The three billing call sites are gone. Nothing under convex/payments/ imports
  Company Monitoring.
- ensureActiveAccount / requireProvisionedAccount get-or-create the root at the
  three genuine first-use entry points (createApiKey with scopes,
  createCompanyForOwner, importCompanyRowForOwner). activeAccountForOwner stays
  the read-only resolver because apiKeys.validateKeyByHash is an internalQuery
  and Convex queries cannot write. update/setState/list keep the read-only path
  — a company cannot exist without a root.
- reconcileAccountEntitlements (new hourly cron) scans entitled roots
  oldest-first via a new by_lifecycle_updatedAt index and re-derives each
  owner's entitlement, driving the same state machine a lapse used to. It scans
  companyMonitoringAccounts, so users who never touch the feature cost nothing.
- The fence-degrade stopgap (tryCompanyMonitoringOwnerFenceCandidates) is
  removed. It existed only to stop a config fault aborting billing; on a genuine
  Company Monitoring path a missing fence secret should fail loudly, and
  terminalize keeps the throwing accessor.

Note the new index is the by_lifecycle the earlier review flagged as dead,
re-added with the consumer that justifies it.

Behaviour deliberately unchanged: exactly one root per owner, anonymous
purchases provision nothing, terminal tombstones still refuse re-provisioning,
the 500-cap and replay semantics, and the 24h grace before destructive purge.
#6005 acceptance criterion 1 was amended to match.

Verification:
- npm run test:convex — 68 files, 1322 tests, all pass
- mutation 1: re-adding the sync call to upsertEntitlements turns the two
  decoupling tests plus the dispute test red
- mutation 2: making the reconciler skip the sync turns the reconciler-lapse and
  dispute tests red
- mutation 3: dropping ensureActiveAccount's re-resolve turns exactly one test
  red — "first use cannot resurrect a deleted owner" — so the tombstone guard is
  isolated, not incidentally covered
- npm run typecheck:api — passes, incl. the Convex export audit
- biome check on all 12 changed source files — clean

The decoupling test proves the absence by ablation rather than by asserting a
negative: with COMPANY_MONITORING_OWNER_FENCE_SECRET deleted, any Company
Monitoring work on the entitlement path would throw, so the grant and the signed
Dodo webhook both succeeding is positive evidence they never reached it.

Closes #6256

Claude-Session: https://claude.ai/code/session_015TeSnvgRFWNVPk3kaY6prH
Moving lapse detection to an hourly reconciler (#6256) opens a window where a
root still reads lifecycle "entitled" after the owner's entitlement expired.
That window is only acceptable because activeAccountForOwner re-derives the
entitlement live on every call, so it delays purge SCHEDULING and never grants
access. Nothing asserted that.

Adds a test that expires an entitlement without reconciling, confirms the root
is still stale-"entitled", and then requires every surface to refuse: a company
write, a company read, and scoped key issuance.

Mutation: replacing activeAccountForOwner's live check with a bare
`Boolean(entitlement)` — i.e. trusting the stale root — leaves the entire rest
of the suite green and turns ONLY this test red. It was the single uncovered
path between the refactor and an access-control hole.

Claude-Session: https://claude.ai/code/session_015TeSnvgRFWNVPk3kaY6prH
)

Review caught a P0 I introduced in the lazy-provisioning commit, from two
independent lenses. Removing the entitlement-write push made lapse detection
one-directional: reconcileAccountEntitlements scanned only lifecycle "entitled",
so it could lapse a root but never restore one — and the billing-side call that
used to restore it is exactly what that commit deleted.

The consequence is the failure ORDINARY_LAPSE_PURGE_GRACE_MS exists to prevent.
Its own comment says the 24h window is there so "a lost successful renewal
webhook gets one authoritative Dodo sweep before customer data is scrubbed", and
that sweep's only wire into Company Monitoring was the deleted call. A customer
whose renewal landed late was locked out of a feature they had paid for, and 24h
later their portfolio was destructively scrubbed while their entitlement was
active.

Three fixes, all in the reconcile/purge path:

- advanceAccountPurge's `pending` branch now re-derives the owner's entitlement
  before starting destruction and reactivates instead, returning "reactivated".
  The `finalizing` branch already did this recheck; it just ran after the
  payload was gone.
- The reconciler scans RECONCILABLE_LIFECYCLES — "entitled" AND
  "entitlement_lapsed" — sharing one batch budget, the same shape
  reapStalledAccountPurges uses for REAPABLE_PURGE_PHASES. The sync was already
  correct in both directions; only the scan predicate excluded restores.
- Per-row work is now scheduled rather than run inline. The cron was one
  transaction over 50 rows, so a single throwing row (a fence conflict is
  reachable) rolled back every cursor advance and would stall lapse detection
  fleet-wide, permanently. The cursor advance also moved before the skip, so a
  skipped row cannot hold the head of the index forever.

Also from review: a first-use test for createApiKey's provisioning path, a
staleness-boundary test (deleting the .lt("updatedAt", staleBefore) clause
previously left the suite green), a batch-cap test, and a corrected comment that
claimed the webhook payload was unsigned when it is signed.

Verification:
- npm run test:convex — 68 files, 1328 tests, all pass
- mutation: reverting RECONCILABLE_LIFECYCLES to ["entitled"] turns exactly the
  renewal-restores test red
- mutation: removing the purge-time recheck turns exactly the
  purge-refuses-for-a-paying-owner test red
- npm run typecheck:api — passes, incl. the Convex export audit
- biome clean

Claude-Session: https://claude.ai/code/session_015TeSnvgRFWNVPk3kaY6prH
@koala73
koala73 force-pushed the codex/issue-6005-company-monitoring-onboarding branch from 489d7c9 to dd6b280 Compare August 9, 2026 11:28
@koala73
koala73 merged commit 5605edc into main Aug 9, 2026
31 checks passed
@koala73
koala73 deleted the codex/issue-6005-company-monitoring-onboarding branch August 9, 2026 16:15
koala73 added a commit that referenced this pull request Aug 9, 2026
…loy (#6232 follow-up) (#6373)

`convex deploy` has failed on main since 5605edc:

    Failed to analyze __tests__/companyMonitoringTestHelpers.js:
    Uncaught TypeError: import.meta unsupported

The Convex bundler skips a file only when its base name holds two or more dots
(node_modules/convex/dist/cjs/bundler/index.js:370). Every other file in
convex/__tests__/ is `*.test.ts` and gets that skip by accident.
`companyMonitoringTestHelpers.ts` has one dot, so Convex pushed it as a backend
module and evaluated its `import.meta.glob` in an isolate that has no
`import.meta` — which fails the WHOLE push, not just that module.

Consequence: none of #6232 reached Convex production. `convex/companyMonitoring/*`,
the `convex/schema.ts` change, the new crons and the `apiKeys.ts` /
`identitySigning.ts` changes are in main and absent from prod, and the next merge
touching convex/ would have failed the same way. Neither `Convex Deploy` nor
`Deploy Railway Reconcile Control` is one of the deploy gate's 20 required
contexts, so main stayed green through all of it.

- Rename the helper to `companyMonitoring.helpers.ts` (two dots) so Convex skips
  it. Same directory, so `import.meta.glob("../**/*.ts")` and the `../schema` /
  `../_generated/api` imports are unchanged, and it still does not match the
  vitest include `convex/__tests__/**/*.test.ts`.
- Rename `setup.ts` to `vitest.setup.ts` for the same reason: it is pushed to
  the backend today and only survives because it holds no `import.meta`. It
  imported `vitest` into the production bundle.
- Add tests/convex-entrypoint-hygiene.test.mjs, which reproduces Convex's own
  entry-point filter and asserts that no pushed module uses `import.meta` and
  that nothing under convex/__tests__ is pushed at all. It runs in `test:data`,
  so the gate-required `unit` job enforces it. Proven red on the pre-rename tree
  naming both files, green after.

npm run test:convex: 69 files, 1404 tests, all pass.

Claude-Session: https://claude.ai/code/session_014iaXRJN9n59i1ZDTd3JLy2
koala73 added a commit that referenced this pull request Aug 10, 2026
Convex Deploy and Deploy Railway Reconcile Control run on push to main and
are outside the deploy gate's required list — they are post-merge, so they
cannot gate the PR that causes them. Both failed on main while main stayed
green: Convex Deploy on #6232 (import.meta unsupported, fixed by #6373)
and the reconcile Worker deploy on #6325/#6326 (missing secrets).

Add a scheduled monitor that reads each push-to-main deployer's run history
on main and fails loudly when the newest run did not deploy. A time-bounded
scan of run history instead of a workflow_run event: the event fires only on
completion, so a workflow that stopped running at all produces no event.

Verdicts, all fail-closed:
- run failure/cancelled/timed_out/skipped -> alarm
- run success but deploy job skipped -> alarm unless it is Convex Deploy
  and the head diff provably touches nothing under convex/ (proven via
  git diff --name-only against the head's parent, with the workflow
  fetching main first so the head exists locally)
- run success but deploy job failed/missing -> alarm
- no completed run in the per-workflow window -> alarm

Monitored: convex-deploy.yml, deploy-railway-reconcile-control.yml,
deploy-worker.yml (path-filtered, no gate anywhere; a missing
CLOUDFLARE_API_TOKEN fails it silently like the reconcile Worker's
missing secrets did).

Wired as .github/workflows/postmerge-deploy-monitor.yml, every 10 minutes,
deliberately NOT gated on green main (the incidents had green main while
the deploy failed) and NOT a deploy-gate status (post-merge; a status on
main is noise). No Slack/Telegram plumbing exists in this repo — the red
scheduled run is the alarm channel, and the verdict logic is pinned by
unit tests plus a live fault-injection check.

Verified live against production 2026-08-10: all three workflows read
green (convex legit-skip, reconcile deployed, worker deployed); fault
injection of a failing convex run produces ALARM/RUN_FAILED (exit 1).
koala73 added a commit that referenced this pull request Aug 10, 2026
* ops(ci): alarm on failed post-merge production deploys (#6376)

Convex Deploy and Deploy Railway Reconcile Control run on push to main and
are outside the deploy gate's required list — they are post-merge, so they
cannot gate the PR that causes them. Both failed on main while main stayed
green: Convex Deploy on #6232 (import.meta unsupported, fixed by #6373)
and the reconcile Worker deploy on #6325/#6326 (missing secrets).

Add a scheduled monitor that reads each push-to-main deployer's run history
on main and fails loudly when the newest run did not deploy. A time-bounded
scan of run history instead of a workflow_run event: the event fires only on
completion, so a workflow that stopped running at all produces no event.

Verdicts, all fail-closed:
- run failure/cancelled/timed_out/skipped -> alarm
- run success but deploy job skipped -> alarm unless it is Convex Deploy
  and the head diff provably touches nothing under convex/ (proven via
  git diff --name-only against the head's parent, with the workflow
  fetching main first so the head exists locally)
- run success but deploy job failed/missing -> alarm
- no completed run in the per-workflow window -> alarm

Monitored: convex-deploy.yml, deploy-railway-reconcile-control.yml,
deploy-worker.yml (path-filtered, no gate anywhere; a missing
CLOUDFLARE_API_TOKEN fails it silently like the reconcile Worker's
missing secrets did).

Wired as .github/workflows/postmerge-deploy-monitor.yml, every 10 minutes,
deliberately NOT gated on green main (the incidents had green main while
the deploy failed) and NOT a deploy-gate status (post-merge; a status on
main is noise). No Slack/Telegram plumbing exists in this repo — the red
scheduled run is the alarm channel, and the verdict logic is pinned by
unit tests plus a live fault-injection check.

Verified live against production 2026-08-10: all three workflows read
green (convex legit-skip, reconcile deployed, worker deployed); fault
injection of a failing convex run produces ALARM/RUN_FAILED (exit 1).

* docs(ci): register the post-merge deploy monitor in the CI/CD table

docs-stats --check (wired into unit) requires every .github/workflows/*.yml
to be listed in ARCHITECTURE.md's CI/CD table. The new monitor bumped
workflowCount 31->32 and its own entry was missing, failing 'exits 0 end
to end on the real repo' in docs-stats-bootstrap-cache /
docs-stats-category-explainer / docs-stats-plan-layer-entitlement.

Regenerate docs/generated/stats.json and add the workflow row next to the
other post-merge monitors.
koala73 added a commit that referenced this pull request Aug 12, 2026
#6480)

`convex/companyMonitoring/admission-snapshot.ts` shipped in #6470 and took
the production Convex deploy down on main:

    InvalidConfig: companyMonitoring/admission-snapshot.js is not a valid
    path to a Convex module. Path component admission-snapshot.js can only
    contain alphanumeric characters, underscores, or periods.

`evaluate_push` validates before pushing anything, so this one filename
rejects the WHOLE deploy — every unrelated convex/ change merged after it
stays out of production, the same blast radius as #6232.

The convex npm package does not check the rule locally; the only "no"
comes from the server, post-merge, where nothing gates. So the rename is
paired with the pre-merge guard that reproduces the rule offline, in the
existing gate-required hygiene test rather than a new workflow.

Claude-Session: https://claude.ai/code/session_019eQ94CDZi5VAfJBh5xzCpE

This branch was successfully deployed

1 active deployment
Preview — dd6b2805 Deployed Aug 9, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(company-monitoring): provision account roots and bounded portfolio onboarding

1 participant