Skip to content

test(company-monitoring): add sealed evidence curation gate - #6488

Merged
koala73 merged 1 commit into
mainfrom
codex/issue-6006-blind-corpus-gate
Aug 12, 2026
Merged

koala73 merged 1 commit into
mainfrom
codex/issue-6006-blind-corpus-gate

Conversation

@koala73

@koala73 koala73 commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Summary

Company Monitoring now has a sealed, fail-closed path from genuine public-source curator manifests to opaque pilot, tracer, and Stage 3 corpus artifacts. This extends the merged deterministic scorer and admission policy without changing the frozen classifier policy.

The offline compiler enforces exact provenance, evidence timestamps and precision, private-company proof, custody versions, bounded excerpts, opaque identifiers, and pairwise disjointness. It emits no raw identity, URL, excerpt, or label. Invalid input returns a stable error code without echoing sealed content or paths.

No raw evidence, predictions, or gold labels are committed in this PR.

Corpus-side evidence

The sealed external v3 receipts record genuine public-source collection and remain outside Git.

Measure Count
Official source records scanned 5,014
Normalized research candidates 877
Verified selected evidence bundles 400
Pilot 100
Tracer 100
Stage 3 candidate 200
US 200
UK 200

The selected source distribution is 100 CPSC recall records, 100 openFDA food-enforcement records, 100 Find a Tender award records, and 100 UK Gazette insolvency notices. The v3 audit passes across opaque ID, occurrence, content fingerprint, corporate family, and primary source origin. Its manifest digests remain unchanged after the review fixes.

Recorded exclusions are:

  • 992 FDA workbook rows without an exact notice URL;
  • 31 duplicate CPSC companies and 103 rows beyond the pilot quota;
  • 333 duplicate openFDA product rows within a firm event and 36 rows beyond the tracer quota;
  • 8 duplicate Find a Tender companies, 19 duplicate notices, and 75 rows beyond the Stage 3 quota; and
  • 38 duplicate Gazette companies, 100 rows excluded by the balanced selection, and 67 rows beyond the Stage 3 quota.

Remaining acceptance gate

The v3 manifests and compiled corpora remain draft. They use pending_pinned_openrouter_model. No prediction, gold-label, forecast, freeze, or score artifact exists.

The next blind step needs these runtime settings outside Git:

  • OPENROUTER_API_KEY
  • COMPANY_MONITORING_CLASSIFIER_MODEL
  • EXA_API_KEYS
  • X_BEARER_TOKEN

The exact resolved OpenRouter model and provider route must be approved and pinned. The runtime must also prove zero data retention, no training, disabled reasoning, and disabled activity logging. The repository has no approved offline discovery-plus-classifier prediction runner, so the official curator evidence cannot be substituted as classifier predictions.

After that gate, create immutable v4 manifests, lock the pilot, generate predictions before any gold labels are opened, and run the existing forecast/freeze/scorer sequence. A weak forecast must remain forecast_warning; a post-score denominator gap must remain incomplete.

Test plan

  • 85 Company Monitoring evaluation, blind-scorer, continuation, CLI, and curation tests pass.
  • npm run typecheck passes.
  • npm run typecheck:contract-tests passes with the new compiler, CLI, and tests included.
  • Focused Biome lint and diff checks pass.
  • The repository pre-push gate passes, including scoped typechecks, boundary checks, edge checks, changed tests, and bundle freshness.
  • Independent cross-model code review completed; all five actionable findings were applied and revalidated.

Code review receipt: status: complete, run 20260812-105839-ca880ee8.

Post-Deploy Monitoring & Validation

No additional production runtime monitoring is required because this PR adds an offline curator CLI and contract tests only. No product request path imports the new module.

  • Validation window: before any v4 pilot lock or prediction run.
  • Owner: Company Monitoring curator/release owner.
  • Healthy signal: audit-split returns 100 pilot, 100 tracer, and 200 Stage 3 rows with the retained v3 manifest digests.
  • Failure signals: digest drift, overlap error codes, a non-opaque CLI error, or any raw source artifact in Git.
  • Mitigation: stop the freeze sequence, retain the sealed v3 receipts, and revert this commit if the offline contract cannot be corrected without changing the approved protocol.

Related

Related: #6006

Related: #6002


Compound Engineering

@mintlify

mintlify Bot commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
WorldMonitor 🟢 Ready View Preview Aug 12, 2026, 7:35 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@vercel

vercel Bot commented Aug 12, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
worldmonitor Ready Ready Preview Aug 12, 2026 7:35am

Request Review

@koala73
koala73 merged commit ffec79a into main Aug 12, 2026
35 checks passed
@koala73
koala73 deleted the codex/issue-6006-blind-corpus-gate branch August 12, 2026 09:56

This branch was successfully deployed

2 active deployments
staging - docs — 3804aae0 Deployed Aug 12, 2026 by mintlify[bot]
Preview — 3804aae0 Deployed Aug 12, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant