Skip to content

Keep .knowledgebase writes to cairn: file-tool hook and frontmatter check (PROF-16469) - #946

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits into
mainfrom
rkennke/PROF-16469-knowledgebase-guard
Nov 6, 2026
Merged

gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits into
mainfrom
rkennke/PROF-16469-knowledgebase-guard

Conversation

@rkennke

@rkennke rkennke commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?:

Enforces the .knowledgebase/ rule: only the cairn skills write it.

  • Hook: .claude/hooks/protect-knowledgebase.py denies Edit, MultiEdit, Write and NotebookEdit into .knowledgebase/, in any checkout of this repo. It resolves relative, .., ~, symlinked and not-yet-existing paths and matches case-blind. Like the gradlew hook, it loads its policy (utils/knowledgebase-write-policy.py) from origin/main falls back to the checkout's policy when origin/main lacks it (a stale fork as origin), and allows only when neither has it.
  • Pi: .pi/extensions/protect-knowledgebase.ts covers Pi's edit and write tools, including Pi's @ prefix and a session started in a subdirectory.
  • Backstop: utils/check-knowledgebase.py runs in qualityChecks and checks every tracked memory for cairn's required frontmatter. It parses exactly as cairn does.
  • AGENTS.md: the cairn block is now cairn's own template, verbatim, so cairn's session-start rewrite changes nothing. The rule is one paragraph after the block.
  • Review: the pre-pr-review skill tells the reviewer to skip .knowledgebase/ content; the review marker still covers the files.

Motivation:

Part A of PROF-16469, decided as D3: .knowledgebase/ stays tracked; only cairn writes it; new memories travel with the change they came from, in its PR or a stacked one; reviews skip their content and check only the frontmatter.

The old "Never review .knowledgebase/" paragraph sat inside the cairn-managed block. cairn's session-start hook rewrites that block and dropped the paragraph, so committing the rewrite deleted the rule.

Additional Notes:

  • Needs a second profiling reviewer: this changes .claude/settings.json, which affects every engineer's Claude Code session.
  • Why a hook and a CI check: cairn writes memories through its own Python, never through the agent's file tools, so the hook cannot get in its way. The hook does not see shell writes (sed -i, cp); the frontmatter check catches a hand-written memory instead.
  • The check covers all tracked memories, not only the branch's changes. It is cheap and needs no base selection.
  • Accepted limits:
    • on Linux, a distinct directory named, for example, .KnowledgeBase/ is also blocked (case-blind match);
    • the check mirrors cairn's stable required subset (id, title, kind, tags, source, slug and file name, dead-* kind), not the full contract; cairn maintenance validates the rest;
    • the hook and settings.json run from the checkout, so a branch can switch the guard off; it stops accidental agent writes, not a hostile branch;
    • the check is never stricter than cairn's contract, so what cairn memorize accepts (for example tags: [], which cairn writes) passes CI;
    • the trusted-ref loader is now copied in three hooks, which name each other.

How to test the change?:

Check Result
bash utils/tests/test_knowledgebase_guard.sh (A1) passed, 41 tests in node:22-bookworm, Pi cases included
bash utils/tests/test_quality_check_scripts.sh (A2, T7 + T4 wiring) passed, 117 groups
check-knowledgebase.py on all 91 memories on main passed
Mutation check: each hook, policy, adapter and checker branch removed or inverted each fails its own test
A4 test-drive: fresh headless Claude Code session in a scratch clone with origin/main at the candidate, asked to Edit a memory denied by the hook; file unchanged
A4: cairn memorize of a new memory in the same clone installed
AGENTS.md cairn block equals cairn's _kb_agents_block() output passed
Agentic review: /code-review (medium) on the full change 0 Critical/High open; bypasses, false blocks and the crash it found are fixed

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles
    credentials of any kind, I've requested a security review (run the dd:platform-security-review
    skill, or file a request via the PSEC review form).
    bewaire also runs automatically on every PR.
  • This PR doesn't touch any of that.
  • JIRA: PROF-16469

🤖 Generated with Claude Code

rkennke and others added 2 commits November 5, 2026 16:28
.knowledgebase/ is written only by the cairn skills, which install memories
through `cairn memorize`. A PreToolUse hook now denies Edit, MultiEdit,
Write and NotebookEdit into it, in any checkout of this repository and
through relative paths, `..` or symlinks. Like the gradlew hook, it loads
its policy from origin/main and allows everything until the policy is
there. A Pi adapter covers Pi's edit and write tools; Pi has no notebook
tool.

Shell writes bypass the hook, so check-knowledgebase.py, run by
qualityChecks, checks every tracked memory for cairn's required
frontmatter.

AGENTS.md now carries cairn's own block verbatim, so cairn's session-start
rewrite changes nothing, and states the rule in one line after it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…OF-16469)

Guard:
- Pi drops a leading `@` and expands `~` before it writes, and may run in
  a subdirectory; the adapter strips `@`, finds the hook at the checkout's
  top level, and the hook expands `~`.
- The hook and policy match `.knowledgebase` case-blind, as macOS's
  default file system does.

Frontmatter check:
- Parse exactly as cairn does, so a memory cairn accepts is never
  rejected; a tracked file missing from the working tree is reported,
  not a crash; also check cairn's source form and dead-* kind rule.

Memories are cairn's content and travel with the change they came from:
the pre-pr-review skill tells the reviewer to skip .knowledgebase/ and
fixes check-knowledgebase.py violations through cairn; the review marker
still covers the files. AGENTS.md spells out the review skip.

Tests: T7 in the quality suite for check-knowledgebase.py, the
qualityChecks wiring, and a guard case for each gap above.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rkennke
rkennke requested a review from a team as a code owner November 6, 2026 10:12
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-06T10:15:36.063792Z 4ba1557 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4ba1557f42

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread utils/check-knowledgebase.py Resolved
@dd-octo-sts

dd-octo-sts Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Scan-Build Report

User:runner@runnervm7aw5e
Working Directory:/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:make -j4 all
Clang Version:Ubuntu clang version 18.1.3 (1ubuntu1)
Date:Fri Nov 6 14:05:33 2026

Bug Summary

Bug TypeQuantityDisplay?
All Bugs2
Logic error
Dereference of null pointer1
Unused code
Dead initialization1

Reports

Bug Group Bug Type ▾ File Function/Method Line Path Length
Unused codeDead initializationreferenceChains.cppapplyOomRamp4971
Logic errorDereference of null pointerperfEvents_linux.cppunregisterThread72026

@datadog-official datadog-official Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

On case-insensitive filesystems, an absolute memory path with different checkout-root casing bypasses both Claude and Pi guards. The frontmatter check does not catch edits that preserve valid metadata.

Open Bits AI session

🤖 Bits Code Review · Commit 4ba1557

Comment thread .claude/hooks/protect-knowledgebase.py Resolved
@dd-octo-sts

dd-octo-sts Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #42281409207 | Commit: 7c6e89b | Duration: 23m 41s (longest job)

❌ 1 of 32 test jobs failed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - ✅ - -
8-ibm - ✅ - -
8-j9 ✅ ✅ - -
8-librca - - ✅ ✅
8-orcl - ✅ - -
11 - ✅ - -
11-j9 ✅ ✅ - -
11-librca - - ✅ ✅
17 ✅ ✅ - -
17-graal ✅ ✅ - -
17-j9 ✅ ✅ - -
17-librca - - ✅ ❌
21 ✅ ✅ - -
21-graal ✅ ✅ - -
21-librca - - ✅ ✅
25 ✅ ✅ - -
25-graal ✅ ✅ - -
25-librca - - ✅ ✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

❌ Failing tests

Test Cells Message
VtableReceiverFrameTest.testVtableReceiverFrameInCpuSamples___5_ musl-17-librca-debug-amd64 Test execution #5 (of up to 5 with at least 1 successes) failed ~> test fails - see cause for details

Retried 1 of 32 cells.

Failed Jobs

Summary: Total: 32 | Passed: 31 | Failed: 1


Updated: 2026-11-06 20:05:36 UTC

…limiters (PROF-16469)

- On a case-insensitive file system, `/x/REPO/.knowledgebase/m.md`
  reaches the memory under `/x/repo`, but git reports the real spelling,
  so the relative path came out as `../REPO/...` and the policy allowed
  the write. repo_relative() now falls back to a case-blind comparison.
- check-knowledgebase.py accepted `---garbage` or `----` as the opening
  delimiter. Delimiters must now be whole `---` lines, as cairn writes
  them; keys still parse as cairn parses them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dd-octo-sts

dd-octo-sts Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 7bd58c58

@zhengyu123 zhengyu123 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agentic review (Claude Code /code-review): 8 findings, ranked by severity. No crash risks. The first two are gaps in how well the guard protects .knowledgebase/; the rest are checker bugs, cleanup, and AGENTS.md rule issues.

Verification: check-knowledgebase.py passes on this tree (91 memories). utils/tests/test_knowledgebase_guard.sh passes: 45 tests, 0 failed. .knowledgebase/ content was not reviewed.

Comment thread .claude/hooks/protect-knowledgebase.py Resolved
Comment thread .claude/hooks/protect-knowledgebase.py Resolved
Comment thread utils/check-knowledgebase.py Resolved
Comment thread utils/check-knowledgebase.py Resolved
Comment thread utils/check-knowledgebase.py Resolved
Comment thread .claude/hooks/protect-knowledgebase.py Resolved
Comment thread .claude/hooks/protect-knowledgebase.py Resolved
Comment thread utils/check-knowledgebase.py Resolved
…(PROF-16469)

Hook:
- With a stale fork as origin, origin/main lacks the policy and the hook
  allowed every write. It now falls back to the checkout's policy (read
  as UTF-8; an undecodable file is skipped, not a crash) and allows only
  when neither has it.
- The trust comment no longer claims the branch cannot weaken the guard:
  the rule comes from origin/main, but the hook and settings.json run
  from the checkout. The README says the same.
- The three hooks that copy git_out and the trusted-ref loading now name
  each other; only this one falls back to the checkout.

check-knowledgebase.py stays never stricter than cairn's contract:
- trailing whitespace on the `---` lines is accepted, as cairn does; a
  leading BOM is still reported, since cairn finds no frontmatter then;
- `deadend-*` ids with kind finding and quoted empty values pass, as in
  cairn; tests pin that parity;
- files are read as UTF-8; shorter docstrings, the rule list moves to
  problems().

Tests: the checkout-policy fallback and its decode guard, a relative
payload cwd (the checkout_of loop guard, timed out in Python, not GNU
timeout), and T7m/p/q/r/s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@datadog-official

datadog-official Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Pipelines

✨ Unblock PR with BitsAI

❌ Errors

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 3 Pipeline jobs failed

DDCI - DataDog/java-profiler | gtest-release-arm64 — 🔧 Needs a code fix, caused by this PR

View more details · View in DDCI

DataDog/java-profiler | gtest-release-arm64 — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

CI Run | test-matrix / test-linux-glibc-amd64 (21, debug, regular)

View more details · View in GitHub Actions

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: e5a05d0 | Docs | View more details | Give us feedback!

@rkennke

rkennke commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-11-06 16:22:26 UTC ℹ️ Start processing command /merge


2026-11-06 16:22:30 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 53m (p90).


2026-11-06 16:35:16 UTC ❌ MergeQueue: The build pipeline contains failing jobs for this merge request

Build pipeline has failing jobs for 16152b7:

⚠️ Do NOT retry failed jobs directly (why?).

What to do next?

  • Investigate the failures and when ready, re-add your pull request to the queue!
  • If your PR checks are green, try to rebase/merge. It might be because the CI run is a bit old.
  • Any question, go check the FAQ.
Details

Since those jobs are not marked as being allowed to fail, the pipeline will most likely fail.
Therefore, and to allow other builds to be processed, this merge request has been rejected and the pipeline got canceled.

@rkennke

rkennke commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-11-06 16:54:05 UTC ℹ️ Start processing command /merge


2026-11-06 16:54:10 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 53m (p90).


2026-11-06 17:05:34 UTC ❌ MergeQueue: The build pipeline contains failing jobs for this merge request

Build pipeline has failing jobs for c6aed08:

⚠️ Do NOT retry failed jobs directly (why?).

What to do next?

  • Investigate the failures and when ready, re-add your pull request to the queue!
  • If your PR checks are green, try to rebase/merge. It might be because the CI run is a bit old.
  • Any question, go check the FAQ.
Details

Since those jobs are not marked as being allowed to fail, the pipeline will most likely fail.
Therefore, and to allow other builds to be processed, this merge request has been rejected and the pipeline got canceled.

@rkennke

rkennke commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-11-06 19:08:17 UTC ℹ️ Start processing command /merge


2026-11-06 19:08:21 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 53m (p90).


2026-11-06 19:38:30 UTC ℹ️ MergeQueue: This merge request was merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants